feat(live): persist media frames alongside audio chunks in save_live_blob - #7062
Open
copybara-service[bot] wants to merge 1 commit into
Open
feat(live): persist media frames alongside audio chunks in save_live_blob#7062copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
copybara-service
Bot
force-pushed
the
copybara/977664760
branch
3 times, most recently
from
September 8, 2026 22:25
97b682c to
cc51c25
Compare
…blob
Previously, `save_live_blob=True` on `RunConfig` only persisted audio streams via `AudioCacheManager` and `AudioCacheConfig`. Any video or media frames streamed during live sessions (e.g. camera feeds or screen captures) were dropped and unrecoverable from session artifacts.
This change upgrades the persistence subsystem to support arbitrary media frames alongside audio chunks, replacing `AudioCacheManager` with a unified `CacheManager` architecture.
Key Architectural Changes:
1. Unified Cache Engine (`live/_cache_manager.py`):
- Replaces `AudioCacheManager` and `AudioCacheConfig` with `CacheManager` and `CacheConfig`.
- Adds dual caching queues (audio chunks and media frames) for both input and output streams.
- Implements MIME-type routing (`audio/*` vs `image/*`/`video/*`).
- Implements FIFO eviction policies enforcing both frame count limits (`max_media_cache_frames=600`) and byte limits (`max_media_cache_size_bytes=100MB`).
- Supports concurrent cache flushing across all active channels via `asyncio.gather(..., return_exceptions=True)`.
2. Artifact Persistence (`artifacts/`):
- Adds concrete `save_media_frames()` to `BaseArtifactService` (raising `NotImplementedError` by default to maintain backward compatibility with custom external implementations) and implements it across `FileArtifactService`, `InMemoryArtifactService`, `GcsArtifactService`, `_ForwardingArtifactService`, and `PerAgentFileArtifactService`.
- Layout: writes individual frames as `frames/frame_{index:04d}.{ext}`, generates a structured `metadata.json` (timestamps, FPS, duration, frame list), and links `frame_0000` for preview compatibility with `load_artifact()`.
- Uses atomic file writes for directory creation and metadata updates.
3. Flow Integration (`flows/llm_flows/`):
- Updates `BaseLlmFlow` and `_LiveLlmFlow` to route incoming and outgoing media blobs to the appropriate cache queues.
- Handles 4-channel concurrent flush on session teardown or turn completion (input audio, input media, output audio, output media).
4. Backward Compatibility:
- `AudioCacheManager` and `AudioCacheConfig` are replaced by `CacheManager` and `CacheConfig`.
- Maintained a deprecated `audio_cache_manager` property on `BaseLlmFlow` returning `self.cache_manager` for backward compatibility with callers and public API checks.
5. Documentation:
- Added `CacheManager` developer guide in `docs/guides/live/cache_manager/` and updated developer guides in `docs/guides/artifacts/artifact_service/` and `docs/guides/live/live_request_queue/` with multimodal persistence and streaming examples.
Testing:
- Added comprehensive unit tests in `tests/unittests/live/test_cache_manager.py` (24 tests covering model validation, configuration, blob routing, FIFO eviction, and concurrent flushes).
- Added multi-backend artifact tests in `test_artifact_service.py` (file, in-memory, GCS).
- Verified `test_live_llm_flow.py`, `test_base_llm_flow_realtime.py`, `test_runners.py`, and `test_forwarding_artifact_service.py`.
- Verified local storage and CLI tests in `test_local_storage.py`, `test_service_factory.py`, `test_cli.py`, and `test_adk_web_server_tests.py`.
- Full subpackage test suite passing.
- All precommit formatters and linters passed cleanly (`run_precommit.sh`).
PiperOrigin-RevId: 977664760
copybara-service
Bot
force-pushed
the
copybara/977664760
branch
from
September 8, 2026 22:42
cc51c25 to
2611108
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(live): persist media frames alongside audio chunks in save_live_blob
Previously,
save_live_blob=TrueonRunConfigonly persisted audio streams viaAudioCacheManagerandAudioCacheConfig. Any video or media frames streamed during live sessions (e.g. camera feeds or screen captures) were dropped and unrecoverable from session artifacts.This change upgrades the persistence subsystem to support arbitrary media frames alongside audio chunks, replacing
AudioCacheManagerwith a unifiedCacheManagerarchitecture.Key Architectural Changes:
Unified Cache Engine (
live/_cache_manager.py):AudioCacheManagerandAudioCacheConfigwithCacheManagerandCacheConfig.audio/*vsimage/*/video/*).max_media_cache_frames=600) and byte limits (max_media_cache_size_bytes=100MB).asyncio.gather(..., return_exceptions=True).Artifact Persistence (
artifacts/):save_media_frames()toBaseArtifactService(raisingNotImplementedErrorby default to maintain backward compatibility with custom external implementations) and implements it acrossFileArtifactService,InMemoryArtifactService,GcsArtifactService,_ForwardingArtifactService, andPerAgentFileArtifactService.frames/frame_{index:04d}.{ext}, generates a structuredmetadata.json(timestamps, FPS, duration, frame list), and linksframe_0000for preview compatibility withload_artifact().Flow Integration (
flows/llm_flows/):BaseLlmFlowand_LiveLlmFlowto route incoming and outgoing media blobs to the appropriate cache queues.Backward Compatibility:
AudioCacheManagerandAudioCacheConfigare replaced byCacheManagerandCacheConfig.audio_cache_managerproperty onBaseLlmFlowreturningself.cache_managerfor backward compatibility with callers and public API checks.Documentation:
CacheManagerdeveloper guide indocs/guides/live/cache_manager/and updated developer guides indocs/guides/artifacts/artifact_service/anddocs/guides/live/live_request_queue/with multimodal persistence and streaming examples.Testing:
tests/unittests/live/test_cache_manager.py(24 tests covering model validation, configuration, blob routing, FIFO eviction, and concurrent flushes).test_artifact_service.py(file, in-memory, GCS).test_live_llm_flow.py,test_base_llm_flow_realtime.py,test_runners.py, andtest_forwarding_artifact_service.py.test_local_storage.py,test_service_factory.py,test_cli.py, andtest_adk_web_server_tests.py.run_precommit.sh).