Repository navigation
V2 chat: inline audio and video players, image cards, and rich agent-posted group messages #1611
Copy link
Copy link
Open
Labels
P2Priority 2: important, scheduled after P0/P1Priority 2: important, scheduled after P0/P1enhancementNew feature or requestNew feature or request
Description
Summary
Agents and actions increasingly return media: call recordings, short camera clips, photos and document scans, usually as short-lived signed links from an OpenAPI action. The React V2 chat does not present these well:
- Audio and video links render as plain links that open the file on a new page, so listening to a recording means leaving the conversation.
- The content security policy only allows
media-src 'self' blob:, so audio and video served from an action's own host cannot play inline even if a player were added. - When an agent posts into a group (collaboration) conversation with the SimpleChat action (
add_conversation_message), the message is stored as an ordinary user message and V2 renders it as plain text. Headings, tables, links and images in an agent-written briefing show up as raw markdown.
User Value
People reviewing an agent's findings can listen to recordings, watch clips and see photos in place, and a briefing an agent posts to a group reads the same way as the agent's own reply.
Proposed Behavior
- Inline audio player. A markdown link whose URL path ends in an audio extension renders as a compact bar with play/pause, the title and elapsed/total time. Expanding it adds a seek slider, stop, volume and mute, and playback speed. Starting one recording pauses any other. Keyboard accessible, with ARIA labels.
- Inline video cards. Links to video files render as a card with native controls (play, seek, volume, fullscreen) and the link text as the caption.
- Image cards. Markdown images render as captioned cards that open in the existing image lightbox.
- Rich agent-posted group messages. Messages created through
add_conversation_messagecarry metadata marking them as agent-authored markdown, and V2 renders them with the same XSS-safe markdown renderer used for assistant replies (no raw HTML). Messages typed by people stay plain text. - Media source allowlist. An administrator-configured list of external
https://origins is appended to the CSPmedia-srcdirective. Entries are validated (scheme, host, optional leading wildcard label and port only; no paths, quotes or semicolons) so the setting cannot inject other directives.
Acceptance Criteria
- Audio links in assistant replies render as the inline player; play/pause, stop, seek, volume/mute and speed work, and only one recording plays at a time.
- Video links render as inline video cards, and image markdown renders as captioned cards with click-to-enlarge.
- Media that fails to load falls back to a normal link.
- Agent-posted group conversation messages render as markdown; messages typed by people are unchanged.
- External media plays only from configured origins; invalid allowlist entries are rejected and logged.
- Functional tests cover link classification, CSP allowlist validation and the agent-message metadata; docs and release notes are updated.
Notes
- Related: Inline image/video gallery cards: collapse to a compact header after render, with a drawer toggle and pagination #1349 (classic UI gallery cards for cited workspace media).
- Interactive maps in V2 are tracked separately in Render interactive Azure Maps results in the V2 chat (pan, zoom, point details) #1609.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P2Priority 2: important, scheduled after P0/P1Priority 2: important, scheduled after P0/P1enhancementNew feature or requestNew feature or request
Type
Projects
- StatusShow more project fieldsIn progress
Activity
paullizer commentedon Oct 2, 2026
Status update: implemented in #1612 (version 0.261.222), stacked on #1608.
What changed
add_conversation_message(personal and shared) are markedposted_via: agent_action,content_format: markdown, and V2 renders only those as markdown, labeled "posted through an agent".CSP_MEDIA_SRC_ORIGINSapp setting adds validated barehttps://origins tomedia-src; other entries are rejected and logged.Validation
test_v2_inline_media_players.py(including the Node run oftest_v2_inline_media_logic.mjs),test_v2_rich_rendering.py,test_v2_message_masking.py,test_v2_citations.py: 37 passed. Docs tests: 13 passed.CSP_MEDIA_SRC_ORIGINS.Follow-ups
Play audio and video inline in V2 chat, show images as cards, and ren…
Add release notes for 0.261.222