V2 chat: inline media players, image cards, agent-posted markdown, and interactive maps (0.261.224)

[Paul Lizer (paullizer)]

Summary

Plays audio and video inline in the V2 chat, shows markdown images as captioned cards, renders messages an agent posts through the Simple Chat action as markdown, and draws interactive maps from Azure Maps action results. Adds an optional, validated allowlist of external media origins to the CSP media-src directive.

Refs #1611 and #1609. Stacked on #1608 (paullizer-react-v2-fix-group-upload-call-agent).

Changes

• Inline audio player (InlineAudioPlayer.tsx): a markdown link whose URL path ends in an audio extension renders as an expanding bar with play/pause, title and elapsed/total time; expanded, it adds a seek slider, stop, volume and mute, playback speed and an open-in-new-tab link. Starting one recording pauses any other.
• Inline video cards (InlineVideoCard.tsx): native controls (play, seek, volume, fullscreen) with a caption.
• Image cards (InlineImageCard.tsx): markdown images become captioned cards that open the existing lightbox (rendered through a portal).
• Fallback (InlineMediaFallback.tsx): media that cannot play shows a short notice and a link.
• Renderer wiring (AssistantMarkdown.tsx, lib/inlineMedia.ts): new a and img components. Raw HTML stays off and react-markdown's default URL transform is unchanged; media sources and links pass through safeMediaUrl / safeMarkdownHref.
• Agent-posted messages: add_conversation_message_for_current_user marks personal and shared messages with posted_via: agent_action and content_format: markdown; V2 renders only those as markdown in the neutral bubble and labels them "posted through an agent". Messages people type stay plain text.
• CSP: CSP_MEDIA_SRC_ORIGINS (new csp_media_sources.py) appends validated bare https:// origins to media-src; anything else is rejected and logged.
• Version 0.261.222.

Validation

• functional_tests/test_v2_inline_media_players.py (7 checks, including the Node run of test_v2_inline_media_logic.mjs): passed.
• test_v2_rich_rendering.py, test_v2_message_masking.py, test_v2_citations.py: passed (37 total with the new test).
• test_simplechat_conversation_messages.py updated for the new keyword and marker; it needs the Cosmos test environment, as before.
• XSS sink checker on the added lines: passed. V2 tsc --noEmit: passed.
• Deployed to a SimpleChat POC deployment and exercised end to end with an agent that returns signed media links.

Docs

• Feature doc: docs/explanation/features/V2_INLINE_MEDIA_AND_AGENT_MESSAGES.md.
• Guides: create-an-action.md (media an action returns, CSP_MEDIA_SRC_ORIGINS), collaborate-in-a-conversation.md (messages an agent posts for you).
• REACT_V2_UI.md CSP note.
• Maps: docs/explanation/features/V2_INTERACTIVE_MAPS.md, the Azure Maps action reference ("What people see in chat"), the vendored-library table and test list in REACT_V2_UI.md, and the 0.261.223 release notes.

Interactive maps (0.261.223)

Replies that used the Azure Maps action now show their map in the V2 chat. Refs #1609.

• Map card under the reply. Opens fitted to every marker, path and area. Pan by dragging; zoom with the buttons, double-click, pinch or Ctrl/Cmd + scroll (a plain scroll keeps scrolling the conversation); Full screen expands the map.
• Details. Hovering a marker, path or area shows its label and description; a click keeps them open until the reader clicks elsewhere on the map or presses Escape.
• Text alternative. List what the map shows lists every marker, path and area as text, for keyboard and screen reader users.
• Vendored OpenLayers 10.6.1 under application/v2_ui/public/vendor/openlayers-10.6.1/ (ol.js, ol.css, LICENSE.md), byte-identical to the published ol@10.6.1 package and to the classic chat's copy, loaded only when a reply has a map. No npm dependency, no CDN asset, CSP unchanged.
• Action output stays text. Titles, labels and descriptions are rendered by React or set with textContent. OpenLayers' attribution control writes its strings as HTML, so it is turned off and the attribution is shown as text in the footer. Tile templates must be /api/azure-maps/tile?... with {z}, {x}, {y}; colours must look like colours; coordinates are range-checked.
• Compact citations. A map citation whose full result was moved to an artifact is fetched once through GET /api/conversation/<id>/agent-citation/<artifact_id>. Only map functions are fetched.
• Tile tokens in older messages. GET /api/get_messages and GET /api/collaboration/conversations/<id>/messages now reissue expired tile tokens through refresh_azure_maps_message_citations, as the classic message route already did, so maps in older replies and in group conversations load their tiles. refresh_azure_maps_function_result no longer parses JSON-text results that cannot be maps.
• Legacy {{map:...}} blocks are hidden from V2 reply text, as the classic chat hides them; the map comes from the tool result.

Validation

• functional_tests/test_v2_inline_maps.py: 7 passed. Vendored bytes match the published SHA-256, no HTML sink, attribution control off, tiles only through the proxy, wiring, both endpoints refresh, and the refresh helper reissues expired tokens in object and JSON-text results without mutating its input.
• functional_tests/test_v2_inline_maps_logic.mjs: 13 passed (Node, real module).
• ui_tests/test_v2_inline_maps.py: 4 passed. Real MessageList and card in Chromium with production CSS: tiles from the proxy, Full screen control, no HTML attribution control, legacy block hidden, hover details, click to pin, Escape, compact citation fetched and drawn, text list, fallback when OpenLayers cannot load.
• functional_tests/test_v2_rich_rendering.py registers OpenLayers with the vendored-library and forbidden npm dependency checks. functional_tests/test_content_screening_history.py gets the new helper in its route namespace. functional_tests/test_v2_enhanced_citations.py now checks the built media-src directive, since 0.261.222 moved that literal out of config.py.
• npm run typecheck and the Vite build pass.
• Also exercised in a browser harness against stored map payloads from a SimpleChat POC deployment: both maps drew with tiles only through the proxy and no console errors.

Not in this change

• Classic chat still shows details on click only; hover tooltips are V2-only for now.
• Conversation exports do not draw maps.

Map tiles in stored replies (0.261.224)

Deploying 0.261.223 to a SimpleChat POC deployment showed the cards drawing correctly but every tile request returning 400. The cause was older than the V2 work.

• Tool results go through sanitize_plugin_invocation_value before they are stored as citations, in chat and in the workflow runner. Its token= rule replaced the map payload's tile proxy token with ***REDACTED***, so every stored map, classic or V2, drew on blank tiles.
• _redact_secret_assignment now keeps the token= value only where it directly follows /api/azure-maps/tile?, which is where build_tile_proxy_url_template writes it. That token is encrypted with the app's secret key, expires, and only works on SimpleChat's signed-in tile proxy. Every other token=, another key after the prefix, a token later in the query, absolute URLs and every other secret are still redacted.
• Maps stored before this change keep the redacted token, which cannot be reissued. They still show no tiles.
• Tests: test_v2_inline_maps.py::test_tool_result_redaction_keeps_only_the_tile_proxy_token runs the real logger and Azure Maps helpers. test_plugin_tool_agent_security_audit.py and test_log_injection_codeql_alerts_fix.py still pass (47).
• Docs: docs/explanation/fixes/AZURE_MAPS_TILE_TOKEN_REDACTION_FIX.md, the V2 maps feature doc and the 0.261.224 release notes.