Repository navigation
Feat/conversation artifacts panel - #4311
Merged
Merged
Conversation
added 18 commits
October 31, 2026 12:13
…files Collect every agent-generated file across a conversation and expose a persistent "view all artifacts (N)" entry above the composer. The entry opens a right-hand panel listing each artifact with icon, size and generation time, plus preview (reusing FilePreviewDrawer) and download. Aggregation covers both attachment paths: streamed marker parts carrying skillFileAttachments and the history-restore skillFileUploadsRegistry. Keys stay unique across streaming chunks whose attachment ids restart. Unit proof lives in conversationArtifacts.test.tsx (aggregation, panel, preview, download fallbacks, entry button). The change reuses the proven SSE files-chunk, preview-drawer and storage contracts without adding a new cross-component boundary, so no formal D1 case is added; the assessment is recorded in CHAT-ARTIFACTS-PANEL-20261031.
Artifacts listed in the conversation artifacts panel can now be re-attached into the composer (WorkBuddy "添加到对话框" parity): - attachment-adapter: files marked via markFileWithExistingUpload carry their MinIO metadata through add(); send() reuses the stored object instead of uploading the same bytes again, keeping minio_files intact - conversation-artifacts: fetchArtifactFile resolves artifact bytes via the backend stream endpoint for object_name with URL fallbacks - artifacts-panel: optional onAddToComposer prop renders a per-entry "add to chat" action with an in-flight guard; omitted in read-only and shared conversations - thread: wires the handler to the main thread composer with success and failure feedback; artifacts without object_name fall back to the regular upload path - i18n: chat.artifacts.addToChat / addedToChat / addToChatFailed (zh/en) - tests: 8 new UTs (22 total) covering the panel action, the fetch fallback chain, and stored-object reuse; adds the missing copyright header to attachment-adapter.ts Change record updated: CHAT-ARTIFACTS-PANEL-20261031.
List every user question asked in the current agent conversation and
jump the message viewport to the selected turn ("问题目录"):
- question-directory utils: collectQuestionDirectory extracts one entry
per user message (first text line truncated for the summary, full text
kept for the tooltip, first attachment name for attachment-only
turns); scrollThreadToMessage locates the message root via
data-message-id (assistant-ui exposes no scroll API), smooth-scrolls
it into view, and briefly highlights the target
- question-directory ui: QuestionDirectoryButton renders a floating chip
on surfaces without a persistent header (main newchat flow) and a
ghost header icon button elsewhere (workbench, share mode, read-only
page); the shared popover lists numbered entries and closes on jump
- thread: directory computed next to the artifacts aggregation and
wired into all four surfaces; navigation stays available while a run
streams and in read-only/share contexts
- i18n: chat.questions.viewAll/title/total/jump (zh/en)
- tests: 8 new UTs (30 with the artifacts regression) covering
collection rules, scroll/highlight behavior, and popover interaction
Change record added: QUESTION-DIRECTORY-JUMP-20261102.
The question directory popover now filters entries as you type: - a search box at the top of the popover list, matching the full question text (not just the truncated summary) case-insensitively so keywords from any line still find the entry; attachment turns match by their file name - unmatched keywords show an explicit empty state - the keyword resets when the popover closes so reopening starts from the full directory again - i18n: chat.questions.search / noMatches (zh/en) - tests: 3 new UTs (11 total in the file) covering keyword filtering, the empty state, and the close-reset behaviour Change record updated: QUESTION-DIRECTORY-JUMP-20261102.
Add a "branch to new chat" action on the assistant message "..." menu. Branching creates a new conversation that copies the configuration (agent, chat mode, knowledge scope, runtime metadata, workbench config) plus every message, unit, search source, and image source up to and including the selected Q&A pair, then switches to the branched thread. The original conversation and its MinIO file references stay untouched; user feedback is not carried over. The turn is addressed by a 1-based index that both sides compute the same way: the frontend counts user messages up to the selected answer, the backend counts its active (deduplicated) user rows in message_index order, so no message-id mapping round trip is needed. Backend: POST /conversation/branch with conversation_id and turn_index; the repository clones rows in one transaction behind the user-quota advisory lock and collapses refresh duplicates / regenerated answers to the newest row per message_index. The branched title reuses the source title with a single "（分支）" suffix within the 100-char column.
Selecting text inside one assistant reply shows a floating "Add to
chat" bubble above the selection. Confirming it (optionally after
typing a comment) stores a pending annotation and renders a count
chip above the composer input; hovering the chip opens the annotation
list where each entry can be edited or deleted, and the trailing
button clears all.
Sending folds every pending annotation into the outgoing message
text as a numbered quote block ("> "-prefixed quoted reply lines, an
optional comment line, and the first-40-char excerpt of the source
reply), followed by a "---" separator and the user-typed text, so the
model resolves the reference from the full conversation history
without any backend or protocol change. Annotation-only sends (empty
composer text) are allowed. The fold is idempotent for the
send-button/form-submit double invocation; a rejected run keeps the
annotations and restores the original draft, and the annotations are
consumed only once the run accepts. Switching threads resets them.
Quotable selections are restricted to the content area of a single
assistant message: reasoning, tool, execution-code, and subagent
containers are excluded, and the bubble is disabled in read-only
threads, share mode, skill-file chats, and the preview layouts.
Upgrade the "add to chat" flow from frontend-only text folding to a LibreChat-style structured channel: - AgentRequest/MessageRequest gain an `annotations` field (snake_case wire items quoted_text/comment/source_excerpt, capped at 10 entries and 1500/500/120 chars, truncated rather than rejected) via the new contracts/annotations.py module. - The user message persists annotations as a `message_annotations` unit (mirroring clarification_response) while the stored message_content stays clean; history reads return the structured field. - The backend merges the numbered quote blocks into the model-facing text at request time: the current turn in prepare_agent_run and past turns in get_historical_context. - The composer takes over the append when annotations are pending (no in-flight uploads) so the optimistic user message carries them as metadata; the user bubble renders read-only reference cards, restored from history after reload. In-flight uploads fall back to the runConfig channel; stale payloads are cleared so they cannot leak into clarification sends or edit resends. Ordinary sends are untouched. - The fold function, its localized labels and the double-invocation guard are removed; formatting now happens in the backend.
lijiayang619
requested review from
Dallas98,
Jasonxia007,
WMC001,
YehongPan,
hhhhsc701 and
jeffwu-1999
as code owners
November 9, 2026 07:14
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved defects can expose internal database diagnostics, disrupt keyboard popover access, allow duplicate artifact operations, and disable bookmarks after annotation-only turns.
4 open issues
What changed in this PR
Adds conversation-wide artifact browsing and document previews, alongside question navigation, persistent bookmarks, conversation branching, and structured reply annotations.
Changes:
- Adds artifact aggregation, preview, download, reattachment, and automatic document opening.
- Introduces question rails, saved bookmarks, numbered branches, and selected-text annotations.
- Extends backend APIs, contracts, persistence, localization, tests, and requirement records.
This pull request requires changes. Unresolved moderate findings cover database-error disclosure, annotation-only history loss, annotation-popover keyboard focus, keyboard clear-all handling, TSV preview routing and automatic-preview eligibility, duplicate artifact additions, bookmark identity mapping, and branching session-expiration handling.
| File | Description |
|---|---|
test/backend/services/test_question_bookmarks.py |
Tests bookmark validation, ownership, and endpoint behavior. |
test/backend/services/test_conversation_management_service.py |
Tests conversation branching service behavior. |
test/backend/database/test_conversation_db.py |
Updates historical-context query mocks. |
test/backend/database/test_branch_titles.py |
Tests branch-title numbering and collisions. |
test/backend/app/test_conversation_management_app.py |
Tests branching endpoint responses. |
test-e2e/changes/requirements/REMOVE-QUESTION-DIRECTORY-POPOVER-20261109.yaml |
Records directory-popover removal requirements. |
test-e2e/changes/requirements/QUESTION-DIRECTORY-RAIL-20261105.yaml |
Defines question-rail navigation requirements. |
test-e2e/changes/requirements/QUESTION-DIRECTORY-RAIL-20261105-verification.md |
Documents rail checks and verification limits. |
test-e2e/changes/requirements/QUESTION-DIRECTORY-JUMP-20261102.yaml |
Records initial question-directory requirements. |
test-e2e/changes/requirements/QUESTION-BOOKMARKS-20261109.yaml |
Defines persistent bookmark behavior. |
test-e2e/changes/requirements/QUESTION-BOOKMARKS-20261109.verification.md |
Documents bookmark checks and migration prerequisites. |
test-e2e/changes/requirements/CHAT-ARTIFACTS-TOP-ENTRY-20261106.yaml |
Defines persistent artifact-toolbar placement. |
test-e2e/changes/requirements/CHAT-ARTIFACTS-TOP-ENTRY-20261106-verification.md |
Documents toolbar lifecycle checks. |
test-e2e/changes/requirements/CHAT-ARTIFACTS-PANEL-20261031.yaml |
Defines artifact aggregation and file actions. |
test-e2e/changes/requirements/BRANCH-NUMBERED-TITLES-20261106.yaml |
Defines numbered branch-title requirements. |
test-e2e/changes/requirements/BRANCH-NUMBERED-TITLES-20261106-verification.md |
Documents title-allocation checks and limits. |
test-e2e/changes/requirements/BRANCH-CONVERSATION-TO-NEW-CHAT-20261102.yaml |
Defines conversation branching requirements. |
test-e2e/changes/requirements/AUTO-OPEN-DOCUMENT-20261109.yaml |
Defines automatic document-preview behavior. |
test-e2e/changes/requirements/AUTO-OPEN-DOCUMENT-20261109.verification.md |
Documents preview checks and limitations. |
test-e2e/changes/requirements/ANNOTATION-SELECTED-TEXT-ONLY-20261109.yaml |
Defines selected-text-only annotation behavior. |
test-e2e/changes/requirements/ANNOTATION-SELECTED-TEXT-ONLY-20261109.verification.md |
Documents selected-text annotation checks. |
test-e2e/changes/requirements/ADD-TO-CHAT-MESSAGE-ANNOTATIONS-20261102.yaml |
Records initial annotation interaction requirements. |
test-e2e/changes/requirements/ADD-TO-CHAT-LOCAL-CONTEXT-20261105.yaml |
Records earlier context-capture requirements. |
test-e2e/changes/requirements/ADD-TO-CHAT-LOCAL-CONTEXT-20261105-verification.md |
Documents context-capture verification. |
test-e2e/changes/requirements/ADD-TO-CHAT-ANNOTATIONS-STRUCTURED-20261102.yaml |
Defines structured annotation requirements. |
test-e2e/changes/bugs/QUESTION-RAIL-STALE-PREVIEW-20261105.yaml |
Records the adjacent-preview regression. |
test-e2e/changes/bugs/QUESTION-RAIL-STALE-PREVIEW-20261105-verification.md |
Documents preview-switching regression evidence. |
frontend/types/conversation.ts |
Adds annotation response types. |
frontend/types/chat.ts |
Extends preview presentation types. |
frontend/tests/component/viewportScrollRescue.test.tsx |
Tests scrolling and streaming-follow behavior. |
frontend/tests/component/rawStreamPreviewViewer.test.tsx |
Checks DOCX wrapper configuration. |
frontend/tests/component/questionBookmarks.test.tsx |
Tests bookmark state and identity mapping. |
frontend/tests/component/inlineFilePreview.test.tsx |
Tests inline preview compatibility. |
frontend/tests/component/docxPreviewBackground.test.tsx |
Tests rendered DOCX paper styling. |
frontend/tests/component/conversationBranch.test.tsx |
Tests branch turn calculation and menu wiring. |
frontend/tests/component/autoArtifactPreview.test.tsx |
Tests automatic-preview lifecycle. |
frontend/tests/component/artifactDocumentPanel.test.tsx |
Tests document sources and panel controls. |
frontend/services/conversationService.ts |
Adds bookmark, branch, and annotation requests. |
frontend/services/api.ts |
Defines bookmark and branch endpoints. |
frontend/public/locales/zh/common.json |
Adds Chinese interaction labels. |
frontend/public/locales/en/common.json |
Adds English interaction labels. |
frontend/hooks/chat/useQuestionBookmarks.ts |
Manages bookmark queries and mutations. |
frontend/components/common/rawStreamPreviewViewer.tsx |
Enables DOCX page-wrapper rendering. |
frontend/components/common/rawStreamPreviewViewer.module.css |
Scopes DOCX page styling. |
frontend/components/common/filePreviewDrawer.tsx |
Adds inline preview presentation. |
frontend/app/[locale]/newchat/utils/question-directory.ts |
Collects questions and tracks navigation. |
frontend/app/[locale]/newchat/utils/message-annotations.ts |
Provides selection and annotation-format helpers. |
frontend/app/[locale]/newchat/utils/conversation-branch.ts |
Computes branch turn boundaries. |
frontend/app/[locale]/newchat/utils/conversation-artifacts.ts |
Aggregates generated files and retrieves attachment data. |
frontend/app/[locale]/newchat/ui/question-directory-rail.tsx |
Adds question-navigation ticks and previews. |
frontend/app/[locale]/newchat/ui/question-bookmark-rail.tsx |
Connects question identities to saved bookmarks. |
frontend/app/[locale]/newchat/ui/conversation-artifacts-context.tsx |
Shares artifact toolbar state. |
frontend/app/[locale]/newchat/ui/attachment.tsx |
Exports shared artifact-action helpers. |
frontend/app/[locale]/newchat/ui/artifacts-panel.tsx |
Lists artifacts and exposes file actions. |
frontend/app/[locale]/newchat/components/use-auto-artifact-preview.ts |
Selects newly generated documents for automatic preview. |
frontend/app/[locale]/newchat/components/clarification-message-card.tsx |
Excludes pending annotations from clarification sends. |
frontend/app/[locale]/newchat/components/artifact-document-panel.tsx |
Hosts inline artifact document previews. |
frontend/app/[locale]/newchat/assistant-ui/viewport-scroll-rescue.tsx |
Restores history and streaming scroll behavior. |
frontend/app/[locale]/newchat/assistant-ui/message-annotation-cards.tsx |
Displays message annotation cards. |
frontend/app/[locale]/newchat/assistant-ui/message-annotation-bubble.tsx |
Captures selected reply text and comments. |
frontend/app/[locale]/newchat/assistant-ui/conversation-branch-menu-item.tsx |
Adds the conversation branch action. |
frontend/app/[locale]/newchat/assistant-ui/conversation-branch-context.ts |
Shares branching availability and controls. |
frontend/app/[locale]/newchat/assistant-ui/composer.tsx |
Integrates annotation-aware message submission. |
frontend/app/[locale]/newchat/assistant-ui/annotation-chip.tsx |
Displays and manages pending annotations. |
frontend/app/[locale]/newchat/adapter/remote-chat-model-adapter.ts |
Sends structured annotations to the backend. |
frontend/app/[locale]/newchat/adapter/conversation-thread-list-adapter.tsx |
Restores annotation metadata from history. |
frontend/app/[locale]/newchat/adapter/attachment-adapter.ts |
Reuses existing uploaded artifacts. |
deploy/sql/migrations/v2.9.0_009_question_bookmarks.sql |
Adds persistent bookmark storage. |
backend/runtime/api/application.py |
Registers bookmark routes. |
backend/management/services/conversation/service.py |
Adds branching and annotation-history handling. |
backend/management/services/conversation/bookmarks.py |
Validates bookmark operations. |
backend/management/services/agent/run.py |
Incorporates annotations into model input. |
backend/management/db/conversation/models.py |
Maps the bookmark storage column. |
backend/management/db/conversation/branch_titles.py |
Allocates numbered branch titles. |
backend/management/db/conversation/bookmarks.py |
Implements owner-scoped bookmark persistence. |
backend/management/api/conversation/question_bookmarks.py |
Adds authenticated bookmark endpoints. |
backend/management/api/conversation/conversation_management.py |
Exposes conversation branching. |
backend/contracts/conversation.py |
Adds annotation and branching contracts. |
backend/contracts/annotations.py |
Defines annotation normalization and formatting. |
backend/contracts/agent.py |
Adds structured request annotations. |
Review effort: Balanced
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
added 5 commits
November 9, 2026 17:04
Merge develop at 3de8384 while retaining conversation branching, reply annotations and artifact preview changes. Resolve overlapping repository imports by retaining the dependencies from both branches and removing duplicates. No conversation function definitions are removed. Existing SQL files match develop. Validation: 464 targeted backend tests passed in isolated temporary directories, covering branch selection/copying, annotations, history, multi-role permissions, database, services and API contracts. The API suite required pytest-asyncio in the temporary test path. Copyright checks passed for all 55 changed source files relative to develop; git diff --check passed. Full application and live integration suites were not run.
jeffwu-1999
approved these changes
Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更
验证
图片：
1.当前对话历史记录
2.查看所有历史产物
3.分支到新对话
4.添加批注
