Repository navigation
[P1] Distinguish retrieved sources from cited documents #1249
Description
Summary
Separate complete retrieved source sets from the smaller set of document and web references explicitly cited in final assistant responses.
User Value
Users can inspect every retrieved result under Sources, see exactly cited documents in conversation navigation and details, and trust that exported reference sections contain actual citations rather than every candidate source.
Proposed Behavior
- Keep each assistant message's Sources disclosure as the full returned document, web, and agent/tool set.
- Persist exact cited document records by matching inline citation IDs in final assistant text.
- Persist exact cited web records by matching normalized URLs in final assistant text.
- Treat agent/tool records as used executions rather than unused retrieval candidates.
- Show only active-response cited documents in Used documents, with a no-migration legacy fallback for historical conversations.
- Keep the conversation details modal's complete source-document inventory and add a Cited marker only to exact tracked documents.
- Make conversation JSON/Markdown/PDF and per-message Word/PowerPoint/email reference sections use exact cited document/web subsets plus executed tools.
- Preserve full source arrays in JSON exports for audit and use historical broad-reference behavior only when tracking is absent.
- Avoid read-time history parsing; rebuild aggregates only on retry/edit, attempt switching, deletion, and fork mutations.
Acceptance Criteria
- Retrieved-but-not-referenced document and web results remain visible under Sources but are excluded from newly tracked references.
- Exact cited document and web subsets are persisted on new streaming, non-streaming, assigned-knowledge, mixed-source, and document-action responses.
- Used documents follows active, visible, non-deleted attempts and retains an explicit historical fallback without a database migration.
- Conversation details lists all source documents and marks only exact tracked documents as Cited.
- Conversation Markdown/PDF references and per-message Word/PowerPoint/email references exclude uncited tracked sources.
- JSON export retains full source arrays and adds exact cited subsets.
- Agent/tool references remain available because they represent executions that occurred.
- Retry, edit, attempt switch, delete, collaboration, and fork flows maintain correct aggregates.
- Functional and UI coverage verifies tracking, compatibility, source disclosure, details badges, and export behavior.
Notes
Expected size: L. Relevant areas include citation parsing, assistant message persistence, conversation metadata, retry/attempt lifecycle, collaboration serialization, conversation details/navigation, and export formatting.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
paullizer commentedon Aug 14, 2026
Implemented in
0.250.202.Validation completed: cited-source tracking 15/15, per-message exports 16/16, PowerPoint 7/7, retry/edit 4/4, fork coverage, route policy suites 12/12, XSS guardrails 4/4, changed-file broken-access-control scan, Python compilation, JavaScript syntax, and diff checks. Authenticated Playwright tests were collected but skipped because no UI base URL/storage state is configured in this session.
paullizer commentedon Aug 18, 2026
Rebased onto latest Development and opened PR #1268.
Version retargeted to
0.250.215because0.250.202was taken upstream by an unrelated fix. Release notes, fix documentation, feature documentation, and test headers updated accordingly.Post-rebase validation: cited-source tracking 15/15, per-message exports 16/16, PowerPoint 7/7, retry/edit 4/4, conversation fork, route policy suites 12/12, XSS guardrails 4/4, version-assertion and release-docs guardrails, plus Python compilation and
node --checkon all changed JavaScript.paullizer commentedon Aug 18, 2026
CodeQL flagged three high-severity polynomial regular expressions (ReDoS) in the new citation tracking module on the first PR run. The inline citation group, visible source reference, and page range patterns each combined unbounded whitespace or lazy quantifiers with variable-length character classes, so malformed model output could force quadratic backtracking.
Fixed by replacing the unbounded quantifiers with bounded repetition, precompiling the page range pattern, and adding a regression test that parses adversarial inputs within a fixed time budget.
CodeQL now reports "No new alerts in code changed by this pull request" and all 12 PR checks pass, including CodeQL Python/JavaScript/Actions analysis, broken-access-control, xss-sink, swagger-route, syntax, and release-notes. PR #1268 is mergeable with a clean merge state.
Distinguish retrieved sources from cited references
paullizer commentedon Aug 19, 2026
Shipped in v0.250.215 via #1268.
Release note entries:
Full notes: https://microsoft.github.io/simplechat/explanation/release_notes/
Closing as completed. This issue stayed open after the fix merged because pull requests target
Developmentwhile the repository default branch ismain, so GitHub never applied theFixes #keyword automatically.Gate inline image and video galleries to cited media only