Repository navigation
V2: Merge multiple files (CSV/Excel, PDF, Word, PowerPoint) through chat orchestration and workflows #1619
Copy link
Copy link
Open
Labels
P1Priority 1: urgent, address before P2 workPriority 1: urgent, address before P2 workenhancementNew feature or requestNew feature or request
Description
Summary
Add a Merge capability to the V2 experience that combines several files into one. It starts with CSV and Excel files that share a structure (a few files interactively in chat, ~100+ files as durable workflow work), then extends to reconciling differing spreadsheet structures, PDFs, Word documents and PowerPoint decks.
User Value
Users regularly receive the same report from many sources (regions, months, teams) and need one combined file. Today SimpleChat cannot do this: native tabular analysis is single-source by contract, the tabular stack has no union/append operation, workflows only publish JSON, and no code concatenates PDFs, Word documents or slide decks. Asking a model to "write a combined CSV" is slow, expensive and lossy for real data.
Proposed Behavior
- Merge is another capability alongside Analyze and Compare, not a replacement for anything.
- Reason:
tabular_merge(and laterdocument_merge) combines authorized sources deterministically into a retained result. No model rewrites rows; counts are exact and verifiable. - Render: the existing
render_filecreates the downloadable file (CSV, XLSX, later PDF/DOCX/PPTX). - Gather (optional):
document_searchfinds the files when the user describes them; Phase 2 addstabular_inspectto read headers for reconciliation.
- Reason:
- V2 chat: users just ask ("merge these three CSVs into one Excel file with a source column"). The orchestrator plans Gather → Merge → Render and the user reviews the plan as usual.
- V2 workflows: a task can use the new Merge document action for large, scheduled or File Sync-triggered merges, with per-file checkpoints and CSV/XLSX outputs. Chat can propose a merge workflow when a request exceeds inline limits, including a Create & run once option for one-off jobs.
- Admin control: a new
mergeentry in Document Actions (enabled by default like Analyze/Compare) with chat/workflow document and row limits. - Formatting choices: PowerPoint and Word merges support "keep source formatting" (default) or "use the first file's theme/styles", chosen per request.
- Classic UI is unchanged; there is no Document Explorer merge button.
Acceptance Criteria
- Phase 0 — Tracking issue and umbrella design doc (
docs/explanation/features/V2_FILE_MERGE.md). - Phase 1 — Same-structure CSV/Excel merge in V2 chat:
tabular_merge(by-name or exact-order schema policies, optional sheet, Source File column),mergedocument action with admin limits, exact-schema CSV/XLSX export profiles, V2 plan card support, tests and docs. - Phase 2 — Reconcile differing structures:
tabular_inspect, union and mapped policies, header row, sheet modes, dedupe/sort, honest exclusion of incompatible files, AI-suggested column mapping (tabular_column_mapping_v1) with a review-first flow. - Phase 3 — Large and recurring merges (100+ files) in V2 workflows: Merge task with selected/all/recent targets and per-source checkpoints, CSV/XLSX workflow outputs and workspace publication, chat proposal hand-off with Create & run once, V2 workflow editor and workflow assistant support.
- Phase 4 — Document assembly foundation (
document_merge,document_assembly_v1, lineage-pinned Render profiles), PDF merge, and Excel workbook with one tab per source file. - Phase 5 — Word (DOCX) merge: keep source styles (default) or use the first document's styles.
- Phase 6 — PowerPoint (PPTX) merge: keep source formatting (default) or use the first deck's theme, with a section per source deck.
- Phase 7 — Scale and security hardening (zip bombs, XXE, encrypted/macro files, external relationships, formula injection), user guide, release notes.
- Every phase ships functional tests, documentation, a version bump and release notes.
Notes
- Related: Complete the orchestration harness with Gather, Reason, Render and explicit file creation #1509 (Gather / Reason / Render harness), [Generated exports] Support universal CSV, JSON, and XML file generation #1071 (universal CSV generation).
- Bounds to respect: records results allow up to 256 columns; CSV exports up to 1,000,000 records; XLSX up to 1,048,575 rows / 5,000,000 cells / 32 MiB; orchestration runs default to a 600 s deadline; workflow loops up to 5,000 items.
- Phase 4+ Render profiles read the authorized originals named in the retained assembly lineage, with access rechecks and revision pinning (
source_changedon drift). - Out of scope: key-based joins (VLOOKUP-style), cross-format conversion, classic UI support.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P1Priority 1: urgent, address before P2 workPriority 1: urgent, address before P2 workenhancementNew feature or requestNew feature or request
Type
Projects
- StatusShow more project fieldsIn progress
Activity
paullizer commentedon Oct 3, 2026
Status: every phase is implemented; the PR is next
All seven phases are implemented on branch
paullizer-multi-file-merge, which isn't pushed yet. It is synced withpaullizer-react-v2-ui(f1aeef1, v0.261.233) and renumbered above it, so this work ships as v0.261.234 – v0.261.242.tabular_mergein V2 chat for same-structure CSV/Excel files; themergedocument action with chat and workflow file and row limits; exact-schema CSV/XLSX export profiles; plan card wordingtabular_inspect; union, mapped and alias column policies; header rows, every-sheet mode, exclusion, duplicate removal and sorting; AI-prepared column mapping (tabular_column_mapping_v1) that code checks before any row is copieddocument_mergein chat for every kind: the merge step retains adocument_assembly_v1description, and Render (assembled_document_v1) assembles the same files again and delivers the file only if it is byte-identical. Hardening: DDE fields refused, template links removed, PowerPoint program/macro actions removed, XML with a DOCTYPE or a non-UTF-8/16 encoding refused before parsing, ZIP size guards, and no live formulas in CSV/XLSX output/sc-temp-files), like file renders since 0.261.233Validation
paullizer-react-v2-ui, a regression sweep of 465 orchestration, workflow, merge, export, docs and admin test files found no new failures. 79 files fail the same way onpaullizer-react-v2-uiwithout this work, and one test that races on a shared cache when shards run in parallel passes on its own. After the final sync (Fix orchestration file generation when /app is root-owned (V2) #1626, Update roadmap status: Phase 7's 7a design, and container-only results and self-authored Run as #1633), the 20 affected suites pass: 930 tests.paullizer-react-v2-uinow authorizes a retained result from its conversation and run without rereading its documents. A merged document is the deliberate exception: it is assembled from the originals when it renders, so they are read with the user's current access. Tests and docs cover this.Differences from the acceptance criteria
Phase 3 is left unticked because four items were changed or not built:
The Phase 4 assembly foundation (
document_merge,document_assembly_v1, lineage-pinned Render profiles) shipped with Phase 7, together with chat document merges.Remaining
paullizer-react-v2-ui. If more PRs land there first, this branch is synced and renumbered again.Docs:
docs/explanation/features/V2_FILE_MERGE.md(umbrella) and phase docs 1–7,docs/guides/merge-files.md, and release notes v0.261.234–v0.261.242.Fix CodeQL alerts for file merge
Document the shared document merge core module (refs #1619)
Drop the unused Sequence import from the merge core (refs #1619)
Add V2 spreadsheet merge capability (Phase 1, refs microsoft#1619)
Reconcile differently structured spreadsheets in V2 merges (Phase 2, …
Merge files as V2 workflow tasks (Phase 3, refs microsoft#1619)
Merge PDFs and build workbooks in V2 workflows (Phase 4, refs microso…
Fix test harnesses broken by the merge phases (refs microsoft#1619)
Merge Word documents in V2 workflows (Phase 5, refs microsoft#1619)
Merge PowerPoint decks in V2 workflows (Phase 6, refs microsoft#1619)
Merge documents in V2 chat and harden file merges (Phase 7, refs micr…
Ask one sheet or a sheet per file before merging spreadsheets (refs m…
Spill large merges into the container's scratch directory (0.261.242)