Use Azure AI Content Understanding for Enhanced document extraction

[Paul Lizer (paullizer)]

Fixes #1277

Summary

Enhanced extraction for PDFs and images now uses Azure AI Content Understanding (prebuilt-documentSearch) instead of Document Intelligence prebuilt-layout. Content Understanding returns layout-aware markdown plus AI-generated descriptions of figures, charts, and diagrams — the structure Document Intelligence never produced.

Standard extraction is unchanged and still uses Document Intelligence prebuilt-read, which remains required for workspaces and chat file uploads.

functions_content_understanding.py returns the exact same [{page_number, content}] shape as extract_content_with_azure_di(), so the pipeline swap is a contained branch rather than a rewrite. It uses REST over requests — no new dependency.

Enhanced never becomes a hard dependency

Content Understanding is only offered in Azure commercial regions and is not available in Azure Government. resolve_enhanced_extraction_engine() falls back to Document Intelligence Layout when the cloud is unsupported, when Content Understanding is unconfigured, or when a request fails at runtime. The reason is recorded on the document and surfaced in workspace tooltips.

The admin UI is environment-aware: Azure Public shows the Content Understanding configuration with a Test Connection button and a setup guide; Azure Government and custom clouds show that Enhanced uses Document Intelligence Layout and there is nothing to configure.

Other behavior

• Auto still samples the first pages with Document Intelligence Layout as the cheaper detector, but now also upgrades when it finds figures or images, not just tables and selection marks.
• Enabling the Enhanced toggle defaults the mode selector to Auto.
• Images inside DOCX/PPTX are pulled out of the OOXML package and analyzed with whichever engine backs the selected mode, then indexed as their own citable chunks. PowerPoint images are attributed to the slide that references them. Cost is bounded by a minimum size filter, content-hash de-duplication, and a per-document cap. This works with Standard extraction too.
• The Change Extraction action now covers images as well as PDFs, and refuses a change to Enhanced while Enhanced is disabled.

Compatibility

Existing setting and per-document field names are preserved, so there is no Cosmos or search index migration. document_intelligence_pdf_image_extraction_mode keeps its read / layout / auto values; layout now means "Enhanced", which routes to Content Understanding when available. New per-document fields (extraction_engine, extraction_engine_reason) are additive.

Upgrade path: the new enable_enhanced_extraction toggle defaults to False, so deployments already set to Enhanced or Auto are migrated on first settings read — the toggle is switched on and persisted. Without this they would have silently downgraded to Standard and then had the stored mode overwritten on the next settings save.

Bugs found in review

Two review rounds surfaced six issues, all fixed here:

#	Severity	Issue
1	High	Upgrade would silently downgrade Enhanced/Auto deployments to Standard, then permanently lose the stored mode
2	High	Declared-size zip check was not a real defense — ZipInfo.file_size is forgeable and CPython decompresses before truncating
3	Medium	Zip-slip: output paths derived from archive entry names could escape the extraction directory on Windows
4	Medium	Mermaid diagrams discarded whenever their description was already inlined in the markdown
5	Medium	Figure label shared a line with its content, so a diagram-only block opened its code fence mid-line — invalid markdown
6	Low	Caption-only figures dropped, even though descriptions are optional in the API schema

On #2, measured directly: a 65 KB entry declaring 4096 bytes peaked at 141 MiB through archive.read(); bounded streaming peaks at 0.29 MiB. Uploaded Office files are now read through archive.open() with a hard byte ceiling, a compression-method whitelist, and caps on entries and relationship parts inspected. Slide relationship parts are parsed with defusedxml (already a pinned dependency).

Testing

Two new test files with 32 tests, including real DOCX/PPTX zip round-trips and behavioral tests of page reconstruction against the documented response shape. Every security fix has a regression test that was confirmed to fail against the old code rather than assumed correct.

The full 857-file functional suite was run against a baseline worktree at the pre-change commit, diffing per test so the repo's many pre-existing failures (stale pinned versions, tests needing live Azure) were excluded:

Slice	Tests	Regressions
0	172	0
1	172	1
2	171	0
3	171	0
4	171	0

The single regression was test_legacy_doc_ole_extraction.py, which loads functions_content.py through importlib with a hand-curated stub set and broke on the new functions_office_media import. Fixed by putting the app directory on sys.path so the test imports the real module rather than stubbing it. Route policy contract tests also pass.

Reviewer notes

• Cost: prebuilt-documentSearch invokes a language model per document, so Enhanced is meaningfully more expensive than Document Intelligence Layout. Auto is the recommended default, which is why enabling the toggle lands there.
• Setup: Content Understanding requires default completion and embedding model deployments on the Foundry resource. The Test Connection button detects that specific failure and explains it rather than surfacing a raw HTTP error.
• Not included: an APIM passthrough for Content Understanding, for parity with the other Azure services. Happy to add it if wanted.

Version bumped 0.250.217 → 0.250.218. Feature doc added at docs/explanation/features/CONTENT_UNDERSTANDING_ENHANCED_EXTRACTION.md, release notes updated with an upgrade-notes section.