Limit content screening intake to uploads and apply metadata edits directly (layer 2 of 4)

[Paul Lizer (paullizer)]

Summary

• Screening intake is limited to uploads. Content screening now checks only documents that enter a workspace through a user upload (personal, group or public, including chat uploads saved as chat_upload workspace documents) or File Sync. Generated documents (files the SimpleChat agent action saves, and artifacts published into a workspace) get a server-managed screening_exemption: {"reason": "generated", "version": 1} field when the version is created, before initial_document_marker runs. They get no screening marker, process normally while screening is on, and scan jobs skip them (screening_not_required). Clients can't set the field: reject_screening_fields refuses any screening* key, update_document refuses it, and responses never include it. A document that already has a screening marker stays screened.
• The publication screening reservation is gone. Removed _consume_publication_reservation, consume_artifact_publication_screening_scan and the reservation, enrollment and consumption code in publication and readiness. Approval now hands a publication straight to processing. A destination without a screening marker counts as screening-complete, and the destination access check stays. A destination held by screening before this version can't be approved; the requester cancels and requests publication again.
• Metadata edits apply directly. User edits and model-generated metadata on screened documents no longer queue a rescan or hold the document. queue_metadata_rescan is replaced by validate_screened_metadata_update, so a rename still can't change a screened file's extension. The release proof, publication checkpoint and scan-job match no longer compare metadata_fingerprint. The group and public metadata APIs return only updated (200); their routes no longer have a queued (202) branch. The V2 Documents explorer no longer handles a queued receipt. Before, that path removed the document from the list and showed "Screening is queued". Now a save is confirmed only by an updated 200 receipt, and any other receipt keeps the draft. Tag edits on a screened document no longer rewrite the released blob's metadata, which would break the ETag-pinned release.
• Impact: generated files are available as soon as processing finishes, and editing metadata no longer makes a released document unavailable. In V2, the edited document stays in the list with its edit shown. There is no data migration: existing documents keep their screening state.

Linked issue

Refs #1621

Layer 2 of 4 in the #1621 stack, stacked on #1628 (paullizer-workspace-knowledge-holds).

Release Notes & Latest Features

• New Feature
• Bug Fix
• UI Enhancement
• Breaking Change
• Internal only

Is this visible to end users?

• Yes
• No

Is this admin-facing (Admin Settings, governance, deployment, config)?

• Yes
• No

Should this become a Latest Feature card?

• Yes
• No
• Already added

Screenshot needed for the card?

• Yes
• No
• Attached

Version bump

• application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only (0.261.229 → 0.261.230)
• deployers/version.txt bumped, or not needed because deployers/ was not changed

Testing / validation

The global interpreter can't collect these tests (cryptography 50 with pyOpenSSL 25.3, and Flask 2.2.5 with Werkzeug 3.1.4). Every Python command below ran in a venv overlay: system site packages plus cryptography 46.0.7, Flask 3.1.3, Werkzeug 3.1.6 and Flask-Session 0.8.0.

• python -m pytest functional_tests/test_content_screening_upload_only_intake.py -q: passed, 127 passed. With the application changes reverted and only the tests applied: 22 failed, 3 errors, 102 passed. The cases that pass either way are the existing client guards, the readiness check and the import check.
• Base-branch comparison: ran python -m pytest <file> -q -rfE on each of the 259 functional_tests files at base 66a29bd60 and at ed2a15a0d, the branch before the V2 follow-up. No new failures. The functional suites that the follow-up commit 8fc81b523 touches were re-run afterwards; those results are listed below.
  • Base: 5591 passed, 257 failed, 258 errors.
  • Branch: 5648 passed, 256 failed, 258 errors.
  • In the 258 files present on both branches, 502 test IDs fail on base and 501 on this branch. The difference is one flaky scheduler timing test that passed on the branch run.
  • test_tabular_row_orchestration_scale.py times out at the same test on both branches.
• Updated suites (python -m pytest <file> -q), branch result vs base:
  • test_content_screening_jobs.py: 112 passed (base: 112 passed).
  • test_content_screening_pipeline.py: 26 passed, 6 failed. The base has the same 6 failures, from the _format_vision_analysis_block harness.
  • test_content_screening_history.py: 11 passed (base: 10 passed).
  • test_group_document_management.py: 238 passed (base: 237 passed).
  • test_public_document_publication.py: 59 passed (base: 63 passed). 8 reservation-latch cases were removed and 4 hold and no-reservation cases were added.
  • test_group_document_fixture_parity.py: 103 passed (base: 103 passed).
  • test_public_document_fixture_parity.py: 63 passed (base: 63 passed).
  • test_group_document_publication.py: 127 passed.
  • test_public_document_management.py: 205 passed.
  • test_analysis_artifact_publication.py: 53 passed.
  • test_orchestration_artifact_publication.py: 13 passed.
  • test_group_document_publication_screening_bootstrap.py tested only the removed reservation and was deleted. Its import check moved to the new file.
• Re-run at 8fc81b523, after the V2 follow-up changed the UI fixtures and the two metadata routes (python -m pytest <file> -q), all passed:
  • test_group_context_fixture_parity.py: 230 passed.
  • test_public_context_fixture_parity.py: 252 passed.
  • test_group_document_fixture_parity.py: 103 passed.
  • test_public_document_fixture_parity.py: 63 passed.
  • test_group_document_management.py: 238 passed.
  • test_public_document_management.py: 205 passed.
  • test_group_screening_fixture_parity.py: 35 passed.
  • test_content_screening_upload_only_intake.py: 127 passed.
• Route policy scripts, re-run at 8fc81b523, all passed:
  • python functional_tests/route_tests/test_route_blueprint_policy_inventory.py: 11/11.
  • python functional_tests/route_tests/test_route_unauthenticated_policy_contract.py: 6/6.
  • python functional_tests/route_tests/test_route_policy_test_coverage.py: 2/2.
• V2 SPA, in application/v2_ui after npm ci:
  • npm run typecheck: passed.
  • npm run build: passed. The only warning is the existing chunk-size warning.
• V2 browser suites (python -m pytest <file> -q), run against the built SPA with local Playwright, PYTHONPATH=ui_tests\fixtures and PLAYWRIGHT_SERVICE_URL unset:
  • ui_tests/test_v2_group_document_management.py: passed, 58 passed. This includes the rewritten test_screened_metadata_edit_applies_directly_and_keeps_the_document_available and the new legacy-queued-202 case in test_malformed_metadata_success_never_discards_the_draft.
    • With the two TS files reverted and the SPA rebuilt, the malformed-receipt test fails on legacy-queued-202, because the old explorer accepted the receipt and closed the dialog.
    • The rewritten test passes either way, because the old explorer already handled updated. The parity tests above pin the server's updated receipt.
  • ui_tests/test_v2_public_documents.py: passed, 67 passed.
  • ui_tests/test_v2_personal_document_scope.py: passed, 12 passed.
  • ui_tests/test_v2_group_document_collaboration.py: passed, 33 passed.
  • ui_tests/test_v2_group_documents.py: passed, 30 passed.
  • ui_tests/test_v2_content_screening.py: 35 passed, 1 failed. The failure is test_admin_policy_is_discoverable_and_editable_before_citations_are_enabled, on the admin screening settings page. It fails the same way against a build of the base branch's V2 source.
• python -m pytest ui_tests/test_content_screening_classic.py -q, run locally with PLAYWRIGHT_SERVICE_URL unset: 46 passed, 3 failed. The failures are the 3 known base failures: test_classic_policy_prerequisites_fail_closed and test_classic_disable_new_scans_preserves_review_holds[True/False].
• Docs, re-run at 8fc81b523:
  • python scripts/build_docs_inventory.py: the only change to docs/_data/app_surface.yml was line endings, restored with git checkout -- docs/_data/app_surface.yml.
  • python functional_tests/test_docs_app_surface_coverage.py: passed, 7/7.
  • python functional_tests/test_docs_site_quality.py: passed, 6/6.
• Guardrails over the added lines (66a29bd60..8fc81b523), all passed:
  • python scripts/check_xss_sinks.py --base-sha 66a29bd60 --head-sha HEAD <38 changed files>: 28 files checked.
  • python scripts/check_broken_access_control.py --base-sha 66a29bd60 --head-sha HEAD <38 changed files>: 26 files checked.
  • python scripts/check_swagger_routes.py application/single_app/route_backend_group_documents.py application/single_app/route_backend_public_document_management.py: 2 changed route files checked.

Known limitation: a metadata edit made through the API while a document is still held is replaced by the inspected copy when the document is released. The workspace interfaces don't offer metadata edits on held documents.

Documentation

• Release notes updated, or not needed: ### **(v0.261.230)** under Breaking Changes, above v0.261.229.
• Feature documentation updated, or not needed:
  • CONTENT_SCREENING_FRAMEWORK.md
  • GROUP_DOCUMENT_MANAGEMENT_APIS.md
  • PUBLIC_DOCUMENT_MANAGEMENT_APIS.md
  • GROUP_DOCUMENT_COLLABORATION_APIS.md
  • PUBLIC_DOCUMENT_ARTIFACT_APPROVAL_APIS.md
  • DOCUMENT_PROVENANCE.md
  • V2_GROUP_DOCUMENT_MANAGEMENT.md
  • docs/guides/review-screened-documents.md
  • Layer 3 rewrites the full screening scope.
• Fix documentation updated, or not needed: docs/explanation/fixes/UPLOAD_ONLY_SCREENING_INTAKE_FIX.md.

Security checklist

• New Flask routes include @swagger_route(security=get_auth_security()): no new routes. The two changed metadata routes keep their decorators and only lose the unreachable 202 branch; check_swagger_routes.py passed.
• Settings sent to non-admin frontends use sanitize_settings_for_user(): no settings responses changed.
• Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS: the V2 TypeScript changes are bundled locally by Vite into static/v2. No new dependencies or external scripts.
• No secrets, keys, connection strings, or local-only artifacts are included. The exemption field is server-managed. Clients can't set it, and is_public_document_field keeps it out of responses.