Clearer saved-result error messages and repaired workflow test harnesses

[Paul Lizer (paullizer)]

Summary

• Clearer messages. AnalysisResultUnavailable used to have one message, "This analysis is unavailable because its source access could not be confirmed." That wording dates from when saved results re-checked their sources. It now picks its message by family:
  • Source: an input read can't read a document now, or the document changed.
  • Container: the conversation, workflow or run that holds the result is gone or isn't yours.
  • Saved result: the result, or something it depends on, is missing or changed.
• The classification lives in explicit frozensets in functions_analysis_access.py. Codes were classified by what each raise site checks, not by name.
• These places now show the family's message: the workflow runner's task error, the "result not saved" error, the loop item pause, saved-analysis explanation, Analyze save, the Analyze publication check, workflow progress, and the history placeholders.
• No behavior change. The exception class, raise and catch sites, HTTP statuses and response code values are unchanged. The one exception is two former bare raises, which now carry container codes that map to the same response codes.
• Repaired test harnesses. Six stale workflow test setups were fixed without loosening any assertion. 38 previously failing tests now reach their assertions and pass, including test_native_source_revocation_does_not_block_the_saved_report, the proof of Stop re-checking saved workflow results against their source documents (screening layer 3a) #1631's behavior.

Linked issue

Refs #1621

Release Notes & Latest Features

• New Feature
• Bug Fix
• UI Enhancement
• Breaking Change
• Internal only

Is this visible to end users?

• Yes
• No

Is this admin-facing (Admin Settings, governance, deployment, config)?

• Yes
• No

Should this become a Latest Feature card?

• Yes
• No
• Already added

Screenshot needed for the card?

• Yes
• No
• Attached

Version bump

• application/single_app/config.py VERSION third segment bumped (0.261.233 → 0.261.234; the base now ships 0.261.233), or not needed because this is docs-only
• deployers/version.txt bumped, or not needed because deployers/ was not changed

Testing / validation

Message families (task 1)

• python -m pytest functional_tests/test_analysis_unavailable_message_families.py → 39 passed. The file:
  • scans every AnalysisResultUnavailable(...) raise in application/single_app and fails on an unclassified code, a bare raise, an unknown family= value or an unresolved computed code. The exc.code raise in _authorize_frozen_document is resolved by walking the WorkflowLoopInputError codes that reauthorize_workflow_loop_document can raise.
  • checks each family's message and that codes are unchanged.
  • runs the real runner sequence for each family, for both the task error and the "not saved" error.
  • checks the chat explanation, save and publication routes, which all return 403 with the family message.
  • checks that the renamed codes keep every response code.
• python -m pytest functional_tests/test_workflow_durable_routes.py → 13 passed. Workflow progress names a failed saved record, and a group membership refusal keeps its access message.

Harness repair (task 2)

• Per-file before/after: each file runs in its own pytest process, with application/single_app on PYTHONPATH, over 468 related files. Before is an untouched worktree of e1e6949; after is this branch.

  • 287 failing nodes before, 249 after. 0 new failures, 38 fixed.
  • test_tabular_row_orchestration_scale.py hits the 25-minute timeout in both runs.

  File	Before (e1e6949)	After
  test_workflow_task_result_handoff.py	9 failed	pass
  test_workflow_data_flow_execution.py	8 failed	pass
  test_workflow_m365_rebase_integration.py	9 failed	pass
  test_workflow_runtime_integration.py	4 failed	pass
  test_analyze_workflow_publication_integration.py	8 errors	pass

• Round-trip harness, without PYTHONPATH: the 13 files built on it went from 1,857 failing tests to 0. The failure was No module named 'functions_workflow_chat_delivery'. All 13 also pass with PYTHONPATH set.

• Isolation: test_analyze_artifact_phase7_rollout_rollback.py ran in one pytest process with the 6 files it used to break. Before: 5 collection errors and an interrupted run. After: 101 passed.

After merging the base (f1aeef1)

• 23 files, each in its own pytest process with PYTHONPATH set, all pass: 868 passed, 0 failed. They are the new family test, the 5 repaired files above, every other test file this PR touches, the 7 other consumers of the inventory harness, and the 5 test files the base changed or added.
• The 13 round-trip harness files, without PYTHONPATH: 1,871 passed, 0 failed.
• Isolation, in one pytest process: 101 passed.
• The checks below were rerun after the merge, against f1aeef1.

Checks

• python scripts/check_xss_sinks.py --base-sha f1aeef13d --head-sha <head> <11 app files> → passed
• python scripts/check_broken_access_control.py --base-sha f1aeef13d --head-sha <head> <11 py files> → passed
• python scripts/check_swagger_routes.py <11 py files> → passed (2 route files)
• python functional_tests/route_tests/test_route_blueprint_policy_inventory.py → 11/11
• python functional_tests/route_tests/test_route_unauthenticated_policy_contract.py → 6/6
• python functional_tests/route_tests/test_route_policy_test_coverage.py → 2/2
• python scripts/build_docs_inventory.py → no content change. It produced a line-ending-only diff, which was restored.
• python functional_tests/test_docs_app_surface_coverage.py → 7/7
• python functional_tests/test_docs_site_quality.py → 6/6
• git diff --check → clean
• No V2 files changed, so V2 typecheck and build were not needed.

Documentation

• Release notes updated: new v0.261.234 section with a Bug Fixes entry, above the base's v0.261.233 section, which is unchanged. The harness repair is internal and not listed.
• Feature documentation updated: the public messages table in UPLOAD_ONLY_CONTENT_SCREENING.md.
• Fix documentation updated: ANALYSIS_RESULT_UNAVAILABLE_MESSAGES_FIX.md and WORKFLOW_TEST_HARNESS_REPAIR_FIX.md.

Security checklist

• New Flask routes include @swagger_route(security=get_auth_security()) (no new routes)
• Settings sent to non-admin frontends use sanitize_settings_for_user() (not touched)
• Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS (no browser changes)
• No secrets, keys, connection strings, or local-only artifacts are included

Decisions and notes for review

Shared codes

• Three raise sites share a code with another family, so they pass family=:
  • resolve_analysis_source_manifest raises analysis_source_manifest_invalid → source. The current lookup failed during an input read.
  • index_analysis_source_manifest raises the same code for an unauthorized assigned document → source.
  • load_workflow_artifact_binding raises generated_artifact_source_unavailable → saved result. The run records couldn't load; the scope checks keep container.
• Bare raises:
  • authorize_analysis_sources and validate_sources now pass analysis_source_unavailable explicitly.
  • _load_authorized_message → analysis_message_unavailable, and _load_authorized_workflow → analysis_workflow_unavailable. Nothing branched on their old default.

Wording

• The runner keys its text by the family's string value, falling back to source text. 70 functional test files compile the runner from source, so this avoids adding names to their namespaces. Unclassified codes fall back to the saved-result wording.
• History placeholders for a saved analysis or saved workflow output no longer say "access could not be confirmed".

Left as is

• These stay unchanged because their response codes are shared with genuine access refusals: the V2 text for workflow_result_access_denied, and generated file output_access_denied. The native bridge's "ownership or source access changed" also stays, because it describes an active computation's input check.

Harness repairs

• The 4 test_workflow_runtime_integration.py failures came from WORKFLOW_SCHEDULED_TRIGGER_TYPES, which _run_authorized_workflow_impl now reads. That fixture already replaces _ensure_execution_context. The session / read_user_settings_snapshot fix was needed for the data-flow entrypoint test.
• The functions_workflow_chat_delivery import failure from Stop re-checking workflow run sources in run history and after a task saves #1628 is the group round-trip harness. That harness loads real modules by file and didn't list the chat delivery contract. With PYTHONPATH set it didn't fail, but it left a stub-bound copy of the contract in sys.modules. Both are fixed.

Merge and renumbering

• paullizer-react-v2-ui at f1aeef1 (Fix orchestration file generation when /app is root-owned (V2) #1626, Update roadmap status: Phase 7's 7a design, and container-only results and self-authored Run as #1633) was merged in with a merge commit, not a rebase. Only release_notes.md conflicted.
• The base ships its own 0.261.233, so this PR is now 0.261.234. A separate commit moves VERSION to 0.261.234 and relabels only the 0.261.233 labels this PR added: both fix docs, the Upload-Only Content Screening notes and the test headers. The base's own 0.261.233 references are unchanged.

Push

• git push origin returned 403: the current token has push: false on microsoft/simplechat. This branch was pushed to the paullizer/simplechat fork, as layers 2, 3a and 3b were.

Still failing, out of scope and unchanged

• test_content_screening_pipeline.py and test_content_screening_reviews.py.
• Tests that need Azure config, V2 node_modules or the dev server.