test: fix V2 test-harness drift (chat image refs, auto-open fixture, harness seeds)

[Paul Lizer (paullizer)]

Summary

• Test-only. On a clean V2 (paullizer-react-v2-ui at 70c255508, VERSION 0.261.213), several functional and UI tests fail because the application moved on and the tests didn't. All 12 changed files are under functional_tests/ and ui_tests/. There are no application, template, static or docs changes, no VERSION bump and no release notes, as in Fail pytest tests that return False (#1572) #1579. There is no end-user or admin impact.

• Kept current with V2. The branch has since merged V2 three times, with rerere off and no conflicts:

  • 520132dc2 (Phase 6a: workflow results reader and Follow up in chat #1592, Phase 6a) as be05eb54f
  • 86f3f3629 (Gate personal workflow instruction drafting on the AI workflow assistant setting (N9) #1596, N9) as 60fefae19
  • b61729627 (Clear the 19 py/log-injection CodeQL alerts on V2 #1597, log-injection CodeQL fix) as e8e3561a0

  Every file was rerun after each merge; see Reruns on later V2 tips. 6a needed one test-side binding fix in the group conversion harness (bf2940574). N9 and Clear the 19 py/log-injection CodeQL alerts on V2 #1597 needed none.

• Must 1: prepared_chat_image_references.

  • Show what was extracted from V2 image uploads, index text-free images, and default image generation to 2025-04-01-preview #1585 added this local to chat_api. The persistence harness's closed exec namespace now seeds it with the route's initial value, next to the other route locals, in one shared _chat_namespace.
  • This fixes the NameError in the 72 failing persistence cases and in the 2 composer-card tests that reuse that boundary.
  • An ast check compares the seed's keys with the route's dict literal, so the next drift fails with a readable message.

• Must 2: the auto-open plan fixture.

  • _plan() now names the conversation the plan was started for, and the mocked plan stream echoes the client's turn_id. The controller's onPlan requires both.
  • Every page check first asserts that the plan was adopted, so a rejected plan can't pass "drawer stays shut".
  • The file uses asserts and keeps its __main__ exit code.

• Should-fix. Eight more files are fixed at the root cause, all test-only:

  • Harness seeds use the real product helpers.
  • A leaked sys.modules stub is restored after its test.
  • The workflow gating check follows the full dependency chain.
  • The group conversion harness gets its M365 history runtime.
  • The revoked-record test follows the run-level access-loss contract.

  Items that need a product decision are under Not fixed.

Linked issue

Refs #1543

Release Notes & Latest Features

• New Feature
• Bug Fix
• UI Enhancement
• Breaking Change
• Internal only

Is this visible to end users?

• Yes
• No

Is this admin-facing (Admin Settings, governance, deployment, config)?

• Yes
• No

Should this become a Latest Feature card?

• Yes
• No
• Already added

Screenshot needed for the card?

• Yes
• No
• Attached

Version bump

• application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only. Not applicable: this PR is test-only. No application code changed, so VERSION is V2's own: 0.261.216 after the b61729627 merge. In each edited test file only the Version: header line changed, to the base's version at the time of the edit: 0.261.213 for the original fixes, and 0.261.214 for the group collaboration file's 6a binding fix. No exact-version assert was added.
• deployers/version.txt bumped, or not needed because deployers/ was not changed. deployers/ is not changed.

Testing / validation

Setup

• I used the pinned test venv (Python 3.12.10, Flask 3.1.3, pytest 9.0.3), run from the repo root with PYTHONPATH=application/single_app;functional_tests and PYTHONIOENCODING=utf-8. Only one test process ran at a time.
• Every run set PYTEST_ADDOPTS=-p no:langsmith_plugin. The venv's langsmith pytest plugin imports rich at startup, and its platform.system() WMI query can hang for minutes on this shared host. The variable was set only in my local runner and is not committed.
• Base counts come from a temporary detached worktree at 70c255508, which I removed afterwards.
• Playwright files ran against a fresh local SPA build: npm ci, then npx vite build, in application/v2_ui. The build output is gitignored. I rebuilt it after the 6a merge. Neither N9 nor Clear the 19 py/log-injection CodeQL alerts on V2 #1597 touches application/v2_ui, so the same build served both later reruns. Every browser count below is passed tests, with 0 skipped.

Commands

• Script: python -u <file>
• pytest: python -u -m pytest <file> -p no:cacheprovider -q -rfE -o faulthandler_timeout=150
• -O: the same two commands with python -O -u

Before and after: files this PR touches

These counts were measured before the V2 merges: the base is 70c255508, and "This PR" is ddb2d3362. The reruns after each merge are in the next section.

File	Mode	Base 70c255508	This PR (ddb2d3362)
functional_tests/test_v2_prompt_attachment_persistence.py (Must 1)	script (runs pytest.main)	72 failed, 11 passed	84 passed
	pytest	72 failed, 11 passed	84 passed
	-O script, -O pytest		84 passed, 84 passed
functional_tests/test_v2_prompt_composer_card.py (Must 1; not edited, it reuses the persistence boundary and the edited test_v2_prompt_composer_card_logic.ts)	script	9/11	11/11
	pytest	2 failed (NameError), 9 passed	11 passed
	-O script, -O pytest		11/11, 11 passed
ui_tests/test_v2_orchestration_auto_open.py (Must 2)	script	1/5	5/5
	pytest	4 failed, 1 passed	5 passed
	-O pytest		5 passed
functional_tests/test_app_settings_auxiliary_writers.py	script (runs pytest.main)	18 failed, 85 passed	103 passed
	pytest	18 failed, 85 passed	103 passed
	-O script, -O pytest		103 passed, 103 passed
functional_tests/test_block_revision_assist.py	script	7/8	8/8
	pytest	1 failed, 7 passed	8 passed
	-O pytest		8 passed
functional_tests/test_orchestration_turn_recovery_identifiers.py	script	5/6	6/6
	pytest	1 failed, 5 passed	6 passed
	-O pytest		6 passed
functional_tests/test_orchestration_plan_revision_routes.py (unittest)	script	Ran 29, FAILED (failures=1)	Ran 29, OK
	pytest	1 failed, 28 passed, 14 subtests passed	29 passed, 14 subtests passed
	-O pytest		29 passed
functional_tests/test_workflow_document_picker_recent_targets.py	script	3/4	5/5 (adds a leak test)
	pytest	1 failed, 3 passed	5 passed
	-O pytest		5 passed
functional_tests/test_v2_admin_workflow_parity.py	script	8/9	9/9
	pytest	1 failed, 8 passed	9 passed
	-O pytest		9 passed
functional_tests/test_group_collaboration_source_storage_fix.py (unittest)	script	Ran 18, FAILED (errors=83)	Ran 20, FAILED (failures=4), see Not fixed
	pytest	83 failed, 14 passed, 27 subtests passed	4 failed, 20 passed, 102 subtests passed
	-O script, -O pytest		the same 4 failures
ui_tests/test_v2_group_participant_invites.py (base: NameError: M365ApprovalRequired)	script (runs pytest.main)	4 failed, 2 passed	7 passed
	pytest	4 failed, 2 passed	7 passed
	-O pytest		7 passed
ui_tests/test_v2_workflow_loops.py (test_revoked_record_refresh_clears_the_previous_page; the file has no __main__)	pytest, whole file	1 failed, 72 passed	73 passed
	-O pytest, whole file		73 passed

Some counts grow because tests were added:

• persistence: 83 to 84 cases (a seed and drift test)
• doc picker: 4 to 5 tests (a leak test)
• group collaboration: 18 to 20 tests (drift and leak tests)
• UI invites: 6 to 7 cases (a drift test)

Reruns on later V2 tips

Each merge used git -c rerere.enabled=false merge origin/paullizer-react-v2-ui, and the #1597 merge also passed --no-ff, as the coordinator asked. None had a conflict or a rerere resolution.

be05eb54f merges 520132dc2 (#1592, Phase 6a; VERSION 0.261.214)

• 6a changed sources these harnesses load: admin_settings_fields.py, functions_collaboration.py, functions_orchestration_context.py, functions_orchestration_memory.py, functions_orchestration_workflow_context.py, route_backend_chats.py, Composer.tsx and WorkflowRunHistory.tsx.
• Its new line in test_v2_admin_workflow_parity.py's EXPECTED_DEPENDENCIES auto-merged. This PR's edits to that file are elsewhere.
• One test-side fix was needed. With the merge alone, group collaboration gave script failures=8, errors=58 and pytest 66 failed, 15 passed, 45 subtests passed. Its drift test named the cause: three 6a globals the harness didn't bind. bf2940574 binds them; see Changes by file and the mutation table. Every other file matched its pre-merge result.

60fefae19 merges 86f3f3629 (#1596, N9; VERSION 0.261.215)

• N9 changed route_backend_workflows.py (draft_workflow_instructions now calls is_workflow_assistant_enabled_for_user), route_frontend_workspace.py and templates/workspace.html. It added functional_tests/test_workflow_draft_instructions_assistant_gate.py.
• No test-side change was needed. No harness here execs draft_workflow_instructions or loads route_backend_workflows.py. The doc picker reads workspace.html, but only its picker markup and window.userGroups lines, which N9 didn't change.

e8e3561a0 merges b61729627 (#1597, log-injection CodeQL alerts fix; VERSION 0.261.216)

• Clear the 19 py/log-injection CodeQL alerts on V2 #1597 changed route_backend_plugins.py, utils_cache.py, route_backend_workflows.py and functions_appinsights.py. The admin plugin settings route (update_core_plugin_settings) no longer logs the request body. It now logs [PLUGINS] Received plugin settings update request. through log_event, with a field_count. On the test side, Clear the 19 py/log-injection CodeQL alerts on V2 #1597 added functional_tests/test_support/log_sanitizer.py and test_log_injection_codeql_alerts_fix.py, and edited test_group_document_collaboration.py and test_group_workflow_fixture_parity.py. None of those files is in this PR.
• No test-side change was needed. The aux-writers harness doesn't load route_backend_plugins.py. Its plugin cases exec repair_plugin and _find_plugin_class from plugin_validation_endpoint.py, with log_event captured, and Clear the 19 py/log-injection CodeQL alerts on V2 #1597 didn't change that file. Clear the 19 py/log-injection CodeQL alerts on V2 #1597's own test covers update_core_plugin_settings. No file in this PR references route_backend_plugins, utils_cache, route_backend_workflows, functions_appinsights or sanitize_log_message.

The results after the 6a, N9 and #1597 merges are identical in every cell. After 6a, the group file and the Playwright files ran at bf2940574. The other Python files ran at be05eb54f, since bf2940574 changes only the group file. After N9, everything ran at 60fefae19, and after #1597, at e8e3561a0.

File	Script	pytest	-O script	-O pytest
persistence	84 passed	84 passed	84 passed	84 passed
composer card	11/11	11 passed	11/11	11 passed
aux writers	103 passed	103 passed	103 passed	103 passed
block revision	8/8	8 passed	8/8	8 passed
turn recovery	6/6	6 passed	6/6	6 passed
plan revision	Ran 29, OK	29 passed, 14 subtests passed	Ran 29, OK	29 passed, 14 subtests passed
doc picker	5/5	5 passed	5/5	5 passed
admin parity	9/9	9 passed	9/9	9 passed
group collaboration	failures=4	4 failed, 20 passed, 102 subtests passed	failures=4	4 failed, 20 passed, 102 subtests passed
UI auto-open	5/5	5 passed	5/5	5 passed
UI invites	7 passed	7 passed	7 passed	7 passed
UI loops (no __main__)		73 passed		73 passed
N9's test_workflow_draft_instructions_assistant_gate.py (not edited; N9 and #1597 reruns)	12/12	12 passed	12/12	12 passed
#1597's test_log_injection_codeql_alerts_fix.py (not edited; #1597 rerun only; its __main__ runs pytest.main)	44 passed	not run	44 passed	not run

The group collaboration failures are the same 4 non_group_invitee subtests as before; see Not fixed.

The checks this PR leaves alone also match after all three merges:

• Admin schema: script 10/11, pytest 1 failed and 10 passed, -O pytest 1 failed and 10 passed. As an -O script it shows 11/11, but only because the failing check is a bare assert, which -O strips. pytest's rewritten assert still runs under -O.
• Doc picker leak check: 2 failed, 5 passed. These are the same 2 tag-filter failures that need Cosmos credentials.
• Strict mode: composer card 11 failed, block revision 8, turn recovery 6. All 25 failures are PytestReturnNotNoneWarning, with no other error.

Investigated and left alone

File	Mode	Base	This PR
functional_tests/test_v2_admin_settings_schema.py (not edited; see Not fixed)	script	10/11	10/11, unchanged
	pytest	1 failed, 10 passed	unchanged

The earlier 11, 8 and 6 counts

The earlier counts for composer card (11), block revision (8) and turn recovery (6) reproduce on base only when pytest treats PytestReturnNotNoneWarning as an error (-W error::pytest.PytestReturnNotNoneWarning).

• Every test in those three files ends with the template's return True for its __main__ runner.
• The pytest reports functional tests that return False as passed #1572 guard deliberately leaves True alone, and the repo's pytest.ini only declares the ui marker.
• In the default mode, the real base failures were 2, 1 and 1. All of them are fixed here.

File	Strict mode, base	Strict mode, this PR
test_v2_prompt_composer_card.py	11 failed	11 failed, all on the warning
test_block_revision_assist.py	8 failed	8 failed, all on the warning
test_orchestration_turn_recovery_identifiers.py	6 failed	6 failed, all on the warning

Cross-test leak check (document picker)

python -u -m pytest functional_tests/test_workflow_document_picker_recent_targets.py functional_tests/test_tag_filter_sanitization.py, both files in one process:

• Base: 3 failed, 3 passed. test_build_tags_filter fails with ImportError: cannot import name 'build_tags_filter' from 'functions_search' (unknown location), because the picker left its stub in sys.modules.
• This PR: 2 failed, 5 passed. These are the same 2 tag-filter failures the file has when it runs alone: a TypeError because it needs Cosmos credentials. Nothing leaks any more.

Mutation checks

Each mutation was applied in the working tree, run, then reverted, and the file hash was checked afterwards. None is committed. Each one fails with a readable message; none is skipped.

Fix	Mutation	Result
Must 1	Delete the prepared_chat_image_references seed	Persistence (pytest): 13 failed, 71 passed. Each boundary failure reads "The shipping persistence boundary now reads 'prepared_chat_image_references', which this harness's closed namespace does not define. Seed it next to the other route locals in the matching *_boundary namespace, using the value the route has at that point." Composer card (script): 9/11, with the same message. A targeted -O pytest run of test_chat_harness_seeds_the_route_initial_image_references fails too: "The chat harness namespace must seed prepared_chat_image_references: chat_api's new-message branch reads it before the user message is written."
Must 1	Rename the seed's input_fidelity key to fidelity	A targeted run of test_chat_harness_seeds_the_route_initial_image_references fails: "chat_api's prepared_chat_image_references keys drifted from ROUTE_IMAGE_REFERENCES_INITIAL: the route added ['input_fidelity'] and dropped ['fidelity']. Update the seed to the route's literal."
Must 2	Drop the fixture's turn_id	Script 1/5. pytest 4 failed, 1 passed. -O pytest 4 failed, 1 passed. Message: "the controller must adopt the streamed plan, but it holds 0 plan(s) and reported 'The planner returned an invalid plan. Please try again.'" Rerun at ddb2d3362, after the CI fix: script 1/5 and pytest 4 failed, 1 passed, with the same message.
Aux writers	Remove the resolve_action_type binding	18 failed: "repair_plugin must reach its settings write, but it logged ["[PLUGIN_REPAIR] Error repairing repair-me: name 'resolve_action_type' is not defined"]"
Block revision	Build the messages for a chart (block_kind='chart') instead of a diagram	1 failed: the positioning phrase is missing from the chart prompt the test read
Turn recovery	Remove the injected image_reference_provenance_from_seeds	Script 5/6. pytest 1 failed: "name 'image_reference_provenance_from_seeds' is not defined"
Plan revision	Three separate mutations: drop the explicit-selection patch; drop source_kind from the manifest stub; drop the strict-manifest patch	Each fails assertFalse with the route's refusal: "A selected document could not be opened…" for the first, "The selected document metadata could not be loaded…" for the other two
Doc picker	Remove the sys.modules restore	The leak test fails (script 4/5; pytest 1 failed): "Modules installed over stubs are still in sys.modules: ['functions_document_actions', 'functions_document_analysis', 'functions_search']"
Doc picker	Change the citations line inside _attach_workflow_search_context (a temporary product edit, reverted)	The scoped source check fails on list(search_context.get('citations') or [])
Admin parity	Two separate mutations: add an unrelated link to the field's chain; replace its require_group_assignment_for_group_workflows link	First: "…also gated on ['allow_user_workflows'], which 'require_group_assignment_for_group_workflows' does not itself require…". Second: "…gated on ['allow_group_workflows', 'allow_group_workflows'], expected 'require_group_assignment_for_group_workflows'"
Group collaboration (script; after the fix the file gives failures=4)	Seven separate mutations, listed below	Results listed below
Group collaboration, 6a binding fix (bf2940574; script)	Three separate mutations, listed below	Results listed below
UI invites	Three separate mutations, listed below	Results listed below
Workflow loops	Two separate mutations, listed below	Results listed below

Group collaboration mutations:

Mutation	Result
Remove the route seeds	failures=3, errors=27. The drift test lists the unbound names.
Remove the history wrapper loop	failures=9, errors=60 (M365PolicyError)
Remove the finally restore	failures=5, including "harness leaked its history service"
Remove the strip seed	failures=8, errors=26
Remove the cancel seed	failures=5, errors=6
Change the blob stub's signature	failures=4, errors=4 (TypeError)
Add a fake history callback	failures=112, "configure_m365_history_runtime should define…"

Group collaboration 6a binding mutations, run after the 6a merge. With the fix in place, the file gives failures=4. Each drift failure ends "Seed these production globals in ConversionHarness, or the code paths that read them fail with NameError."

Mutation	Result
Remove _carry_summary_into_collaboration from the functions loaded with ast	failures=8, errors=58. The drift test names it as read by ensure_personal_collaboration_for_legacy_conversation and ensure_group_collaboration_for_legacy_conversation.
Remove the message_uses_workflow_result binding	failures=8, errors=26. The drift test names it as read by _carry_summary_into_collaboration and _copy_legacy_personal_messages_to_collaboration.
Remove the withhold_workflow_result_message binding	failures=5: the 4 known failures plus the drift test, which names it as read by _copy_legacy_personal_messages_to_collaboration. No fixture message uses a workflow result, so only the static drift check catches this one.

UI invites mutations:

Mutation	Result
Remove the strip seed	5 failed, 2 passed, 4 errors, plus the drift message
Remove the image-URL seed	Only the drift test fails, including under -O
Remove the history wrapper loop	4 browser tests fail waiting for "was invited"

Workflow loops mutations:

Mutation	Result
Point the queued 403 at attempts/9	1 failed, 1 error. The access-loss alert never appears ("Locator expected to be visible"), and the fixture teardown reports "An expected failed request was never made."
Change the 403 to a 404	1 failed: "Locator expected to contain text 'You no longer have access to this workflow run.'" The actual text was "This workflow run history is no longer available. Cached run details were removed. Reopen this run to check access again."

Guardrails

After the merges, run at e8e3561a0: <venv python> run_v2_guardrails.py <worktree> b6172962777fd7a9678b00818c43a33752c8fe87 HEAD harness. The base is the PR's V2 merge-base, b61729627. Measured from 70c255508, the diff would also include the application changes from 6a, N9 and #1597. Exit code 0. Scope line: "12 changed files; app .py 0, xss surface 0, route .py 0, v2 ts/tsx 0". All 6 checks pass:

• python-syntax now reports "491 top-level single_app .py files compile". The 3 extra are 6a's new functions_workflow_result_followup.py, functions_workflow_result_masking.py and functions_workflow_result_reader.py. N9 and Clear the 19 py/log-injection CodeQL alerts on V2 #1597 added none.
• malicious-pr-review reports "No blocker findings detected in reviewed scope", with the same 8 advisory warnings on the same lines as before.
• The earlier run at 60fefae19, with base 86f3f3629, gave the same results.

Before the merges, with base 70c255508675bb0de7b7940c0ec90d5980f8b529, the results were the same. That ran at ddb2d3362, and earlier at 82cb3452a, where the auto-open warnings were one line higher. The table shows that run.

Check	Result
broken-access-control	PASS: "no changed application .py files"
xss-sinks	PASS: "no changed .js/.html/.py/.ts/.tsx/.jsx/.mjs application files"
swagger-routes	PASS: "no changed single_app .py files"
python-syntax	PASS: "488 top-level single_app .py files compile"
malicious-pr-review	PASS: "No blocker findings detected in reviewed scope." It raises 8 advisory keyword warnings, all on changed test lines (see below).
v2-ts-sinks (added lines)	PASS: "no sink patterns in added lines of 0 ts/tsx files". This counts application V2 .ts/.tsx only. The one changed .ts file is the test functional_tests/test_v2_prompt_composer_card_logic.ts.

The 8 advisory warnings, all on changed test lines:

• "AI, plugin, agent, or workspace boundary marker": ui_tests/test_v2_orchestration_auto_open.py L273, L276 and L279. These are the docstring, the _drive_plan(..., visible_conversation=OTHER_CONVERSATION) call and the assertion message of test_manual_plan_for_another_conversation_stays_shut, and they name a conversation.
• "security control, sanitization, or audit marker":
  • The same file, L295, L297 and L299: the _run script runner's try / except / traceback failure reporting.
  • ui_tests/test_v2_workflow_loops.py L909 and L916: the "Complete saved records" list locator and the access-loss alert filter.

CI

• First run, at 82cb3452a: the CodeQL results check failed with "1 new alert including 1 error". The alert was py/uninitialized-local-variable on ui_tests/test_v2_orchestration_auto_open.py L225: "Local variable 'page' may be used before it is initialized."
  • CodeQL doesn't treat pytest.skip() as non-returning, so it saw a path from the fixture's HarnessUnavailable handler to yield page.
  • The other checks passed: all three Analyze jobs (actions, javascript-typescript, python), enforce-branch-flow, malicious-pr-security-review and license/cla.
• Fix, in ddb2d3362: the yield page moved into the try statement's else clause, with no change in behavior.
  • After the fix: pytest 5 passed, script 5/5, -O pytest 5 passed (its 1 warning is pytest's own -O PytestConfigWarning, also seen before the fix). pyflakes is clean.
  • The skip path still works. A throwaway probe plugin made harness_page raise HarnessUnavailable, and the run gave 1 passed and 4 skipped, each with the reason "skipped the browser-driven checks: probe: v2_ui toolchain missing".
• At 60fefae19, after the 6a and N9 merges: all 7 checks pass:
  • Analyze (actions)
  • Analyze (javascript-typescript)
  • Analyze (python)
  • the CodeQL results check, with no new alerts
  • enforce-branch-flow
  • malicious-pr-security-review
  • license/cla
• At e8e3561a0, after the Clear the 19 py/log-injection CodeQL alerts on V2 #1597 merge: the same 7 checks pass. The CodeQL results check reports "No new alerts in code changed by this pull request".
• The access-control, python-syntax, swagger and XSS workflows are path-filtered to application/**, so they don't run on this test-only diff. The guardrails above run their checks locally.

Scope

• git diff --stat b61729627..HEAD, against the PR's V2 merge-base: 12 files changed, 646 insertions, 153 deletions, all under functional_tests/ or ui_tests/. Before the merges, 70c255508..ddb2d3362 was the same 12 files, with 639 insertions and 153 deletions. The 7 extra insertions are all in the group collaboration file, from bf2940574's 6a bindings. That commit's Version: change only rewrites a line the PR had already changed.

• No conftest.py, functional_tests/test_support/ or ui_tests/fixtures/ file changed. The pytest reports functional tests that return False as passed #1572 guard blocks are untouched.

• Two test modules also serve as harnesses for another test:

  • test_v2_prompt_composer_card.py reuses the persistence file's boundary.
  • ui_tests/test_v2_group_participant_invites.py imports ConversionHarness from functional_tests/test_group_collaboration_source_storage_fix.py.

  Neither is designated shared test infrastructure, so there is no fix doc.

Documentation

• Release notes updated, or not needed. Not needed: the PR is test-only, with no VERSION bump.
• Feature documentation updated, or not needed. Not needed.
• Fix documentation updated, or not needed. Not needed: no shared test infrastructure changed (conftest.py, functional_tests/test_support/, ui_tests/fixtures/).

Security checklist

• New Flask routes include @swagger_route(security=get_auth_security()). Not applicable: no routes were added or changed.
• Settings sent to non-admin frontends use sanitize_settings_for_user(). Not applicable: no settings code changed.
• Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS. Not applicable: no browser assets changed, and the UI tests serve the locally built SPA.
• No secrets, keys, connection strings, or local-only artifacts are included

Changes by file

functional_tests/test_v2_prompt_attachment_persistence.py (dc26690c5)

• Root cause: Show what was extracted from V2 image uploads, index text-free images, and default image generation to 2025-04-01-preview #1585 made chat_api read prepared_chat_image_references, and the boundaries now also read content-check and orchestration names. The closed exec namespace had none of them, so every chat-path boundary raised NameError.
• Fix: one shared _chat_namespace seeds the route's initial value next to the other route locals. The other names are bound to the real pure helpers, with chat checks off. No production statements are copied, so the L13 rule holds. A boundary NameError now becomes an AssertionError that names the missing local. A new test compares the seed with the route's literal using ast.

functional_tests/test_v2_prompt_composer_card_logic.ts (dc26690c5)

• Root cause: once the NameError was gone, the dispatchPlan mock still used the old planner contract, which normalizePlan and isPlanRunnable now reject.
• Fix: the mock now uses planner contract 2, with a runnable step.

ui_tests/test_v2_orchestration_auto_open.py (13cf075f3, ddb2d3362)

• Root cause: onPlan adopts a plan only if it has plan_id, run_id and turn_id, and only if its conversation_id equals the current conversation. _plan() set neither the conversation nor the turn.
• Fix: _plan() names conv, and the mocked stream echoes the client's turn_id. Each check first asserts adoption: one plan request for conv, and one held plan with the requested turn. The file is converted to asserts with a module-scoped harness page, and the __main__ exit code is kept. The fixture yields the page from the try statement's else clause, so the yield is reached only when the harness opened (see CI).

functional_tests/test_app_settings_auxiliary_writers.py (d5906cdf6)

• Root cause: since 73ff7d6, repair_plugin calls resolve_action_type and _find_plugin_class. The route's own except swallowed the resulting NameError and returned 500, so the test only saw assert 0 == 1.
• Fix: the test imports the shipped functions_action_manifest.resolve_action_type and extracts the real _find_plugin_class with ast. It also captures the route's log_event, so a missing name fails with the logged error.

functional_tests/test_block_revision_assist.py (a598ef3fb)

• Root cause: 26e8cae renamed ASSIST_SYSTEM_PROMPT to DIAGRAM_ASSIST_SYSTEM_PROMPT, so the test raised AttributeError.
• Fix: the test reads the system message that build_assist_messages actually sends for a diagram edit.

functional_tests/test_orchestration_turn_recovery_identifiers.py (82e1cc751)

• Root cause: after Show what was extracted from V2 image uploads, index text-free images, and default image generation to 2025-04-01-preview #1585, _persist_planned_turn calls image_reference_provenance_from_seeds, which the hand-built namespace didn't have.
• Fix: the real pure helper from functions_orchestration_context is injected through the existing import_app_module seam.

functional_tests/test_orchestration_plan_revision_routes.py (a55863564)

• Root cause: since 0401fdb, planning checks picked documents against the strict current-authority manifest. The harness stubs candidate discovery to ([], []), so planning refused with "A selected document could not be opened".
• Fix: for that one test only, the production explicit-selection branch of resolve_candidate_documents is restored (it makes no service call). The strict-manifest seam is answered from the test's authority stub, which now carries source_kind.

functional_tests/test_workflow_document_picker_recent_targets.py (75c9cfe32)

• Root cause: 778c10f moved the inline hybrid_citations update into _attach_workflow_search_context. Separately, the test left stub functions_search, functions_document_analysis and functions_document_actions modules in sys.modules.
• Fix: the source contract is now scoped with ast to the helper's body and the runner's call to it. A context manager installs the stubs and restores those three entries afterwards. A new test fails if any module installed over the stubs outlives its test.

functional_tests/test_v2_admin_workflow_parity.py (dffb8aadc)

• Root cause: group_workflow_allowed_group_ids now has a two-link depends_on chain, so the check crashed with "'list' object has no attribute 'get'".
• Fix: the check reads each link through the app's iter_field_dependencies. The expected gate must be a link with equals: True, and any other link must be one the gate itself requires. This PR doesn't touch EXPECTED_DEPENDENCIES, because 6a (Phase 6a: workflow results reader and Follow up in chat #1592) edits it. It was L56–61 on the base. 6a's new enable_chat_workflow_results entry auto-merged at L59, so it is now L56–62. The PR's hunks in this file are at L5 (the Version: line) and from L251 on.

functional_tests/test_group_collaboration_source_storage_fix.py (bb83f2ed1, bf2940574)

• Root cause: conversion now publishes M365 history, and the route catches M365ApprovalRequired and M365PolicyError. The shared ConversionHarness had no history runtime and didn't define those exception names. It failed with "History sharing authorization is not configured" and with NameError.
• Fix: the harness builds a real M365HistoryService from production's nested history callbacks, loaded with ast from configure_m365_history_runtime, over the in-memory stores. The service is scoped to each conversion and restored in finally.
  • The harness also seeds the route's M365 exceptions, strip_pending_action_references, a delivery-cancel recorder and the runs container.
  • Loud stubs guard the M365 publication paths these tests can't reach.
  • New drift tests fail on unbound globals and on a history service that leaks out of a conversion.
• After the 6a merge (bf2940574):
  • Root cause: 6a changed the collaboration functions the harness loads. Both ensure_*_collaboration_for_legacy_conversation functions now carry the summary through _carry_summary_into_collaboration. That helper, and _copy_legacy_personal_messages_to_collaboration, read message_uses_workflow_result. The copy function also reads withhold_workflow_result_message. The drift test named all three as unbound.
  • Fix: _carry_summary_into_collaboration is loaded from production with ast, like the other collaboration functions. The two masking functions are imported from 6a's functions_workflow_result_masking, which imports nothing from the app, and bound in the harness namespace. Version: is 0.261.214.

ui_tests/test_v2_group_participant_invites.py (bb83f2ed1)

• Root cause: the test uses that ConversionHarness, and the helpers it serves read globals that were never bound, hence NameError: M365ApprovalRequired.
• Fix: the test binds the globals its served helpers read: the real strip_private_chat_checks, plus loud image stubs. It has its own drift test.
• After the 6a merge: the file isn't edited. Its served helpers are built from the harness namespace, so they inherit bf2940574's bindings, and its 7 cases pass.

ui_tests/test_v2_workflow_loops.py (82cb3452a)

• Root cause: since ad8f421, a 403 on a paged read calls onAccessLost. WorkflowRunHistory then replaces the expanded run with a run-level alert, so the "Complete record inspection" region unmounts.
• Fix: the test asserts the run-level contract, as test_v2_workflow_publication_completion.py does: the 403 alert shows, the queued failure was consumed, and the panel, the records list and the old record text are gone.

Decisions made autonomously

1. One shared seed for Must 1. The seed is in the shared _chat_namespace that every chat-path boundary uses. All six persistence paths are fixed, not only the two the composer card reaches.
2. Missing local names fail readably. A boundary NameError is re-raised as an AssertionError that names the missing local and says where to seed it.
3. The composer-card TS mock moved to planner contract 2. This was a second drift, hidden behind the NameError. It's in the same commit as Must 1.
4. The auto-open stream echoes the client's turn_id, as the server does, instead of a fixed literal. The adoption check is stricter than "planCount == 1": it requires one plan request for the started conversation and one held plan with the requested turn. A module-scoped fixture reuses hb.harness_page.
5. Missing names are bound to real helpers, never copies. Each one is the shipped function, loaded by import or ast:
   • resolve_action_type
   • _find_plugin_class
   • image_reference_provenance_from_seeds
   • iter_field_dependencies
   • strip_private_chat_checks
   • the production M365 history callbacks
6. Plan revision fix is scoped. The explicit-selection branch and the strict-manifest seam are patched only for the one failing test, not in the shared base harness, so the other 28 tests keep their setup.
7. The group collaboration functional test is fixed too, although the list didn't include it. The UI invites failure starts in its ConversionHarness, and its 83 errors have the same root cause.
8. The revoked-record test follows the product. Its new contract is the deliberate run-level access-loss behavior from ad8f421, not the old panel-scoped alert.
9. The admin schema item is reported, not fixed. As the brief warned, it is a product mismatch, so I didn't edit the expectation.
10. The strict-mode counts are explained, not "fixed". return True stays; see the 11, 8 and 6 section above.
11. Base counts come from isolated runs, one file per process, on a detached base worktree.
12. The langsmith pytest plugin is disabled only in my local runner (PYTEST_ADDOPTS), because of the WMI startup hang on this shared host. Nothing about it is committed.
13. V2 merges, only when asked. At the coordinator's request I merged 520132dc2 (Phase 6a: workflow results reader and Follow up in chat #1592, as be05eb54f), 86f3f3629 (Gate personal workflow instruction drafting on the AI workflow assistant setting (N9) #1596, as 60fefae19) and b61729627 (Clear the 19 py/log-injection CodeQL alerts on V2 #1597, as e8e3561a0, with --no-ff). Each used git -c rerere.enabled=false merge, with no rebase, and was pushed as a fast-forward. The PR stays a draft.
14. 6a's new helpers are bound from production, in the harness. _carry_summary_into_collaboration is loaded with ast, like the other collaboration functions. The masking functions are the real ones from functions_workflow_result_masking, which has no app imports. Binding them in ConversionHarness fixes the functional file and the UI invites file together, so the UI file needed no edit.

Not fixed (with causes)

1. functional_tests/test_v2_admin_settings_schema.py::test_defaults_match_their_field_type (script 10/11; pytest 1 failed).
   • Cause: in application/single_app/admin_settings_fields.py, the core-plugin-toggles field m365_trusted_download_hosts is declared "type": "textarea", which the test maps to str, but its "default" is []. The settings default in functions_settings.py is also []. The failure reads "core-plugin-toggles.m365_trusted_download_hosts: default [] is not str".
   • Whether the field type or the default should change is a product decision, so I didn't edit the expectation.
2. functional_tests/test_group_collaboration_source_storage_fix.py: the 4 non_group_invitee subtests. These are test_rejections_do_not_read_history_or_mutate_stores and test_v1_route_preserves_rejection_statuses_without_mutation, each for both storage types.
   • Cause: a rejected invitee shouldn't cause a history read. In functions_collaboration.py (line numbers as of the b61729627 merge), ensure_personal_collaboration_for_legacy_conversation (L1060–1064) and ensure_group_collaboration_for_legacy_conversation (L1278–1284) call prepare_m365_history_publication first. It reads the source history with SELECT * FROM c WHERE c.conversation_id = @id ORDER BY c.timestamp ASC. Only afterwards does create_*_collaboration_conversation_record validate the invitees, for example through _normalize_group_conversation_participants. That order was introduced in b8a7541, and 6a, N9 and Clear the 19 py/log-injection CodeQL alerts on V2 #1597 didn't change it.
   • The test is right. The fix is to reorder product code.
3. Strict-mode PytestReturnNotNoneWarning failures (11, 8 and 6) in the composer-card, block-revision and turn-recovery files.
   • Cause: they come from the template's return True for the __main__ runner, not from drift.
   • Removing it would break each runner's exit code. It belongs with the tracked conversion of template-style tests, and the repo's pytest.ini doesn't turn the warning into an error.

Follow-ups

• Done: after 6a (Phase 6a: workflow results reader and Follow up in chat #1592) landed, all 12 files were rerun on the merge, and again after N9 (Gate personal workflow instruction drafting on the AI workflow assistant setting (N9) #1596); see Reruns on later V2 tips. 6a's changes to route_backend_chats.py around chat-turn persistence didn't affect the persistence or composer-card suites.
• Done: rerun after Clear the 19 py/log-injection CodeQL alerts on V2 #1597 landed, at e8e3561a0; every cell matches. This corrects an earlier note here: the aux-writers harness doesn't bind route_backend_plugins.py's plugin route. It binds repair_plugin and _find_plugin_class from plugin_validation_endpoint.py, which Clear the 19 py/log-injection CodeQL alerts on V2 #1597 didn't change. Clear the 19 py/log-injection CodeQL alerts on V2 #1597's own test covers the route's new logging and passes 44/44.
• No fixture message in the group collaboration harness uses a workflow result, so only the static drift test guards the withhold_workflow_result_message binding. A fixture message with a workflow result would add behavioral coverage.
• Product: decide the field type versus the default for m365_trusted_download_hosts.
• Product: validate invitees before prepare_m365_history_publication reads history.
• The base plan-revision harness's resolve_candidate_documents stub ignores explicit selections. A future test that picks documents needs the same scoped helper, or the base harness could adopt it.
• The group harness's fake runs container doesn't filter on has_active_request.
• Workflow UI suites pytest.fail unless application/single_app/static/v2 is a fresh local Vite build. A note in ui_tests.instructions.md would help.
• In one process, the auto-open file's module-scoped hb.harness_page may clash with pytest-playwright's page fixture.
• On Windows hosts, the langsmith pytest plugin's rich import can hang in a WMI query before collection; -p no:langsmith_plugin avoids it.
• test_tag_filter_sanitization.py needs Cosmos credentials, even for its pure checks.
• The doc-picker and loops files have pre-existing pyflakes warnings; the count is unchanged.
• semantic_kernel raises a Pydantic deprecation warning at import.
• The malicious-PR review raises advisory markers on test docstrings and runner lines.