Skip to content

Phase 6c: in-plan wait for a quick saved workflow - #1732

Merged
Paul Lizer (paullizer) merged 26 commits into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-phase-6c-plan-waits-for-quick-workflow
Oct 9, 2026
Merged

Paul Lizer (paullizer) merged 26 commits into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-phase-6c-plan-waits-for-quick-workflow

Conversation

@paullizer

@paullizer Paul Lizer (paullizer) commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Phase 6c, in-plan wait. With the new admin setting Wait For Quick Workflows In Chat (enable_chat_orchestration_workflow_run_wait, default off), a chat plan can start one of the user's saved quick personal workflows (workflow_run), wait for that run to finish, and use its result in later compose steps of the same plan. Example: "Run my sales-digest workflow, then compare its totals with the Q3 report in my workspace." Before this change, the planner dropped any run step whose output a later step consumed, and the result only came back as a separate 6b post.
  • The server decides, never the model. quick_run_eligibility() derives a deterministic quick-run rule from the stored workflow and the settings. It's checked when the plan is normalized (the catalog's waitable flag), again when the run starts, and on every continuation. A model-written workflow_run_waits marker is always discarded.
  • Bounded wait. The wait lasts at most min(admin cap, the plan's remaining budget minus the time it needs to write the answer). The cap is chat_orchestration_workflow_run_wait_max_seconds, default 300 s, range 60–1800.
    • A run that finishes in time reaches compose only through the existing workflow_results reader, which re-authorizes and fences the content as untrusted.
    • A failed or cancelled run says so, and the existing dependency-failure rules apply.
    • On a timeout, the step says "Still running. Its result will be posted to this chat when it finishes." The plan completes without failing, and the existing 6b post-back delivers the result later.
  • Exactly one delivery. An ETag compare-and-set on the run document's chat_delivery record decides whether the plan's answer or the 6b post-back delivers the result. That's the same record the 6b worker claims, and the decision lives in its plan_wait field. A consumed result is marked delivered / used_in_answer with no notice, so nothing posts and no bell fires, and the V2 run card says "Its results were used in a chat answer."
  • Off is unchanged. With the setting off, planner normalization and execution are byte-identical to before, shown by an off-golden test captured from the base commit. Run plans stay on the manual approval floor, and hand-off (one_time) workflows are never waitable. No new routes.

For Phase 8 replay (#1724): the wait kind is exported as SAVED_WORKFLOW_RUN_WAIT_KIND = 'saved_workflow_run' in application/single_app/functions_orchestration_workflow_run_wait.py. The plan marker shape is plan['workflow_run_waits'] = {step_id: {'version': 1, 'workflow': <catalog handle>}}.

Linked issue

Part of #1546
Refs #1543

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
    • 0.261.301 (base) → 0.261.309 (final), in its own final commit. Originally assigned .307; after V2's tip landed at .307 (PR Render Microsoft 365 pending-action cards in V2 chat (#1722 gap 2) #1731) and the sibling Phase 8 replay branch claimed .308, the coordinator reassigned this PR to .309. Every self-authored .307 reference (test headers/assertions, docs, the release-notes header) was renumbered to .309 in its own preceding commit, with config.py left untouched there; the VERSION bump is the strictly final commit.
  • deployers/version.txt bumped, or not needed because deployers/ was not changed
    • Not bumped: nothing under deployers/ changed.

Testing / validation

All commands use the pinned test venv (...\aa1bf1e0-b10d-4312-8dbb-143e9e9f9966\files\venv\Scripts\python.exe) with PYTHONPATH=application/single_app;functional_tests, PYTHONIOENCODING=utf-8, PYTHONUTF8=1, and -p no:langsmith_plugin -p no:cacheprovider. Every new Phase 6c test file was also run as pytest -O, as a plain script, and as an -O script; all four ways pass (only the plain-pytest counts are itemized below).

New Phase 6c tests — all 8 files, one process:

  • test_orchestration_workflow_run_wait_{off_golden,eligibility,plan_rules,planner,execution,delivery,claims,results}.py → 170 passed, 0 failed

Neighbours (one file/group at a time, per the shared-machine rule):

  • 10 non-wait test_orchestration_workflow_run* files → 458 passed
  • 7 test_orchestration_workflow_results* files → 190 passed
  • 13 test_workflow_chat_delivery* files → 383 passed
  • 7 native-wait/continuation files (test_orchestration_native_waiting_continuation, test_orchestration_native_results, test_orchestration_native_infrastructure_runtime, test_orchestration_native_infrastructure_failures, test_orchestration_native_checkpoint_receipts, test_orchestration_native_adapter, test_orchestration_continuation) → 277 passed
  • test_orchestration_actions_admin.py + test_admin_settings_field_contract.py → 10 passed, 14 subtests passed
  • A broader 111-file sweep across admin-settings, orchestration, workflow-handoff, workflow-results, workflow-run, workflow-chat-delivery and V2-admin-settings files → 100 passed; 11 tests failing across 10 files, plus 1 slow timeout. Every failure and the timeout were triaged against a temporary detached worktree of base 817f9dd07 and confirmed identical — see Known base failures.

Docs:

  • python scripts/build_docs_inventory.py → regenerated docs/_data/app_surface.yml
  • test_docs_app_surface_coverage.py (__main__) → 7/7 passed
  • test_docs_site_quality.py (__main__) → 6/6 passed
  • test_docs_link_integrity.py (__main__) → 2/5 passed — identical on base, see Known base failures
  • test_docs_release_notes_integrity.py (__main__) → 0/1 passed — identical on base, see Known base failures

V2 typecheck/build:

  • npm --prefix application/v2_ui run typecheck → clean
  • npm --prefix application/v2_ui run build → clean
  • npm --prefix application/v2_ui run build -- --outDir ../../ui_tests/artifacts/orchestration-plan-editor → clean (the second bundle test_v2_workflow_run_wait.py needs)

V2 unit tests (Node's test runner, .mjs):

  • test_v2_workflow_run_status.mjs + test_v2_workflow_run_tracker.mjs → 61 passed (both modified, to add the used_in_answer delivery reason)
  • test_v2_orchestration_run_progress_status.mjs → 12/12 passed (modified, to add the waiting-workflow line)

V2 Playwright (ui_tests/, pytestmark = pytest.mark.ui), one file at a time, free memory checked ≥3 GB first:

  • test_v2_workflow_run_wait.py (new) → 7/7 passed
  • test_v2_notifications_bell.py → 10/10 passed
  • test_v2_workflow_run_card.py (pre-existing, unmodified) → 40/41 passed — the 1 failure is pre-existing on base, see Known base failures
  • test_v2_workflow_run_tracker_spa.py (pre-existing, unmodified) → 5/6 passed — the 1 failure is pre-existing on base, see Known base failures
  • test_v2_orchestration_plan_card.py (pre-existing, unmodified; run as python test_v2_orchestration_plan_card.py, since it predates the pytest-fixture convention and drives its own __main__) → 6/7 passed — the 1 failure is pre-existing on base, see Known base failures

Guardrails: 6/6 PASS, 0 blockers — see the Guardrails section below.

Post-merge catch-up (V2's tip merged in at push time, 08def99f7, PR #1731):

All validation above ran pre-merge. After merging origin/paullizer-react-v2-ui (which had advanced 14 commits past this branch's prior base — M365 pending-action cards (#1731), the V2 agents catalog (#1729), a scope-response-contract fix, and a group-switch-latency fix) and renumbering .307 → .309, the coordinator asked for a "light catch-up," not a full re-run. This is what ran, all green:

  • py_compile on every file the merge auto-resolved or touched (functions_orchestration_composition.py, functions_orchestration_continuation.py, config.py) — clean.
  • test_orchestration_workflow_run_wait_off_golden.py → 9/9 passed (the single most load-bearing guard: setting off ⇒ unchanged behavior).
  • Eligibility + execution + delivery files together → 88/88 passed.
  • npm --prefix application/v2_ui run typecheck → clean; run build → clean (free memory checked first: 4.99–5.30 GB, all above the 3 GB floor).
  • python scripts/build_docs_inventory.py → no diff (the inventory was already in sync post-merge; no hand-merge of app_surface.yml was needed).
  • test_docs_app_surface_coverage.py (__main__) → 7/7 passed. test_docs_site_quality.py (__main__) → 6/6 passed.
  • test_docs_link_integrity.py (__main__) → 2/5, and test_docs_release_notes_integrity.py (__main__) → 0/1 — both confirmed unchanged from the pre-merge known-base-failure baseline above.
  • Guardrails re-run against the new base 08def99f7 (see the Guardrails section) — 6/6 PASS, 0 blockers, identical scope.

Not re-run, by design: the full 170-test Phase 6c suite, the 111-file neighbour sweep, and the Playwright suite. The merge touched no Phase 6c runtime file directly (config.py and functions_orchestration_composition.py auto-merged on disjoint hunks; release_notes.md was the only real conflict, resolved by keeping every section from both sides), so the pre-merge full results above still describe the code that shipped.

Documentation

  • Release notes updated, or not needed
    • A new ### **(v0.261.309)** section at the top of docs/explanation/release_notes.md (renumbered in place from .307 once the coordinator reassigned the version — content unchanged). Merging V2's tip (08def99f7) brought in V2's own ### **(v0.261.306)** and ### **(v0.261.305)** sections directly beneath it; both are preserved byte-identical, as is every older section.
  • Feature documentation updated, or not needed
    • New docs/explanation/features/CHAT_ORCHESTRATION_WORKFLOW_RUN_WAIT.md.
  • Fix documentation updated, or not needed
    • Not needed: this PR adds a feature; it doesn't fix a filed bug.
  • Admin docs, docs/admin/orchestration.md:
    • a capability row after Run Workflows From Chat
    • a "Waiting for a quick workflow" subsection
    • a Limits row for the cap
    • troubleshooting rows
  • docs/_data/features.yml claims enable_chat_orchestration_workflow_run_wait, and docs/_data/app_surface.yml was regenerated with scripts/build_docs_inventory.py.
  • User guide: docs/guides/trigger-a-workflow.md has a new subsection, "Use a quick workflow's result in the same answer".
  • docs/reference/chat-controls.md is unchanged because no V2 controls changed. CHAT_ORCHESTRATION_WORKFLOWS_ROADMAP.md is untouched.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security())
    • N/A: no new routes. route_frontend_admin_settings.py is the one route file touched, only to save/coerce the two new keys inside its existing handler; its route signatures and @swagger_route(security=get_auth_security()) decorators are unchanged, confirmed by the swagger-routes guardrail.
  • Settings sent to non-admin frontends use sanitize_settings_for_user()
    • The two new keys are read server-side; the admin settings form is an admin route and isn't sanitized, per the repo rule.
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
    • The V2 changes are TS sources built into the local static/v2 bundle.
  • No secrets, keys, connection strings, or local-only artifacts are included

Authorization and untrusted content

  • Re-authorization on every continuation (_reauthorize_wait). Each check confirms:

    • the wait and Run Workflows From Chat are still configured, and the user still passes the run and results gates
    • the conversation is still the user's, not deleted, and still private (refresh_workflow_planning_privacy)
    • the workflow is still the user's own personal workflow, not being deleted, and still quick

    Any failure releases the hold and ends the wait. The step then says "stopped waiting", and 6b's own rules decide whether the result is posted.

  • Content. Workflow output reaches the plan only through functions_workflow_result_reader.py, through the existing workflow_results compose path. Each read re-authorizes and checks the sha256 of what was consumed, and the text stays fenced as untrusted. The step output itself holds only ids, status, timestamps and a result pointer, never content.

  • XSS. Workflow names render as React text nodes only. The server summary is built with clean_catalog_text(), and hostile names are covered by ui_tests/test_v2_workflow_run_wait.py.

Design decisions (D1–D7)

D1. Quick-run rule and cap.

  • The rule is code constants, not settings. Only the cap is a setting. It's evaluated by quick_run_eligibility(workflow, settings), which reads nothing else and refuses anything it can't read. A workflow is quick only if all of these hold:
    1. The wait is configured. The switch is on, along with personal workflows, Run Workflows From Chat and Use Workflow Results In Chat.
    2. It's personal, with no group_id. A group workflow isn't the user's alone to wait for.
    3. durable_execution is true. Only a durable run records its progress where a later continuation can read it.
    4. It's not being deleted, and not paused on Microsoft 365 (M365_ACTIVE_STATES). Those runs wait for a person or for sign-in.
    5. It's not a one-time hand-off (one_time, top-level or in origin). A hand-off is large work sent out of the chat on purpose, and its result is always posted back.
    6. No For each or Repeat until anywhere in the flow; a flow nested too deep to inspect also refuses. These loops run as long as the data makes them.
    7. It's not a structured (v3/flow) workflow. Its result can't be read back into a chat answer by the workflow_results reader.
    8. No task with approval.required. Approval waits for a person.
    9. No m365_run_as_user_id. That can pause for the user's sign-in.
    10. No task waits on a review or indexing completion policy. Those wait on other people or other systems.
    11. No File Sync, either as the trigger or a pre-step. It waits for files to sync.
    12. One to five tasks (QUICK_RUN_MAX_TASKS = 5). A small task count keeps a run inside a few minutes, given the per-task timeouts.
  • Plan rules on top, computed by the server in compute_workflow_run_waits:
    • exactly one enabled, non-optional workflow_run step
    • no step that needs the user's live session: web_search, url_fetch, deep_research, agent_invoke, action_invoke, workflow_handoff or workflow_propose
    • every step capability must be one a headless continuation can still run
    • only compose steps may read the run's result
    • at least one enabled compose binds it through a required input
    • the final response doesn't select the run step itself
    • require_member_of_workflow_user must be off
  • The cap is a setting, chat_orchestration_workflow_run_wait_max_seconds, default 300 s, range 60–1800 s.
    • The 5-minute default covers the short digests people run from chat. It also keeps the wait inside the default 15-minute run timeout once the answer reserve is subtracted: 900 − (90 + 180) = 630 s is more than 300 s, so the cap applies.
    • The minimum of 60 s equals WAIT_MIN_SECONDS. Anything shorter is dominated by scheduler latency; see D4.
    • The maximum of 30 minutes matches the longest step timeout. Longer runs are already covered by the 6b post-back.
  • Bound formula: deadline = min(now + cap, plan_deadline − (90 s + step_timeout × steps that use the result, transitively)). There's no wait at all when less than 60 s would be left.

D2. Where the "consumed in plan" mark lives.

  • It lives on the workflow run document's existing chat_delivery record, which is the record the 6b worker claims, in a new chat_delivery.plan_wait field. That field holds {version, state: holding|consumed|released, orchestration_run_id, step_id, until, held_at, consumed_at|released_at, updated_at}.
  • Every hold, consume and release is a read–decide–replace with the run document's _etag (MatchConditions.IfNotModified), retried on 412.
  • Hold. A hold is placed only on an unposted record, and it lapses on its own at deadline + 120 s.
  • The 6b worker defers its next attempt to until while a hold is live. It never posts a consumed record, because consuming writes status: delivered, notice_kind: none, outcome_reason: used_in_answer and message_id: null.
  • The bell notifies only on a posted notice, so a consumed result produces no bell.
  • The V2 run card reads delivery.reason === 'used_in_answer' from the existing status projection, which gained one key, and shows "Its results were used in a chat answer."

D3. What later steps and the final answer see.

Outcome Run step compose input Answer note 6b
Finished in time Completes with wait.outcome = consumed: run id, status, timestamps, result pointer The run's result, re-read through the reader (re-authorized, sha-checked, fenced untrusted) "Ran X. Its results were used in this answer." Never posts
Failed / cancelled Fails with workflow_run_failed / workflow_run_cancelled Blocked by its required input, under the existing dependency-failure rules "…failed, so its results couldn't be used…" / "…was cancelled before it finished…" Never posts; the run card shows the failed or cancelled status
Timed out Completes: "Still running. Its result will be posted to this chat when it finishes." An app-owned fenced note saying it's still running, with no content "Started X. It was still running when this answer was written." Posts once when the run finishes
6b posted first (lost race) Completes: "…its result is posted to this chat separately." An app-owned fenced note, no content "Ran X. Its results are posted to this chat separately." Already posted
Stopped waiting (re-authorization failed, run deleted) Completes: "The plan stopped waiting…" An app-owned fenced note, no content "Started X. The plan stopped waiting for it." Its own rules; the plan promises nothing

D4. Scheduler polling and lease.

  • The scheduler polls every 30 s, takes up to 4 due runs per tick, and holds a 120 s scan lock. A waiting run becomes due when its 45 s recovery lease expires.
  • Worst-case extra delay after the workflow run finishes is about 75 s (45 s lease plus 30 s poll), plus queueing when more than 4 runs are due in one tick.
  • Timeout. A timeout is noticed by the first claim after the deadline, so up to about 75 s late. The hold's 120 s grace covers that window, so 6b doesn't post while the plan could still consume.
  • A longer backlog. If one outlasts the grace, the hold lapses and 6b claims the record. The plan's consume then fails its CAS and reports posted. Exactly one delivery still happens; only the route changes.
  • The scheduler code itself doesn't change.

D5. Planner prompt.

  • What changes. The prompt changes only when the wait is configured: the workflow catalog marks waitable entries, and one hint line explains that a waitable run's result may feed compose. Both are computed server-side from quick_run_eligibility and the plan rules.
  • Gating. With the setting off, the catalog and the prompt are byte-identical to before; the off-golden test covers this.
  • The model's marker is ignored. The planner pops any model-written workflow_run_waits, and the schema recomputes the marker from the server-only planning context. A stored plan's marker is revalidated with stored_workflow_run_waits.

D6. V2 changes. These are the minimum needed.

  1. The plan card's waiting line names the workflow, for example Waiting for "Sales digest" to finish (up to 5 min)., instead of the generic "Waiting for results". The line comes from the server's step summary and is shown as text.
  2. The run card shows "Its results were used in a chat answer." for the used_in_answer delivery reason, because there's no posted message to jump to.
  3. The client accepts the new reason.

The timeout text needs no new UI, since the step summary and answer note already carry it. No controls, tracker or bell changes.

D7. The admin turns the setting off mid-wait.

  • The next continuation's _reauthorize_wait returns wait_disabled. It releases the hold, and the step ends "stopped waiting" without promising a post. 6b then posts when the run finishes, if 6b is still on.
  • If the settings change instead stops the plan first (the execution fingerprint changes: result_input_changed), the hold lapses on its own at deadline + 120 s, and 6b posts once.

Decisions made autonomously

  • Only a run started by this step waits. A run that already existed, or a retry of a terminal waited attempt (which links to the existing run), never waits. The hold has to be in place before the run can finish.
  • A step with nothing queued gets no wait record. When a marked step's start is refused, it reports the refusal exactly as today.
  • A failed or cancelled run is consumed. The answer reports the failure, and the step fails with workflow_run_failed / workflow_run_cancelled, so dependent steps follow the existing dependency-failure rules and 6b doesn't post a second failure notice.
  • Re-authorization failure promises nothing. The answer says "The plan stopped waiting for it." and doesn't promise a post, because 6b's own rules (shared or deleted chat, lost access) may refuse to post.
  • A run started without post-back reports "still running" unposted. Its text points to the run in Workflows instead of promising a chat post.
  • The not-waitable skip text promises no post-back. It says: "A plan can use a workflow's results only when the workflow is a quick run and the rest of the plan can continue without you, so it was left out. Ask to start it in a new message."
  • compose behavior:
    • composed is true only when compose actually prepared the run's content.
    • compose ignores a run value that carries no wait record.
    • The existing compose policy text is unchanged.
  • The lineage counts only consumed waits as results used by the plan.
  • require_member_of_workflow_user must be off for any wait. Under it, a headless continuation can't prove membership.
  • D7 behavior is as described above: release on re-authorization; otherwise the hold lapses.
  • The off-golden was captured from the base commit (817f9dd07) with ids redacted, then compared byte-for-byte with the setting off.
  • V2:
    • The waiting line comes from the server's step summary rather than being composed on the client.
    • There's no timeout-specific plan-card UI.
    • The run card has a used-in-answer text.
    • orchestrationPlan.ts matches the literal capability id 'workflow_run'.
    • The client doesn't cross-check used_in_answer against delivered.
  • Hostile workflow names are neutralized: summaries go through clean_catalog_text(), and React renders them as text.
  • The admin form's depends_on lists only the in-tab prerequisites. Use Workflow Results In Chat is on another tab, so the runtime enforces it through wait_configured() and the help text names it.
  • Version .309, as assigned by the coordinator: originally .307, reassigned once V2's own tip reached .307 (PR Render Microsoft 365 pending-action cards in V2 chat (#1722 gap 2) #1731) and the sibling Phase 8 replay branch claimed .308.

State mapping

State Plan card (step) Answer note chat_delivery record Bell Run card under the answer
Waiting (held) Waiting for "X" to finish (up to N min).; dependents show "Waiting for required results." (not written yet) status: pending, plan_wait.state: holding, until = deadline + 120 s Nothing Not shown yet; the plan is still running
Consumed (finished in time) Step complete: "The saved workflow finished; its result is ready for this plan." "Ran X. Its results were used in this answer." status: delivered, outcome_reason: used_in_answer, notice_kind: none, message_id: null, plan_wait.state: consumed None Finished; "Its results were used in a chat answer." (no jump link)
Failed or cancelled run Step failed (workflow_run_failed / workflow_run_cancelled); compose blocked by its required input "…failed, so its results couldn't be used. Its run in Workflows says why." / "…was cancelled before it finished, so it has no results to use." status: delivered, used_in_answer, plan_wait.state: consumed None Failed / cancelled status
Timeout Step complete: "Still running. Its result will be posted to this chat when it finishes." "Started X. It was still running when this answer was written." plan_wait.state: released; 6b proceeds 6b bell when it posts "Posting results…", then posted, with a jump to the post
6b won the race Step complete: "The saved workflow finished; its result is posted to this chat separately." "Ran X. Its results are posted to this chat separately." Delivered by 6b 6b bell Posted, with a jump to the post
Stopped waiting (re-authorization failed or run deleted) Step complete: "The plan stopped waiting for the saved workflow…" "Started X. The plan stopped waiting for it." plan_wait.state: released; 6b's own rules Per 6b Per 6b
User stopped the plan Cancelled "Started X. The plan stopped waiting for it." plan_wait.state: released Per 6b Per 6b
Setting off / not quick / not waitable Today's behavior: a consumed run step is dropped with the not-waitable text; an unconsumed run is started and posted by 6b Today's Today's Today's Today's

Mutation kill table

Each mutation was applied by hand, the listed tests were run and confirmed to fail, the mutation was reverted, and git diff --quiet -- application confirmed a clean revert (pre-version-bump commits checked git diff --quiet -- application ':!application/single_app/config.py' instead). ..._X.py abbreviates test_orchestration_workflow_run_wait_X.py. Full log is a session artifact (mutations_kill.log, mutations_check.log).

# Mutation Site Catching test(s) Result
M1a Keep consumed run steps with the setting off (wait_configured ignores the switch) functions_orchestration_workflow_run_wait.py ..._off_golden.py, ..._planner.py KILLED
M1b Keep consumed run steps with the setting off (schema never rejects a consumed run step) functions_orchestration_schema.py ..._off_golden.py, ..._planner.py KILLED
M2 Treat an ineligible workflow (a group workflow) as eligible functions_orchestration_workflow_run_wait.py ..._eligibility.py, ..._planner.py KILLED
M3a Drop the deadline (the wait never times out) functions_orchestration_workflow_runs.py ..._execution.py KILLED
M3b Drop the admin cap from the bound (wait the plan budget only) functions_orchestration_workflow_run_wait.py ..._delivery.py KILLED
M4a Skip the compare-and-set (write the delivery record without the ETag) functions_orchestration_workflow_runs.py ..._delivery.py KILLED
M4b Consume without marking the post-back delivered (post twice) functions_workflow_chat_delivery.py ..._delivery.py, ..._execution.py KILLED
M5 Skip re-authorization while waiting functions_orchestration_workflow_runs.py ..._execution.py KILLED
M6 Allow a hand-off (one-time) run to be waited on functions_orchestration_workflow_run_wait.py ..._eligibility.py, ..._planner.py, ..._execution.py KILLED
M7 Break native tabular claiming functions_orchestration_continuation.py ..._claims.py, test_orchestration_continuation.py KILLED
M7b Break saved-workflow-run claiming functions_orchestration_continuation.py ..._claims.py KILLED
M8 Skip the headless-capability check functions_orchestration_workflow_run_wait.py ..._plan_rules.py KILLED
M9 Skip the session-needing-step check functions_orchestration_workflow_run_wait.py ..._plan_rules.py KILLED
M10 (V2) Remove used_in_answer from the client delivery reasons workflowRunStatus.ts test_v2_workflow_run_status.mjs, test_v2_workflow_run_tracker.mjs KILLED
M11 (V2) The finished-results helper says "posted" for used_in_answer workflowRunStatus.ts test_v2_workflow_run_status.mjs KILLED
M12 (V2) Drop the waiting line from describeRunProgress orchestrationPlan.ts test_v2_orchestration_run_progress_status.mjs KILLED

16/16 KILLED.

Known base failures (not caused by this PR)

All confirmed identical against a temporary detached worktree of base 817f9dd07 (removed after each comparison), so none of these are caused by this PR. That triage predates the push-time merge of V2's tip to 08def99f7; per the coordinator's "light catch-up, not a full re-run" instruction, the 111-file sweep and the V2 Playwright pre-existing failures below were not independently re-triaged against 08def99f7. The two docs-specific known failures were re-run post-merge (see "Post-merge catch-up" under Testing/validation) and are confirmed unchanged at the new head.

From the full 111-file neighbour sweep (10 files, 11 failing tests, 1 timeout):

  • test_admin_settings_group_shared_regions.py::test_shared_controls_are_not_duplicated
  • test_admin_settings_nav_map.py::test_every_nav_destination_exists
  • test_admin_settings_safe_int_fallback_fix.py::test_version_alignment_for_safe_int_fix
  • test_admin_settings_sidebar_card_parity.py::test_every_top_level_card_has_sidebar_destination
  • test_admin_settings_tab_preservation.py::test_admin_settings_html_structure
  • test_admin_settings_template_composition.py::test_no_functional_test_reads_the_template_uncomposed
  • test_orchestration_planner_failure_diagnostics.py::test_three_pinned_documents_allow_analyze_but_explicit_search_stays_required
  • test_settings_secrets_exposure_hardening.py::test_admin_secret_redaction_and_preservation
  • test_v2_admin_settings_rail_collapse.py::test_the_rail_collapses_to_labelled_icons
  • test_v2_admin_settings_schema.py::test_defaults_match_their_field_type and ::test_gated_fields_inherit_their_gate_s_own_conditions
  • test_orchestration_workflow_handoff_imports.py — TIMEOUT at 300s (slow, not failing; an import-cycle probe unrelated to this PR)

None of these touch workflow runs, workflow results, 6b delivery, native waits, or V2 workflow-run/plan-card surfaces — they're admin-settings-shell, nav-map and secrets-hardening checks unrelated to Phase 6c.

Docs (named as known failures in the task brief):

  • test_docs_link_integrity.py (__main__) — 2/5 passed. The 3 failures: a dead link to V2_MEDIA_GALLERIES_AND_VIEWER/DOCUMENTS_DRAWER_GENERATED_FILES_FIX, and 2 unresolved features.yml links (CONTENT_SCREENING_FRAMEWORK, CHAT_CONTENT_CHECKS).
  • test_docs_release_notes_integrity.py (__main__) — 0/1 passed.

V2 Playwright, discovered during this PR's own validation (pre-existing files, unmodified by this branch):

  • test_v2_workflow_run_card.py::test_the_card_follows_each_live_state_the_status_route_reports — expects the M365 reconnect link at /profile?tab=settings#m365-connection-status; the app currently serves /settings?tab=preferences&section=m365-workflow-connection.
  • test_v2_workflow_run_tracker_spa.py::test_the_tracker_stays_off_after_the_app_fails_to_start — "The app never refreshed its bootstrap."; unrelated to this branch's 3 touched V2 files (WorkflowRunCard.tsx, orchestrationPlan.ts, workflowRunStatus.ts).
  • test_v2_orchestration_plan_card.py::test_timed_plan_shows_a_countdown — "a timed plan must render a role=timer element"; the countdown timer is a pre-run display path this branch doesn't touch.

Guardrails

Run twice, both clean. The second run is the one that counts, since it reflects the final head after the push-time V2 merge.

  1. Pre-merge: run_v2_guardrails.py <worktree> 817f9dd07 HEAD 6c, base = the V2 commit this branch was based on at the time (itself picked up by an earlier mid-session V2 catch-up merge). 6/6 PASS, 0 blockers, 542 app .py files compiling.
  2. Post-merge (final, base = 08def99f7, the V2 tip merged in at push time — PR Render Microsoft 365 pending-action cards in V2 chat (#1722 gap 2) #1731): same 45 files relative to that base (20 app .py, 24 XSS-surface, 1 route .py, 3 V2 .ts/.tsx), unchanged scope and verdict; python-syntax now compiles 543 app .py files (+1, from V2's own tip advancing).
Check Result (post-merge, base 08def99f7)
broken-access-control PASS (20 files)
xss-sinks PASS (24 files)
swagger-routes PASS (1 changed route file; no new routes, decorators unchanged)
python-syntax PASS (543 files compile)
malicious-pr-review PASS (746 pattern findings, 0 blockers; verdict "Needs investigation" is expected noise from security/sanitization-keyword matches in XSS-guardrail tests, the off-golden fixture and settings-sanitization code)
v2-ts-sinks PASS (0 hits in 3 changed .ts/.tsx files)

6/6 PASS, 0 blockers (both runs).

CodeQL

CodeQL has since completed a real PR diff — the earlier fork-PR limitation noted below resolved once GitHub finished the configuration the coordinator had already flagged as "still in progress." gh api "repos/microsoft/simplechat/commits/2e174aa9704396d51170766f6bf4be7d91171b3c/check-runs?check_name=CodeQL" now returns conclusion success, with:

New alerts in code changed by this pull request: 6 notes

All three Analyze jobs (actions, javascript-typescript, python) and the aggregate CodeQL check pass. This result supersedes the earlier neutral/"configurations not found" read in this section's history.

The 6 new notes (gh api repos/microsoft/simplechat/check-runs/113615996428/annotations), every one notice/note severity (the lowest CodeQL severity, none security-severity-rated):

# Rule File Line Why it's left as-is
1 py/cyclic-import functions_orchestration_schema.py 85 A deliberate, already-commented lazy-import boundary: "The wait rules import this module lazily, so importing them here does not form a cycle." functions_orchestration_workflow_run_wait.py never imports functions_orchestration_schema at module level; CodeQL's whole-graph check still flags the edge regardless of which side is lazy.
2 py/cyclic-import functions_orchestration_workflow_run_wait.py 230 The functions_orchestration_registry import is already function-local (inside a try: block, not at module top-level) for exactly this reason.
3 py/cyclic-import functions_workflow_limits.py 276 My new getter follows the identical lazy-functions_settings-import pattern already used by all 8 getters in this file, each with the same comment: "Settings initialize application clients; load them only at a request boundary." Same established technique, new call site, not a new pattern.
4 py/import-and-import-from functional_tests/test_orchestration_workflow_run_wait_execution.py 33 Both forms are needed: from ... import (PLAN_WAIT_CONSUMED, ...) for direct constant references, and import ... as delivery because monkeypatch.setattr(delivery, 'utc_now', ...) (line 262) must patch the module object itself, not a name copied out of it.
5 py/import-and-import-from ui_tests/test_v2_workflow_run_wait.py 41 Both forms are needed: import test_v2_orchestration_plan_editor as editor_tests for editor_tests.make_plan(...)/editor_tests.RUN call sites, and from ... import (connect_options, editor_assets, editor_browser) because pytest fixtures must be bound by their own name in the test module's namespace to be discovered — editor_assets/editor_browser are used as fixture parameters at line 161.
6 py/unused-import ui_tests/test_v2_workflow_run_wait.py 50 run_assets is a pytest fixture (from test_v2_workflow_run_card.py) used as a fixture parameter at line 177 (def card_ui(editor_browser, run_assets):); pytest resolves fixtures by name against the module's imports, which a plain usage scan can't see — hence the pre-existing # noqa: E402, F401. The same rule already appears 3 times in the base-branch baseline below for this identical pytest convention.

None of the 6 are fixed: each fix would either reintroduce a real import cycle, break monkeypatch.setattr on the actual module object, or break pytest's fixture discovery. All 6 match an established, already-documented pattern used elsewhere in this codebase — see the baseline immediately below, which carries the same 3 rules for the same reasons.

Base-branch baseline (paullizer-react-v2-ui, queried via code-scanning/alerts?ref=refs/heads/paullizer-react-v2-ui&state=open, 10 open alerts, all severity note) — unchanged, none in a file this PR touches:

Alert(s) Rule File
#4159, #4154, #4153 py/cyclic-import functions_orchestration_recovery.py
#4150 py/cyclic-import functions_orchestration_rendering.py
#4149 py/cyclic-import functions_orchestration_checkpoints.py
#4158, #4157 py/unused-import test_conversation_delete_hns_directory_cleanup.py
#4152 py/unused-import test_orchestration_terminal_wait_retry.py
#4151 py/unused-import test_orchestration_settings_runtime_state_binding.py
#4148 py/import-and-import-from test_orchestration_file_sign_in_publication.py

No alerts are dismissed here — that stays a maintainer action.

Follow-ups

  • Storage failure between consume and persist. If storage fails after the consume CAS but before the plan persists its step result, the result stays in Workflows; the run card shows "used in a chat answer". A recovery path could re-read it.
  • Release the hold promptly when a plan stops elsewhere (recovery_changed, result_input_changed) instead of waiting for it to lapse at deadline + 120 s.
  • A public alias for the results reader's _reader_outcome, used by the wait.
  • A push wake-up when a waited run finishes, instead of scheduler polling (removes the ~75 s worst case).
  • A replay parity test once Phase 8 (Plan replay: repeat an approved chat plan as a personal workflow (#1550) #1724) lands: replay refuses SAVED_WORKFLOW_RUN_WAIT_KIND.
  • A harness-level scheduler test for the new wait kind. Today's coverage is at the continuation and claim layer.
  • A "held for a plan" delivery state for V2. During a hold, a run card shown somewhere else (the Workflows page) says "Posting results…".
  • Wording. The recovery notice says "computation", which is generic, for a waited workflow.
  • Base test fixes noticed: test_v2_workflow_run_card.py's stale M365_HREF, the tracker's focus-refresh throttle in test_v2_workflow_run_tracker_spa.py, and the missing M365 endpoints in the test_v2_notifications_bell.py harness.

Paul Lizer (paullizer) and others added 25 commits October 8, 2026 10:43
Adds enable_chat_orchestration_workflow_run_wait (default off) and
chat_orchestration_workflow_run_wait_max_seconds (default 300, 60 to
1,800) with their V2 admin fields, the Classic Chat Orchestration pane
inputs, the Classic form clamp, and validation on the settings writer.
Nothing reads either setting yet.

Part of microsoft#1546. Refs microsoft#1543.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The help listed only some of the rules, so an admin could conclude that a Microsoft 365 run-as or file-sync workflow can be waited on. Both the V2 field and the Classic pane now name every rule, plus the plan-level limits that come from resuming without the user's sign-in.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When the in-plan wait setting is on and the saved workflow passes the quick-run rule, the planner offers it as waitable, the schema compiles a server-side wait marker, and drop_workflow_runs keeps the consumed run step. With the setting off, behavior is unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…pare-and-set

A plan waiting on a saved workflow run holds the run's 6b post-back only while
no post of that outcome has begun. It uses the result through a compare-and-set
on the run's delivery record, and lets go on timeout so the post-back posts it
exactly once. The worker waits out a live hold.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the saved_workflow_run wait kind to the executor's resume path, the run step's hold, observe, consume and timeout logic, the reply note's wait lines, and two failure codes for runs that fail or are cancelled while a plan waits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Compose reads a run the plan waited for only as fenced, untrusted notes: a result the plan took is read again through the workflow result reader with its digest, and any other outcome becomes an application-owned note. The answer's lineage carries the taken result so later reads re-authorize it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Covers holding the post-back while waiting, using a result that finishes within the bound, the timeout hand-back with exactly one post, failed and cancelled runs, re-authorization failures mid-wait, refused continuations, and the finish-versus-timeout race in both orders. A run started without the post-back now reports it is still running rather than that the plan stopped waiting.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
With the wait setting off (absent, false, string and numeric values, and on
without the 6b post-back), planner context, prompt, payload, repair turn,
drop_workflow_runs, normalize_plan and execution are byte-identical to the
fixture captured from e1657ef (0.261.304). Random plan, run and turn ids
and timestamps are redacted; two base captures were identical.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…swer

- The compact progress line shows the waiting workflow_run step's server
  summary (for example, Waiting for "Sales digest" to finish (up to 5 min).).
  Other waits keep the generic line.
- The run card reads a delivery marked used_in_answer as "Its results were
  used in a chat answer." It never says the results were posted, and it
  shows no jump button. The tracker and the bell never announce it.
- .mjs unit tests plus a Playwright test with hostile workflow names.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Plan replay (microsoft#1724) took 0.261.306 after V2 moved to 0.261.305.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…it rules in the research harness

The V2 admin renderer evaluates each field's own conditions, so the cap field
now repeats its gate's enable_chat_orchestration_workflow_runs condition.

The offline research harness executes only explicitly supplied definitions;
plan validation now asks the import-light wait rules about every plan, so the
harness seeds the real compute/stored wait functions into the schema namespace.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the feature doc, admin rows for Wait For Quick Workflows In Chat and its
wait cap, the guide section on using a quick workflow's result in the same
answer, the features.yml claim, the regenerated docs inventory and the
v0.261.307 release notes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
waited_run_consumers_valid previously required only that some enabled compose
step bind the run's output through a required input. That let a plan mark a
step as waiting even when the final response never read that compose step,
directly or transitively, so the answer could finish without the result while
the 6b post-back stayed held for the plan.

Add a reader-reachability pass: starting from the run step, walk enabled
compose steps whose required inputs bind the run or another reader, and
require the final response's own step_id to land in that set. A final
response with no step_id, a non-string step_id, or one equal to the run step
itself is rejected up front.

Covers: a reader whose text never reaches the answer, an answer that reaches
the reader only through an optional input, a final response selecting a
disabled step, a missing final response, and a final response without a
step_id. Also confirms a second, unused reader is harmless as long as the
answer itself needs the result, and that the stored-marker fast path accepts
the same plan.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
_finalize passed composed=bool(prepared) to workflow_run_note, so the note
could say a consumed run's result was used in the answer even when the reply
itself was not written from the plan's steps, for example when only unplaced
generated images populated prepared. workflow_results_note next to it already
required error is None as well.

Compute workflow_result_reads once, before building either note, and pass
that same flag as workflow_run_note's composed argument. Both notes now agree
on what "used in this answer" means. _wait_line's branching was already
correct; only the flag _finalize handed it was wrong.

Adds a test that drives a consumed wait through to completion and checks
workflow_run_note's text both ways: with composed=True, "Its results were
used in this answer."; with composed=False, "This answer couldn't use its
results; they're in its run in Workflows." (coordinator note 3: "compose
fails after a consume").

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Picks up 14 V2 commits since the prior merge base (817f9dd),
including M365 pending-action cards, the V2 agents catalog, the
scope-response contract fix and the group-switch latency fix. No
Phase 6c file conflicted except release_notes.md, resolved by
keeping this branch's section on top of V2's .306 and .305
sections, all above the shared .304 section.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
V2's tip landed at .307 (PR microsoft#1731) and the sibling Phase 8 replay
branch has claimed .308, so this PR's version moves to .309 per the
coordinator. Updates every self-authored test header, docstring,
assert_app_version_at_least floor, feature/admin/guide doc mention
and the release-notes section header. Leaves config.py VERSION and
every other branch's own .307 references (V2 M365 pending actions,
the roadmap doc) untouched; the VERSION bump itself follows as the
final, separate commit.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread application/single_app/functions_orchestration_schema.py
Comment thread application/single_app/functions_orchestration_workflow_run_wait.py
Comment thread application/single_app/functions_workflow_limits.py
Comment thread functional_tests/test_orchestration_workflow_run_wait_execution.py
Comment thread ui_tests/test_v2_workflow_run_wait.py
Comment thread ui_tests/test_v2_workflow_run_wait.py
@paullizer
Paul Lizer (paullizer) marked this pull request as ready for review October 9, 2026 01:52
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit 271b301 into microsoft:paullizer-react-v2-ui Oct 9, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants