Skip to content

Phase 3c: Ask AI tab in the V2 workflow editor (0.261.211) - #1593

Merged
Paul Lizer (paullizer) merged 17 commits into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-phase-3c-workflow-ask-ai-tab
Oct 1, 2026
Merged

Paul Lizer (paullizer) merged 17 commits into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-phase-3c-workflow-ask-ai-tab

Conversation

@paullizer

@paullizer Paul Lizer (paullizer) commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Ask AI tab in the V2 workflow editor (Phase 3c, which completes Phase 3). A new side-panel tab after Changes edits a personal workflow from a plain-language request. It calls 3b's POST /api/user/workflows/assist and applies each answer through 3a's applyAssist as one AI change. Highlights, Previously:, per-field Revert, the Changes list and the AI-change review on the first save all work as they do for 3a. The tab never saves.
  • Each turn gets a card. The card lists what changed, taken from the editor's own diff of the draft it sent against the answer, never from the server's list. It also has Jump to, the documents read as context, any warnings and Undo this change. The editor is locked while a request runs, and Cancel request stops it with nothing changed. On a 409 the draft is kept and Reload workflow is offered, and Retry-After is honored. Draft with AI fills a task's empty instructions.
  • Where it shows. The tab appears only when features.enable_workflow_ai_assistant is on and the editor can change a personal workflow. Group workflows, readers, unsupported drafts and workflows with an active run don't get it. It also works on a Phase 4 workflow proposal opened from chat with Edit. Admins get no new setting: the existing Enable AI Workflow Assistant now shows the tab.
  • Plain text, keyboard and focus. Replies, errors, warnings, document names and the admin's Markdown rate-limit message are rendered as plain text, never as HTML or Markdown. The panel is an ARIA tab list, the thread is a live log, and Escape closes the open menu first, then the panel, returning focus to the toggle. The tab also works on a narrow screen. A1's shared thread gained opt-in options that are all off by default, so the image, chart, diagram and plan editors don't change.

Linked issue

Closes #1548
Part of #1543

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

No new setting: the existing Enable AI Workflow Assistant (enable_workflow_ai_assistant, added in 3b) now also shows the tab and Draft with AI. docs/admin/workflow.md says so.

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Not yet, because a card needs a screenshot. It's listed as a follow-up for when the guide's screenshot exists.

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Version bump

Testing / validation

Everything ran on head 9ee340bba, one file per run, in sequence, with the V2 bundle built from that head. "Normal and -O" means both runs gave the same result. The CodeQL fix that followed, dc9d22ad1, changes only two browser-test files, and both Ask AI browser files passed again on it (see CodeQL). The three coordinator review rounds after that, 439d1115b, c17339aa8 and 360207d97, reran the tests they affect; see Coordinator review. After Phase 5 merged, a9dd15b28 merged the V2 tip 5fc50a593. What reran on that head is listed at the end of this section, under After the Phase 5 merge.

# in application/v2_ui
npm run build                     # tsc -b && vite build
# functional tests, from the repo root
$env:PYTHONPATH = "application/single_app;functional_tests"
python -u -m pytest functional_tests/<file> -q -p no:cacheprovider -W ignore
python -O -u -m pytest functional_tests/<file> -q -p no:cacheprovider -W ignore
# Playwright
$env:PYTHONPATH = "application/single_app;functional_tests;ui_tests/fixtures"
$env:PLAYWRIGHT_SERVICE_URL = ""
python -u -m pytest ui_tests/<file> -q -p no:cacheprovider -W ignore
  • npm run build: passed. Its chunk-size warning is pre-existing. npm run typecheck and npm run build passed again on 360207d97.

New tests

Target Result
functional_tests/test_v2_workflow_ask_ai.py 7 passed (normal and -O); python functional_tests/test_v2_workflow_ask_ai.py 7/7. 7 passed (normal and -O) again on 439d1115b, c17339aa8, 360207d97 and a9dd15b28, with 175 logic checks
functional_tests/test_v2_workflow_ask_ai_parity.py 12 passed (normal and -O), again on a9dd15b28
ui_tests/test_v2_workflow_ask_ai.py 25 passed; 25 passed again on dc9d22ad1; 29 passed on 439d1115b; 31 passed on c17339aa8; 32 passed on 360207d97, 9a9590ce0 and a9dd15b28
ui_tests/test_v2_workflow_ask_ai_proposal.py 3 passed; 3 passed again on dc9d22ad1, 360207d97 and a9dd15b28

Shared components (N7)

Target Result
functional_tests/test_v2_assist_thread.py 14 passed (normal and -O), again on a9dd15b28
functional_tests/test_v2_plan_editor_references.py 10 passed (normal and -O)
functional_tests/test_v2_chat_context_picker.py 9 passed, 1 failed (normal and -O): base, see Known base failures
functional_tests/test_v2_image_editor.py 16 passed (normal and -O)
ui_tests/test_v2_assist_thread.py 10 passed
ui_tests/test_v2_plan_editor_references.py 11 passed
ui_tests/test_v2_orchestration_plan_editor.py 35 passed
ui_tests/test_v2_image_editor.py 3 passed
ui_tests/test_v2_workflow_editor.py 9 passed; 9 passed again on 360207d97
ui_tests/test_v2_workflow_change_tracking.py 24 passed; 24 passed again on 360207d97
ui_tests/test_v2_orchestration_workflow_proposal_card.py 22 passed; 22 passed again on 360207d97 and a9dd15b28

3b's server tests and other regressions

Target Result
functional_tests/test_workflow_assist_field_parity.py 23 passed
functional_tests/test_workflow_assist_candidate_parity.py 15 passed
functional_tests/test_workflow_assist_request.py 147 passed
functional_tests/test_workflow_assist_scenarios.py 49 passed
functional_tests/route_tests/test_workflow_assist_policy.py 64 passed
functional_tests/test_v2_workflow_change_tracking.py 4 passed
functional_tests/test_orchestration_workflow_proposal_editor_round_trip.py 5 passed
functional_tests/test_group_workflow_file_sync_client_parity.py 179 passed
functional_tests/test_v2_workflow_unsupported_schedule_readonly.py 463 passed
functional_tests/test_workflow_alert_client_parity.py 84 passed
functional_tests/test_workflow_calendar_schedule_client_parity.py 361 passed
functional_tests/test_workflow_draft_v2_round_trip.py 19 passed
functional_tests/test_workflow_file_sync_analyze_targets.py 11 passed
functional_tests/test_workflow_flow_layout.py 9 passed
functional_tests/test_v2_prompts_workbench.py 22 passed
functional_tests/test_v2_prompt_composer_card.py 9 passed, 2 failed: base
functional_tests/test_v2_chart_editor.py 23 passed
functional_tests/test_v2_diagram_editor.py 18 passed
functional_tests/test_block_revision_assist.py 7 passed, 1 failed: base
functional_tests/test_assist_thread_submission_id.py 61 passed, 62 subtests passed
functional_tests/test_message_block_revisions.py 18 passed
functional_tests/test_orchestration_plan_revision_routes.py 28 passed, 1 failed: base
functional_tests/test_v2_elicitation_answers.py 2 passed
functional_tests/test_orchestration_context_picker.py 11 passed
node functional_tests/test_workflow_field_drafts.js exit 0
node functional_tests/test_workflow_settings_client.js exit 0
ui_tests/test_v2_chat_context_selection.py 43 passed
ui_tests/test_v2_elicitation_composer.py 18 passed
ui_tests/test_v2_prompt_composer_experience.py 56 passed
ui_tests/test_v2_image_reference_generation.py 4 passed
ui_tests/test_v2_workflow_loops.py 72 passed, 1 failed: base
ui_tests/test_v2_workflow_flow_authoring.py 30 passed on 360207d97
ui_tests/test_v2_personal_workflow_file_sync.py::test_editing_personal_sources_is_one_change_and_jump_lands_in_file_sync 1 passed on 360207d97
python ui_tests/test_v2_orchestration_composer.py 6/6. Under pytest: 1 passed, 5 failed (harness)
python ui_tests/test_v2_orchestration_elicitation.py 6/6. Under pytest: 1 passed, 5 failed (harness)

Guardrails, docs and the base check

  • The coordinator's local guardrail runner, python run_v2_guardrails.py <worktree> 4f778386b 9ee340bba 3c, mirrors the Development-only PR checks: broken access control, XSS sinks, Swagger routes, Python syntax and the malicious-PR review, plus a TS/TSX sink scan. All passed. The malicious-PR review's warnings are informational, with no blockers.
  • python scripts/build_docs_inventory.py: no inventory change.
  • python functional_tests/test_docs_app_surface_coverage.py: 7/7. python functional_tests/test_docs_site_quality.py: 6/6. python functional_tests/test_docs_latest_release_catalog_parity.py: 6/6. All three passed again on 360207d97.
  • python functional_tests/test_docs_link_integrity.py: 2/5. python functional_tests/test_docs_release_notes_integrity.py: 0/1. Both are base failures, with the same results on 360207d97.
  • python functional_tests/test_v2_ui_local_assets.py: 4/4 on 360207d97.
  • Three-dot diff against paullizer-react-v2-ui at 5fc50a593, on a9dd15b28:
    • 32 files, +6692/−122, the same as against 4f778386b at 9a9590ce0. It was 31 files, +6588/−116 at c17339aa8; 360207d97 adds useWorkflowAuthoring.ts, and 9a9590ce0 adds one test line.
    • config.py changes only 0.261.212 → 0.261.213
    • release_notes.md only adds lines (+20/−0). The v0.261.213 section sits above Phase 5's v0.261.212 section, and the .212, .210 and .209 sections are byte-identical to the V2 tip.
    • No added line contains 0.261.211, and git grep 0.261.211 finds nothing.

After the Phase 5 merge

Phase 5 (#1594) merged first and took 0.261.212. 702f1c5cb renumbers this PR from 0.261.211 to 0.261.213 on its own: 40 lines in 14 files, each changing only the version. a9dd15b28 then merges the V2 tip 5fc50a593 with rerere off. Only config.py, kept at .213, and release_notes.md conflicted; the .213 section goes on top, with Phase 5's .212 section byte-identical below it. docs/admin/workflow.md and docs/_data/features.yml merged cleanly, with both sides kept. Rerun on a9dd15b28, in sequence, with a fresh V2 build:

  • npm run typecheck and npm run build: passed.
  • functional_tests/test_v2_workflow_ask_ai.py, test_v2_workflow_ask_ai_parity.py and test_v2_assist_thread.py: 33 passed (7, 12 and 14), normal and -O. python functional_tests/test_v2_workflow_ask_ai.py: 7/7, with 175 logic checks.
  • ui_tests/test_v2_workflow_ask_ai.py and ui_tests/test_v2_workflow_ask_ai_proposal.py: 35 passed (32 and 3). ui_tests/test_v2_orchestration_workflow_proposal_card.py: 22 passed. I reran it because Phase 5 changed MessageList.tsx, which renders the proposal cards.
  • Docs: test_docs_app_surface_coverage.py 7/7, test_docs_site_quality.py 6/6 and test_docs_latest_release_catalog_parity.py 6/6. python scripts/build_docs_inventory.py makes no inventory change. test_docs_link_integrity.py (2/5) and test_docs_release_notes_integrity.py (0/1) are base failures; see Known base failures.
  • The CI guard scripts against 5fc50a593: check_xss_sinks.py passed for 17 files and check_broken_access_control.py for 1. check_swagger_routes.py skipped, because no changed Python file defines a route. py_compile passed for all 7 changed Python files, and the route policy suites passed 10/10, 4/4 and 2/2.

Documentation

  • Release notes updated, or not needed: ### **(v0.261.213)**, above Phase 5's v0.261.212 section, has "Ask AI Tab In The V2 Workflow Editor" and "Draft With AI For Workflow Task Instructions".
  • Feature documentation updated, or not needed:
    • WORKFLOW_AI_ASSISTANT.md covers the Ask AI tab: when it's offered, the thread, sending, applying, the turn card, failures (including what's locked while a reload loads), Draft with AI and accessibility. It also has usage, the tab's tests and the known limitations.
    • V2_SHARED_ASSIST_THREAD.md has "Options for the workflow editor" (countCodePoints, retain and held threads, text and sendText, cancelledMessage, contextDocumentsOnly, the renderReply wrapper) (N10).
    • WORKFLOW_EDITOR_CHANGE_TRACKING.md explains how Ask AI and Draft with AI use applyAssist and revertTurn. Since review part 3 it also says that Jump clears a block focus request, and that every tracked field carries a field key that Jump finds before the highlight renders.
    • Also updated: docs/admin/workflow.md, docs/guides/create-a-workflow.md ("Ask AI to change a workflow") and docs/_data/features.yml.
  • Fix documentation updated, or not needed: not needed, because this isn't a fix.

The guide declares the media slot docs/images/guides/create-a-workflow-ask-ai.png. It shows a placeholder until a screenshot is added.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security()): this PR adds no routes, and its only backend change is VERSION.
  • Settings sent to non-admin frontends use sanitize_settings_for_user(): no settings are sent. The tab reads the existing per-user bootstrap flag features.enable_workflow_ai_assistant.
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS: the tab is part of the V2 bundle, and test_v2_workflow_ask_ai.py checks that it imports only local modules.
  • No secrets, keys, connection strings, or local-only artifacts are included

Decisions made autonomously

  1. The candidate is rebased onto the live draft before it's diffed or applied. 3b's candidate went through JSON, so it lost the editor's undefined values and NaN (gotcha 25). workflowAssistViolation compares ownField values, so without the rebase a live id: undefined against a missing id would refuse every new-draft turn. The coordinator confirmed this is required.
    • Wherever the candidate holds the JSON of the live value, rebaseAssistCandidate keeps the live value and its identity. On v3, rebased.flow === live.flow when the flow is unchanged.
    • Tasks are matched by a unique id, or by position when ids are missing or duplicated.
    • It restores only keys that were undefined, so a key the model deleted stays deleted.
    • A forbidden change survives the rebase, so applyAssist still refuses it.
  2. Thread key. A saved workflow uses workflow:personal:<id>, and a new or proposal draft uses workflow:new:<id>, one per editor mount.
    • The conversation lives in A1's in-memory thread store, under A1's limits.
    • The editor holds its thread while it's open (retain), so the thread isn't swept, and using it never sweeps a chat's idle image or plan thread.
  3. Impact confirmation. When 3a asks the user to confirm an impact, Send stays busy until they answer (N4). The card tracks confirm and decline, and the stale-draft guard stays as the backstop.
  4. Stale draft and 409.
    • If the draft changed while a request ran, the turn fails and offers Retry. It never merges.
    • On a 409 the draft is kept and Reload workflow is offered. After a discard confirmation, a new onReload prop from WorkflowsSection reopens the saved version under a fresh editor key.
  5. The cancel message is "Cancelled. Nothing was changed.", because the server writes nothing.
  6. time_zone is sent only when the editor's time-zone list has the browser's zone. Otherwise it's left out and the server's default applies.
  7. Ask AI about this task appears on task rows and task nodes only, not on structural flow blocks.
  8. Draft with AI reuses /api/workflows/draft-instructions, appears only where Ask AI does, and only on a task whose instructions are empty.
    • A draft longer than the editor's 12,000-character WORKFLOW_TASK_INSTRUCTIONS_LIMIT isn't added, and the message says so.
    • N9: that route checks workflow access, not enable_workflow_ai_assistant, so hiding the button whenever Ask AI is hidden is the conservative choice.
    • The classic Draft Workflow Instructions button is unchanged.
  9. Earlier editing sessions. A card from an earlier editor mount, on the same saved workflow reopened in the same page, hides Jump to, including on its warnings. Undo this change stays and reports "This turn can no longer be undone." Replay says those changes were made in an earlier editing session and may not have been saved.
  10. Replay sends only completed turns, as {role, text}.
    • The assistant item is the reply plus "Changes applied: …", or a note that those changes were undone. A rejected turn says "No changes were applied.", and a pending confirmation is marked as not applied.
    • It sends at most the last 10 whole exchanges (20 items), so it never starts with an orphan assistant item.
    • It strips the server's control-character class and keeps tab, LF and CR.
    • It caps each item by code points after adding the suffix, without splitting a surrogate pair.
  11. References. The # picker offers documents only (contextDocumentsOnly).
    • A tag or workspace chip that still arrives, for example through Edit and resend, is refused before anything is sent.
    • More than 20 documents gets "Attach at most 20 documents."
    • Deduplication and order come from A2's planDraftReferences and canonicalPlanReferences(value, 20).
  12. Raw fetch (N8). ApiError doesn't expose Retry-After, and 503 provider throttling carries only the header.
    • The request keeps apiUrl, CREDENTIALS_MODE and the JSON Accept and Content-Type headers.
    • A non-JSON 200, such as a sign-in page after the session expired, gets "The assistant's answer couldn't be read. Nothing was changed."
  13. While a reload loads, the draft is locked, but the editor can still close (review part 1).
    • Locked: the fields, Save workflow, undo and redo (the buttons and Ctrl+Z, Ctrl+Y and Ctrl+Shift+Z), flow authoring, the Changes tab, Send, the composer and quick actions, and each card's Undo this change. The card takes a reloading prop rather than reusing assist.busy, because busy also covers a 429 wait.
    • Cancel and Escape stay live, so a slow reload can't trap the user. The request token makes closing safe: a reload that finishes after its editor closed is ignored.
    • If the reload finishes while the "Discard changes" confirmation is open, the reloaded editor replaces it. Nothing is lost, because the user already chose Discard and reload.
    • Reload workflow sits outside the fieldset that a save disables, so a reload can start while a save is in flight. Test (d) pins that a save landing during a reload keeps the editor closed with one write. Disabling Reload while saving is listed as a follow-up.
  14. Jump to drops a pending block focus request (review part 3). A jump places focus itself, so jumpToChange clears the block focus request that the step before it made, through a new clearFocusRequest in useWorkflowAuthoring.ts. Without it, after Draft with AI on Flow, the canvas could take focus back from the drafted field.
  • Every caller of jumpToChange gets this. For a jump from a card, the Changes tab or a warning, any earlier request has already been carried out, so clearing it changes nothing.
  • The alternatives touched shared code more: a guard in the canvas's focus effect would break focus after moving or removing a block, and telling onRestore where a restore came from would change 3a's history.

Real-server parity (N1)

functional_tests/test_v2_workflow_ask_ai_parity.py connects the tab's TypeScript to 3b's real code. It reuses the scenario builders in test_workflow_assist_candidate_parity.py and their esbuild/node runner.

  • Requests. The test builds requests with buildWorkflowAssistRequest from editor-normalized drafts:

    • a new draft, a proposal draft through Phase 4's real normalize, a saved v2 draft with its revision, and a saved v3 draft
    • with focus, time_zone, references, a 20-item conversation with emoji, tabs and new lines, and a 2,000-code-point instruction

    parse_assist_request accepts each one, and each is posted as its exact text through the real Flask route with a scripted model.

  • Responses. Every candidate-parity scenario's real 200 body, including "new draft" and "new draft with a shared reference", passes parseWorkflowAssistResponse.

    • After rebaseAssistCandidate, workflowAssistViolation(live, rebased) is ''.
    • verifyAssistChanges gives exactly the server's changes[]. The key, kind, label, owner label, focus key and node id all match, with the same wording. There are no exceptions.
  • N12: no disagreement between 3b's server and 3a's editor turned up.

Coordinator review

Part 1 (439d1115b)

  • R1: the reload race. reloadEditing in WorkflowsSection.tsx reopened the editor and cleared the dirty flag as soon as its two reads finished, even if the editor had been closed, saved or replaced by another workflow's editor while they loaded. In the worst case, another workflow's unsaved edits vanished without a prompt.
    • An editorRequest token (useRef) is bumped when an editor opens, closes or saves. reloadEditing takes a fresh one and returns after its reads if the token changed.
    • The draft is locked while the reload loads (decision 13).
    • Four browser cases: (a) Discard and reload, then Escape and Discard changes: no editor reappears. (b) Close and discard, then open another workflow and type: that editor stays, with the typed text. (c) A failed reload unlocks the draft it kept, with its AI change and Undo this change. (d) A save that lands during a reload keeps the editor closed, with one write. None of them writes a workflow except (d)'s save.
  • T1: pruneThreads's held-thread guarantees are pinned. Three logic checks: another conversation's sweep keeps an idle held workflow thread; the 50-thread cap keeps a held least-recently-touched thread, evicts the next oldest and leaves 50; and a held keepKey that isn't marked quiet sweeps nothing.
  • Reran: ui_tests/test_v2_workflow_ask_ai.py 29 passed. functional_tests/test_v2_workflow_ask_ai.py 7 passed (normal and -O), with 175 logic checks. functional_tests/test_v2_assist_thread.py 14 passed (normal and -O).

Part 2 (c17339aa8)

  • T2: Ctrl+Z and Ctrl+Y are pinned while the draft is locked. Two browser cases on the structured workflow, with an undo and a redo both available. Focus is on a flow block, which is not a text field and stays focusable inside the locked fieldset, so its keys reach workflow history.
    • One holds an Ask AI request, the other holds a reload. Ctrl+Z, Ctrl+Y and Ctrl+Shift+Z change nothing: the name, the Changes count and the alerts stay as they were.
    • Positive control: after Cancel request, or after the reload fails, the same keys on the same block undo, redo and redo again.
    • No source change. The docs now say the reload lock includes the keys.
  • Reran: ui_tests/test_v2_workflow_ask_ai.py 31 passed. functional_tests/test_v2_workflow_ask_ai.py 7 passed (normal and -O).

Part 3 (360207d97)

  • F1: Draft with AI focused Task name on a slow device. I used option (a): every tracked field always carries data-workflow-field-key, and the jump looks for the change key, then the field key, then the item.
    • Why (a). It doesn't depend on render timing, where another frame or a retry would only make the race rarer. 3a's highlight stays deferred, so a keystroke still never waits for the diff.
    • List root cause. Draft with AI applies its change and asks for the jump in the same pass. The jump's one animation frame looked only for data-workflow-change-key, which the highlight sets one deferred render later (useDeferredValue). On a slow CPU the jump ran first, fell back to the task item and focused its first control, Task name.
    • A second root cause on Flow, found by the throttled Flow case. The Draft with AI button disables itself while it works, so focus fell to the page, and history recovery (recoverAuthoring) asked the canvas to focus the block in the same batch as the change. The canvas could then take focus from the field after the jump. jumpToChange now clears that request (decision 14).
    • Unchanged. An impact confirmation still doesn't jump. The Run as re-approval notice isn't touched. On v3 the jump still selects the block by nodeId before it looks for the field.
    • Tests. The List case and a new Flow case wrap the click and the focus checks in a CDP throttle at rate 6, reset to 1 in finally. Each checks focus on the drafted instructions, again after a 1 s settle, and in a focusin log that may hold only the button and the field. The undo case now also jumps to a task field the turn no longer changes, so there's no highlight to find.
    • Fail-before and pass-after. With the three source changes reverted, the undo-jump case failed 3/3, the Flow case 3/3 and the List case 2/3. On 360207d97 all three passed in each of 5 runs.
    • Determinism. Throttling slows the main thread but not frame timing, so the List case on its own is timing-dependent: it failed in 6 of 8 runs without the field-key path. The undo-jump case and the focus log don't depend on timing.
    • Side effect. Jump to a task field that a later Undo or Revert put back now focuses that field, not the task's Task name.
  • Reran on 360207d97: ui_tests/test_v2_workflow_ask_ai.py 32 passed, the proposal file 3, change tracking 24, flow authoring 30, the workflow editor 9, the proposal card 22 and the File Sync jump case 1. functional_tests/test_v2_workflow_ask_ai.py 7 passed (normal and -O). Typecheck, build and the docs suites passed.
  • Follow-up 9a9590ce0. CodeQL flagged the throttle helper's CDP session as possibly unbound (see CodeQL below). The fix changes only that helper's control flow. ui_tests/test_v2_workflow_ask_ai.py passed 32/32 again, and both throttled cases passed on their own.

Mutation checks

For each mutant I changed the source, rebuilt as needed, confirmed that the named tests failed, then restored the source.

Mutant Caught by
M1: stale-draft guard removed Playwright stale-draft test
M2: trust the server's change list instead of the editor's diff logic test; Playwright client-diff test
M3: reply rendered with dangerouslySetInnerHTML functional and Playwright hostile-text tests
M4: rebase returns the raw candidate parity, logic and Playwright proposal tests
M5: rebase always keeps the live value parity, logic and Playwright "Done when" tests
M6: retain: false proposal held-thread test
M7: hold never released proposal held-thread test
M8: DiagramEditor passes retain: true options-default guard
M9: client limit 2,001 limit-parity guard
L18: a held keepKey still sweeps when it isn't marked quiet T1 logic check
L18b: the sweep drops held threads T1 logic check
L18c: the 50-thread cap evicts held threads T1 logic check
R1-guard: reloadEditing ignores the token reload cases (a) and (b)
R1-close-bump: closing doesn't supersede the reload reload case (a)
R1-saved-bump: saving doesn't supersede the reload reload case (d)
R1-finally: the reload lock is never released reload case (c)
R1-lock: draftLocked ignores the reload reload cases (a) and (c)
R1-blocked: undo and redo aren't blocked during a reload reload case (a)
R1-card: the card's Undo ignores the reload reload case (c)
T2-capture-off: if (false && draftLocked && …) both T2 cases
T2-capture-assist-only: if (assistPending && …) the reload T2 case; the Ask AI case still passes
T2-capture-z-only: Ctrl+Y isn't captured both T2 cases
T2-capture-meta-only: only Meta is captured, not Ctrl both T2 cases
F1-attr-off: no data-workflow-field-key on the field the undo-jump case; the List and Flow cases failed too in the final run
F1-query-off: the jump skips the field-key query the undo-jump case; the List and Flow cases failed too in the final run
F1-clear-off: jumpToChange doesn't clear the focus request the Flow Draft with AI case's focus log; the undo-jump case still passes, as it should
F1-revert: all three source changes reverted (fail-before) the undo-jump case 3/3 and the Flow case 3/3; the List case 2/3

One nuance on M4: the blank-Create Playwright case alone still passes, because newWorkflowDefinition() has no id key. The proposal and parity cases catch it.

Removing the token bump in openEditor is equivalent today. Every path to it, the list's Edit and Create behind the modal editor and a deep link that waits until no editor is open, runs only after the open editor closed, and closing already bumped the token. The bump stays as a guard. The coordinator's L2a, which drops the replay's turn slice but keeps the 20-item cap, is equivalent too, because a completed exchange always yields two items.

CodeQL

All 11 checks passed on the merge head a9dd15b28, including Analyze (python), and CodeQL reported "No new alerts in code changed by this pull request", with no annotations (check run 110328185558). All 11 had also passed on 9a9590ce0, with no new alerts.

On 360207d97 every other check passed, but CodeQL raised one new error, again in this PR's own browser tests:

  • py/uninitialized-local-variable at ui_tests/test_v2_workflow_ask_ai.py:1450, in the cpu_throttled helper that part 3 added.
    • CodeQL doesn't treat pytest.skip() as raising, so it saw a path where session is read after the except without being bound. At runtime the skip raises, so no test was affected.
    • 9a9590ce0 moves the throttle, the yield and the reset into the try statement's else clause, so they only run once the CDP session exists. This follows the repo's other CDP helper in test_v2_workflow_alert_notices.py, which does its work inside the try. A browser without the DevTools protocol still skips.

All 11 checks passed on dc9d22ad1 as well, with no new alerts.

On 9ee340bba every check passed as well, but CodeQL raised two new notes, both in this PR's own browser tests:

  • py/empty-except at ui_tests/fixtures/workflow_ask_ai.py:181. AssistStub.release() ignored a PlaywrightError with no comment. That error means the browser already cancelled the request, so there's no route left to answer. dc9d22ad1 adds a comment saying so.
  • py/unused-import at ui_tests/test_v2_workflow_ask_ai_proposal.py:42–54, for card_ui, connect_options, editor_assets and editor_browser.
    • These are pytest fixtures, which pytest finds by name. CodeQL skips an imported fixture only when it can trace the import to the @pytest.fixture definition.
    • They came in through the bare module name test_v2_orchestration_workflow_proposal_card, which resolves only through sys.path, and three of them were re-exports from other modules.
    • dc9d22ad1 imports each one from the module that defines it, with a ui_tests.-qualified path.

After the fix, ui_tests/test_v2_workflow_ask_ai_proposal.py passed 3/3 again and ui_tests/test_v2_workflow_ask_ai.py passed 25/25.

Known base failures

None of these are in files this PR changes.

  • test_v2_chat_context_picker.py::test_the_composer_no_longer_carries_a_write_only_selection asserts onPickerOpenChange={setPickerOpen} in Composer.tsx. The V2 tip has onPickerOpenChange={(open) => {, and this PR doesn't touch Composer.tsx.
  • test_v2_prompt_composer_card.py: test_the_server_records_what_the_bubble_needs and test_the_typescript_logic_checks_pass.
  • test_block_revision_assist.py::test_the_prompt_is_honest_about_positioning: functions_block_revision_assist has no ASSIST_SYSTEM_PROMPT anymore; it's now CHART_ASSIST_SYSTEM_PROMPT.
  • test_orchestration_plan_revision_routes.py::PlanRevisionRouteTests::test_document_removal_is_not_lost_when_editor_opens fails with "A selected document could not be opened…". It's backend only.
  • ui_tests/test_v2_workflow_loops.py::test_revoked_record_refresh_clears_the_previous_page: the "no longer have access" alert never appears in the Complete record inspection region of WorkflowExecutionHistory.tsx, which this PR doesn't touch. This PR's WorkflowsSection.tsx change only adds the reload wiring. The coordinator's brief listed this test as a known base failure.
  • ui_tests/test_v2_orchestration_composer.py and ui_tests/test_v2_orchestration_elicitation.py set their module-level _PAGE only in their main() runner. They need python <file>, where each passes 6/6, not pytest.
  • Docs. test_docs_link_integrity.py (2/5) finds broken links, none in this PR's lines. Two of them are features.yml links in the content-screening entry, which is the same on the base. test_docs_release_notes_integrity.py (0/1) finds generated release pages stale for 149 releases on a9dd15b28, 148 of them already stale on the base 5fc50a593. Against 4f778386b it was 148 and 147.

Honest notes

  • The route-parity case, which posts every request the tab builds through 3b's real Flask route, was added late, in 0228a36f7.
  • The M8 and M9 guards were added during the docs pass, in 9ee340bba.
  • CodeQL's two notes, both in this PR's own browser tests, turned up only after the PR was opened. dc9d22ad1 fixes them.
  • CodeQL's error on part 3's throttle helper also turned up only after the push. 9a9590ce0 fixes it.
  • Two hiding paths have no browser case of their own: an active run, and lost access. Both hide the tab through the same read-only check as a reader's editor, which the reader case covers.
  • The coordinator's review found the reload race (R1); two gaps in my tests, pruneThreads's held-thread guarantees (T1) and the Ctrl+Z/Ctrl+Y capture (T2); and Draft with AI's focus on a slow device (F1), which my unthrottled browser tests missed. 439d1115b, c17339aa8 and 360207d97 fix them.

Follow-ups

  • A Latest Feature card, once the guide's screenshot docs/images/guides/create-a-workflow-ask-ai.png exists.
  • Regenerate the generated release-notes pages (test_docs_release_notes_integrity.py).
  • Fix the base failures above. The prepared_chat_image_references break from Show what was extracted from V2 image uploads, index text-free images, and default image generation to 2025-04-01-preview #1585 in test_v2_prompt_composer_card.py is the most recent.
  • Give the assistant File Sync operations. Phase 4 made personal File Sync editable.
  • Support structural flow edits: For each and If blocks, and adding or moving tasks in a flow. Today the model explains instead.
  • Use structured build errors from Phase 2's dry run, so the correction round can repair them.
  • Port flowUnsupportedReason into the Python workflowForSave.
  • Optionally add a global in-flight cap.
  • Decide whether /api/workflows/draft-instructions should check enable_workflow_ai_assistant on the server as well. That's a product decision, so this PR flags it and doesn't change it. The classic Draft Workflow Instructions button doesn't follow the setting today.
  • Give the active-run and lost-access cases their own browser tests.
  • Disable Reload workflow while a save is in flight. A reload can start mid-save today; the request token keeps the outcome safe (reload case (d)), but there's no reason to offer it.
  • Check where focus lands when an editor that a reload reopened is closed. Its opener may no longer be in the page; I haven't verified this.
  • A stored conversation is out of scope for v1 (roadmap §9).

Closes #1548
Part of #1543

Adds the workflow editor's Ask AI side-panel tab on the shared assist thread,
calling POST /api/user/workflows/assist. Answers are checked against the draft
they were sent with, diffed client-side, and applied through applyAssist so
they are highlighted, undoable and reviewed before saving.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cancel the running turn before the thread is released, and remember a key an
editor held until its thread leaves the store, so a late answer never sweeps
the chat's idle image transcripts. Mark the locked editor fieldset aria-busy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- test_v2_workflow_ask_ai_logic.ts: 169 node checks for the request
  builder, response guard, code-point counting, client-side diff,
  candidate rebase, turn replay, errors and Draft with AI.
- test_v2_workflow_ask_ai.py: driver plus source-level checks
  (plain-text rendering, gating, default-off thread options).
- test_v2_workflow_ask_ai_parity(.py/_logic.ts): the editor's real
  requests go through 3b's parser and service, and the scripted answers
  come back through the editor's guard, rebase, diff and applyAssist.
- The fallback change wording now matches the server's summary rule.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
microsoft#1585 took 0.261.210, so 3c moves up one. Mechanical: only version strings change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Playwright suite for the Ask AI tab (25 items) over 3b's real assist
  pipeline with a scripted model: Done when, # placement, context
  documents, lock/Cancel/Retry/stale guard, the request contract,
  code-point limit, undo with skipped keys, every status, hostile text,
  Jump to (task field and flow block), hidden-tab cases, keyboard,
  narrow screens, Draft with AI and quick actions.
- Proposal-draft tests: Edit from the proposal card sends base null and
  a draft without an id, and Save reviews AI changes before accepting.
- Footer: on narrow screens Cancel and Save wrap together to a second
  row instead of wrapping the Save label.
- A one-word error such as the role check's "Forbidden" shows the
  sentence in its message.

Refs microsoft#1548

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
List the new dependencies in the order A2's guards expect, and update A1's guards for the opt-in code point count and the quick actions that leave the input alone. UTF-16 counting stays the default.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Through the proposal card's Edit: an Ask AI turn and closing the editor keep the chat's idle image thread, and once the editor lets its thread go the chat prunes as before.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…lizer-phase-3c-workflow-ask-ai-tab

# Conflicts:
#	application/single_app/config.py
Every tab-built request is now POSTed as its exact text to POST
/api/user/workflows/assist through 3b's route harness (the real route
body, helpers and gates on a closed Flask app over a scripted model), and
the tab replays the route's real 200 bodies. A new test checks that the
route read exactly the bytes the tab sent, ran the pipeline once for the
signed-in user, and answered uncached JSON.

Refs microsoft#1548

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Feature docs: the Ask AI tab, its thread, requests, rebase and change
  check, turn cards, failures, Draft with AI and accessibility
  (WORKFLOW_AI_ASSISTANT.md); the shared thread's opt-in options and the
  workflow limit (V2_SHARED_ASSIST_THREAD.md); AI-authored changes now
  coming from Ask AI and Draft with AI (WORKFLOW_EDITOR_CHANGE_TRACKING.md).
- Admin and guide: the setting now shows the tab and Draft with AI; a new
  "Ask AI to change a workflow" section and two troubleshooting rows.
- Release notes: v0.261.211 above microsoft#1585's v0.261.210, which is unchanged.
- test_v2_workflow_ask_ai.py: the four chat editors pass none of the
  shared thread's new options, and the client's instruction limit matches
  ASSIST_INSTRUCTION_MAX_LENGTH and is counted in code points.
- workflowAssist.ts: correct a comment about per-field Revert.

Refs microsoft#1548

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread ui_tests/fixtures/workflow_ask_ai.py Fixed
Comment thread ui_tests/test_v2_workflow_ask_ai_proposal.py Fixed
Paul Lizer (paullizer) and others added 4 commits October 1, 2026 00:12
Explain the ignored PlaywrightError in AssistStub.release(), and import
card_ui's fixtures by package path from the modules that define them, so
CodeQL can see they are pytest fixtures rather than unused imports.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review part 1 for microsoft#1593.

R1: WorkflowsSection's reloadEditing reopened the editor and cleared the
dirty flag after its reads finished, even if the editor had been closed,
saved or replaced by another workflow's editor in the meantime. A request
token now supersedes the reload when an editor opens, closes or saves, so
a late reload is ignored. While a reload loads, the draft it is about to
discard is locked (fields, Save, undo and redo, flow authoring, the
Changes tab, Send, the quick actions and each card's Undo). Cancel and
Escape still close the editor. A failed reload unlocks the draft.

T1: the logic checks now pin the held-thread guarantees of pruneThreads:
another conversation's sweep keeps an idle held workflow thread, the
50-thread cap keeps a held least-recently-touched thread, and a held
keepKey sweeps nothing even when it isn't marked quiet.

Browser tests cover a reload whose editor is closed, replaced by another
workflow's editor or saved while it loads, and a reload that fails.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review part 2 for microsoft#1593.

T2: no test pressed Ctrl+Z or Ctrl+Y while the draft was locked, so the
editor's key capture could be removed without a failure. Two browser
cases now press Ctrl+Z, Ctrl+Y and Ctrl+Shift+Z on a focused flow block,
which is not a text field, with an undo and a redo both available: once
while an Ask AI request runs and once while a reload loads. Nothing
changes. After Cancel request, or after the reload fails, the same keys
on the same block undo and redo.

The feature doc now says the reload lock includes the keys.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Draft with AI jumps to the instructions it drafted in the same pass that
applies the change, but the highlight that carries the change key renders
one deferred pass later. On a slow device the jump ran first, found no
change key and fell back to the task item, whose first control is Task
name. Every tracked field now carries its field key from its first render,
and the jump looks for the change key, then the field key, then the item.

On the Flow surface the disabled Draft with AI button dropped focus to the
page, so history recovery asked the canvas to focus the block, batched with
the change. jumpToChange now clears that block focus request, because the
jump places focus itself.

The List and Flow Draft with AI cases run under 6x CPU throttling and check
focus on the drafted instructions, again after a settle, and in a focus log.
The undo case also jumps to a field the turn no longer changes, which has
no highlight.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread ui_tests/test_v2_workflow_ask_ai.py Fixed
CodeQL (py/uninitialized-local-variable) does not model pytest.skip as
raising, so it flagged session in cpu_throttled as possibly unbound.
Move the throttle, yield and reset into the try statement's else
clause so they only run once the session exists. Behavior is
unchanged: a browser without the DevTools protocol still skips.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) marked this pull request as ready for review October 1, 2026 10:12
Paul Lizer (paullizer) and others added 2 commits October 1, 2026 06:20
microsoft#1594 (Phase 5) merged into paullizer-react-v2-ui first and took
0.261.212, so 3c moves to 0.261.213. This is a mechanical renumber of
this PR's own files: config.py, the release-notes heading, the docs'
version lines and mentions, and the functional and UI test headers and
version assertions. Every changed line differs only by the version.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Brings in Phase 5 (microsoft#1594, 0.261.212). Conflicts resolved:
- application/single_app/config.py: keep VERSION 0.261.213 (3c).
- docs/explanation/release_notes.md: the v0.261.213 section goes on top,
  then Phase 5's v0.261.212 section byte-identical, then v0.261.210 and
  earlier unchanged.

docs/admin/workflow.md and docs/_data/features.yml merged cleanly; both
sides are kept.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit 677c8f3 into microsoft:paullizer-react-v2-ui Oct 1, 2026
11 checks passed
Paul Lizer (paullizer) added a commit to paullizer/simplechat that referenced this pull request Oct 1, 2026
Status update 4 for the chat orchestration workflows roadmap (docs only).

- Status table: 3c done (microsoft#1593, v0.261.213), so Phase 3 is complete;
  Phase 5 done (microsoft#1594, v0.261.212); 6a in review (microsoft#1592).
- 3c shipped block and its follow-ups; Phase 5 shipped block and its
  follow-ups; 6a in-review note; 6b run deep-link guidance now covers
  Phase 5's workflowRunHref.
- Base failures on paullizer-react-v2-ui that the phase PRs report,
  with verified causes.
- Gotchas 62 (result answers outside the message list) and 63 (Retry
  and Edit of a result-backed turn).

Refs microsoft#1543

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Paul Lizer (paullizer) added a commit to paullizer/simplechat that referenced this pull request Oct 1, 2026
Mechanical: the 0.261.213 stamps become 0.261.214 in 6a's own
files: VERSION, the release-notes heading, the docs' version and
front-matter lines, the test headers' Version and Implemented in
lines, the assert_app_version_at_least floors and the Playwright
bootstrap seed. 0.261.213 now belongs to Phase 3c (microsoft#1593).

Part of microsoft#1546

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Paul Lizer (paullizer) added a commit to paullizer/simplechat that referenced this pull request Oct 1, 2026
Brings in the V2 workflow editor's Ask AI tab (0.261.213). Conflicts:
- config.py keeps VERSION 0.261.214.
- docs/admin/workflow.md takes 3c's "Enable AI Workflow Assistant"
  row, with the Ask AI tab and Draft with AI wording, and keeps the
  "Use Workflow Results In Chat" row directly after it.
- release_notes.md keeps the 0.261.214 section above 3c's 0.261.213
  section. Every older section is unchanged.
features.yml and the rest of workflow.md merged cleanly.
app_surface.yml is unchanged: 3c added no settings, and the
regenerated inventory matches the merged file.

Part of microsoft#1546

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants