Skip to content

Fix Orchestrate in shared chats, Microsoft 365 steps and agents in plans - #1676

Merged
Paul Lizer (paullizer) merged 5 commits into
paullizer-react-v2-uifrom
paullizer-orchestrate-mode-chat-bugs
Oct 7, 2026
Merged

Paul Lizer (paullizer) merged 5 commits into
paullizer-react-v2-uifrom
paullizer-orchestrate-mode-chat-bugs

Conversation

@paullizer

Copy link
Copy Markdown
Contributor

Summary

  • Orchestrate action and agent steps work again, including Microsoft 365. They now trust the signed-in session the way manual chat does: access to the conversation, run and exact agent or action is checked on every read, but configuration is no longer captured and compared. Microsoft 365 action steps stopped failing with "A required retained result is unavailable or changed", local agents run as plan steps (including their own Microsoft 365 actions), and an @agent/model tag seeds the plan. Web search, URL fetch and Deep Research keep configuration attestation.
  • Orchestrate respects shared conversations. The composer applies the shared send rule first, so "person hi" is posted to that person instead of being planned. Only the person who started the conversation plans in it; the question and answer are posted to the shared thread for everyone. Plans live in a hidden backing record with the shared conversation's ID, and every lookup by that ID (reopen, images, summary, export, uploads, scope lock, citations, MCP, Microsoft 365 cards, chat stream) treats it as part of the shared conversation. Deleting the shared conversation deletes the backing.
  • Microsoft 365 in shared conversations: your request is your consent. Asking for your own Microsoft 365 data in a shared conversation (chat or plan) shares what that request reads, recorded as a shared_by_request audit event, without a pending approval. Workflow Run as approvals and history publication approvals are unchanged.
  • This deliberately relaxes the agent/action attestation layer (approved for functionality first). A lighter mid-run edit check is a possible follow-up.
  • Known limitations (documented in the shared conversations fix doc): participants see the question and answer but not the plan card or run details, files and charts a plan generates aren't copied into the shared thread, and the planner reads earlier planned turns only.

Linked issue

Fixes #1659
Fixes #1660
Fixes #1661

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only (0.261.268 → 0.261.269)
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Testing / validation

All runs are on this branch rebased onto paullizer-react-v2-ui at 7bc21b6, Python 3.12 on Windows, with PYTHONPATH=application/single_app;functional_tests (plus a local shim that sets werkzeug.__version__ for the installed Werkzeug).

New and changed tests, each with python -m pytest <file> -q -p no:cacheprovider:

  • functional_tests/test_orchestration_session_trusted_sources.py: passed (9)
  • functional_tests/test_orchestration_m365_actions.py: passed (18)
  • functional_tests/test_orchestration_shared_conversations.py: passed (10)
  • functional_tests/test_orchestration_shared_backing_lookups.py: passed (12)
  • functional_tests/test_m365_shared_request_consent.py: passed (12)
  • functional_tests/test_m365_provider_core.py: passed (124)
  • Updated attestation-era tests: test_orchestration_external_bootstrap.py passed (72), test_orchestration_external_sources.py passed (39, plus 102 subtests), test_orchestration_external_configuration_capture.py passed (110), test_orchestration_external_preflight_adapter.py passed (227), test_orchestration_local_agent_capture.py passed (30), test_orchestration_turn_recovery_identifiers.py passed (6), test_document_provenance_versions_and_guards.py passed (161)
  • node functional_tests/test_v2_shared_orchestration_routing.mjs: passed (8/8)
  • python ui_tests/test_v2_shared_orchestration_routing.py: passed (4/4). Three of the four fail against the upstream composer.

Regression comparison:

  • I ran 253 functional test files, one python -m pytest functional_tests/<file> -q -p no:cacheprovider each. They cover every test_orchestration_*, test_agent_delegation* and *m365* file, plus the collaboration, conversation, export, scope lock, retention, MCP, upload and provenance tests for the changed lookups. Result: 8,767 passed, and 149 failed in 31 files.
  • I ran the same 31 files on upstream 7bc21b6 without this change. They give the identical 149 failures, so these are pre-existing in this environment. For example, test_orchestration_plan_revision_routes.py fails setup on both with ImportError: cannot import name 'enabled_required'. No regressions.
  • Before the rebase, I also ran the first commit on its own against the 28 orchestration, Microsoft 365 and agent delegation files it affects. Its only failures were the same pre-existing upstream ones.

UI:

  • npm run typecheck and npm run build in application/v2_ui: passed.
  • All 27 ui_tests/test_v2_orchestration_*.py files, after npm --prefix .\application\v2_ui run build -- --outDir ..\..\ui_tests\artifacts\orchestration-plan-editor: 26 passed. test_v2_orchestration_plan_editor_backend.py errors in setup, because it reuses test_orchestration_plan_revision_routes.py, which has the pre-existing ImportError above.
  • Other V2 chat UI tests on the same build passed: test_v2_collaboration_ux.py (4), test_v2_new_chat_reset.py (7), test_v2_chat_image_uploads.py (15), test_v2_chat_deleted_messages.py (2), test_v2_inline_maps.py (5), test_v2_media_galleries.py (5), test_v2_plan_editor_references.py (11), test_v2_sidebar_conversation_scroll.py (9) and test_v2_workflow_ask_ai_proposal.py (3).

Docs and route policy:

  • python functional_tests/test_docs_app_surface_coverage.py: 7/7 passed
  • python functional_tests/test_docs_site_quality.py: 6/6 passed
  • python functional_tests/route_tests/test_route_blueprint_policy_inventory.py: 12/12 passed
  • python functional_tests/route_tests/test_route_unauthenticated_policy_contract.py: 7/7 passed
  • python functional_tests/route_tests/test_route_policy_test_coverage.py: 2/2 passed

Not run, because they need a deployment. After deploying, repeat the scenarios from the issues:

  • In a shared chat with Orchestrate on, "person hey" is posted with no plan.
  • In a shared chat, the person who started it asks "M365 what are my emails". The plan uses the agent, and every participant sees the answer.
  • In a personal chat with Orchestrate on, "what are my emails" works.
  • App Insights shows no external_configuration_unavailable for action_invoke or agent_invoke steps.

Documentation

  • Release notes updated, or not needed (docs/explanation/release_notes.md, 0.261.269)
  • Feature documentation updated, or not needed (docs/admin/orchestration.md, docs/guides/microsoft-365-conversation-data.md)
  • Fix documentation updated, or not needed (ORCHESTRATION_SESSION_TRUSTED_ACTION_AGENT_STEPS_FIX.md, ORCHESTRATION_SHARED_CONVERSATIONS_FIX.md)

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security()) (no new routes)
  • Settings sent to non-admin frontends use sanitize_settings_for_user() (no settings payload changes)
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

Paul Lizer (paullizer) and others added 2 commits October 6, 2026 18:46
Action and agent steps now trust the signed-in session the same way manual chat
does, instead of re-attesting the live configuration. Microsoft 365 action steps
no longer fail with "A required retained result is unavailable or changed", and
local agents (including agents with Microsoft 365 actions) can run as plan steps.

- Session-trusted references for action and agent sources (access is still
  re-checked on every read; older attested results stay readable).
- Agent steps enter a Microsoft 365 step scope for the agent's own actions.
- Specific refusal codes are kept in diagnostics, and integration failures get
  a dedicated message.
- Composer seeds an explicit @agent or @model tag into the Orchestrate plan.

Fixes #1660
Fixes #1661

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Composer: in a shared conversation, Orchestrate applies the manual chat send
  rule first. A message for people only is posted to the thread with no plan.
- The conversation owner's requests are planned against a hidden backing record
  at the shared id. The question and answer are mirrored into the shared thread
  after membership is rechecked. Lookups by the shared id skip the backing,
  republished answers update the shared copy, and deleting the shared
  conversation deletes the backing.
- Microsoft 365: in a shared conversation the requester's explicit request is
  their consent, recorded as a shared_by_request audit event. Workflow "Run as"
  approvals are unchanged.
- Fix docs, admin and guide updates, release notes, and tests for this change
  and the previous commit.

Fixes #1659

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread functional_tests/test_orchestration_shared_conversations.py Fixed
Comment thread functional_tests/test_orchestration_shared_conversations.py Fixed
Comment thread application/single_app/functions_orchestration_collaboration.py Fixed
Comment thread functional_tests/test_orchestration_shared_backing_lookups.py Fixed
Comment thread functional_tests/test_m365_provider_core.py Fixed
Comment thread application/single_app/functions_orchestration_collaboration.py
Comment thread application/single_app/functions_orchestration_collaboration.py
Comment thread application/single_app/functions_orchestration_collaboration.py
Comment thread application/single_app/functions_orchestration_collaboration.py
Comment thread application/single_app/functions_collaboration.py
Paul Lizer (paullizer) and others added 2 commits October 6, 2026 20:25
Use the personal conversation authorization helper for summary reads, clarify collaboration message return shapes, and remove CodeQL error/notice patterns from shared Orchestrate tests and helpers.

Refresh affected isolated tests so they cover the new route dependencies and version policy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Capture container reads, HTTP calls and lookups before asserting in the
  shared Orchestrate tests, so no checked operation runs inside an assert.
- Add test_orchestration_collaboration_codeql_alerts.py: the collaboration
  mirror helper returns three values on every path, the orchestration module
  uses one collaboration import style, and both collaboration modules import
  cold and offline in either order, in normal and optimized Python.
- List the new test in the shared conversations fix doc.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread application/single_app/functions_orchestration_collaboration.py
Comment thread application/single_app/functions_orchestration_collaboration.py
… 0.261.270

The Operations settings parity (#1675) took 0.261.269, so this pull request's
version, release notes section and version references move to 0.261.270.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit d8eeec1 into paullizer-react-v2-ui Oct 7, 2026
11 checks passed
Paul Lizer (paullizer) added a commit that referenced this pull request Oct 7, 2026
…ditors to 0.261.271

Brings in the shared Orchestrate fixes (#1676), which took 0.261.270, the
version this pull request had moved to.

- config.py: VERSION 0.261.271 (both sides had moved to 0.261.270).
- release_notes.md: this pull request's section moves to v0.261.271 above
  the base's v0.261.270 Orchestrate section, which is kept unchanged.
- Renumbered this pull request's 30 other 0.261.270 references; the fix doc's
  version transition now reads 0.261.270 -> 0.261.271. The base's own
  0.261.270 references are untouched.
- No other files changed on both sides; docs/_data/app_surface.yml
  regenerated and unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants