Replace traced readiness query polls with untraced waits - #1848
Open
DABH wants to merge 4 commits into
Open
Conversation
Several contrib tests polled a workflow query (ready, ran, invoked) until the workflow reached its signal wait. A query issued while a workflow task is pending is buffered by the server and attached to every attempt of that task, so the poll can run twice or time out client-side before an attempt completes, and in the tracing tests each poll also runs through the instrumented client and worker interceptors. Add tests.helpers.wait_for_workflow_idle, which waits via describe() until the workflow is RUNNING with no pending workflow task, activity, child or Nexus operation, and use it in the langgraph summary tests, whose workflow parks on a signal wait right after its first task. The OpenAI Agents tracing tests also run on the time-skipping test server in CI, which never reports pending_workflow_task and returns no events for wait_new_event history polls on a running workflow. Those tests instead poll history through the plain client for a workflow task completion after the activity completion (assert_event_subsequence), which is exact on both servers. Remove the now-unused ready/ran/invoked queries and their backing flags.
DABH
marked this pull request as ready for review
September 11, 2026 05:39
xumaple
approved these changes
Sep 11, 2026
There was a problem hiding this comment.
🟢 Approval recommended
The test synchronization changes are consistent, targeted, and account for time-skipping server limitations.
Pull request overview
Replaces traced readiness queries with untraced synchronization to reduce tracing-test flakiness.
Changes:
- Adds a shared workflow-idle polling helper.
- Uses idle polling in LangGraph tests.
- Uses workflow-history polling in OpenAI tracing tests.
File summaries
| File | Description |
|---|---|
tests/helpers/__init__.py |
Adds wait_for_workflow_idle. |
tests/contrib/openai_agents/test_openai_tracing.py |
Replaces readiness queries with history polling. |
tests/contrib/langgraph/test_summary_fn.py |
Replaces query polling with idle waits. |
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Build wait_for_workflow_idle on assert_eventually so it shares the timeout, interval and cancelled-RPC retry of the other helpers, while still failing fast when the workflow is no longer running. Name the pending work in the timeout message and print the status name instead of its integer value. Document that timer-only waits count as idle, that the time-skipping test server never reports a pending workflow task, and that the OpenAI history probe assumes the first completed activity precedes the park.
A cancelled RPC used to retry immediately with no sleep and no deadline check, so a persistently cancelled probe spun until pytest's timeout and hid the helper's own diagnostic. Retry on the normal interval and give up at the deadline instead. Also name the elapsed timeout in the wait_for_workflow_idle message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
tests.helpers.wait_for_workflow_idle(handle): polls untraceddescribe()until the running workflow has no pending workflow task, activity, child or Nexus operation, and fails fast if the workflow is no longer running. The docstring records two limits: a timer-only wait counts as idle, and the time-skipping test server never reportspending_workflow_task.test_summary_fn.py: theran/invokedquery polls becomewait_for_workflow_idle.test_openai_tracing.py: the fourreadyquery polls become an untraced history poll (plain client) forACTIVITY_TASK_COMPLETEDthenWORKFLOW_TASK_COMPLETED.assert_eventually: a cancelled RPC now retries on the normal interval and gives up at the deadline instead of spinning.Why
A readiness query is attached to every attempt of a pending workflow task, so it can run twice or time out client-side, and in tracing tests each poll runs through the instrumented client. The OpenAI tests use history because CI also runs them on the time-skipping server, which never reports
pending_workflow_taskand returns no events forwait_new_eventpolls on a running workflow. #1830 does the same for LangSmith and can adopt the shared helper once both land.Testing
--flake-finder --flake-runs=20per converted test under CPU load: 120/120 on the dev server, 80/80 for the OpenAI tests on time-skipping (the originals also passed, so this is hardening, not a reproduced failure). After the follow-up commits: both files pass on the dev server and under time-skipping (the converted langgraph tests skip there), 12/12 repeated runs with four workers, lint and type checkers clean.