Skip to content

fix(core): max_turns no longer clobbers a tripped input guardrail exception in streaming - #4606

Merged
seratch merged 2 commits into
openai:mainfrom
shoemoney:fix/max-turns-clobbers-guardrail-exception
Aug 23, 2026
Merged

fix(core): max_turns no longer clobbers a tripped input guardrail exception in streaming#4606
seratch merged 2 commits into
openai:mainfrom
shoemoney:fix/max-turns-clobbers-guardrail-exception

Conversation

@shoemoney

Copy link
Copy Markdown
Contributor

Summary

RunResultStreaming._check_errors() (src/agents/result.py, ~1047-1055) has this branch:

if (self.max_turns is not None and self.current_turn > self.max_turns and not self._max_turns_handled):
    max_turns_exc = MaxTurnsExceeded(f"Max turns ({self.max_turns}) exceeded")
    max_turns_exc.run_data = self._create_error_details()
    self._stored_exception = max_turns_exc

_max_turns_handled defaults to False and is only ever set True inside the opt-in error_handlers["max_turns"] path in run_internal/run_loop.py. If you don't register a custom max_turns handler (the default, and almost every caller), the flag never flips, so this branch is true on every single call to _check_errors() once current_turn > max_turns, and it unconditionally overwrites self._stored_exception with a brand-new MaxTurnsExceeded each time.

_check_errors() is called three times inside stream_events() (~939, ~975, ~1000), and one of those calls is in the finally block, so it always runs at least once more after the main loop has already broken out.

Input guardrails default to run_in_parallel=True (src/agents/guardrail.py:100). So the sequence that breaks is:

  1. A guardrail is still running in the background when the model starts blowing through turns.
  2. Turn count passes max_turns. _check_errors() runs, guardrail queue is still empty (guardrail hasn't finished), no guardrail exception yet, so nothing weird happens on that call.
  3. The guardrail finishes and trips. A later _check_errors() call drains the guardrail queue and correctly sets self._stored_exception = InputGuardrailTripwireTriggered(...).
  4. stream_events() exits its main loop and hits the finally block, which calls _check_errors() one more time. The guardrail queue is now empty (already drained in step 3), but current_turn > max_turns is still true and _max_turns_handled is still False, so the branch fires again, builds a fresh MaxTurnsExceeded, and stomps the InputGuardrailTripwireTriggered that was already stored.

The caller ends up with MaxTurnsExceeded instead of InputGuardrailTripwireTriggered. The documented except InputGuardrailTripwireTriggered: pattern silently never fires. This only happens in the default no-custom-handler path — if you register error_handlers={"max_turns": ...}, the handler path sets _max_turns_handled = True and none of this happens, which is exactly why it went unnoticed.

Fix

One line: set self._max_turns_handled = True right after storing the MaxTurnsExceeded in the default branch, mirroring what the handler path already does. Once it's set, subsequent _check_errors() calls skip the max_turns branch entirely and leave whatever's already in self._stored_exception alone.

Test plan

Added test_max_turns_does_not_clobber_input_guardrail_tripwire to tests/test_stream_input_guardrail_timing.py, using the existing ScriptedModel fixture setup already used by the other tests in that file and in test_max_turns.py. It runs an agent with max_turns=1, a tool that always gets called again (guaranteeing the turn limit is hit), and an input guardrail that trips after a short asyncio.sleep (simulating a real moderation call that resolves slightly after the fast scripted model turns). It asserts the caller sees InputGuardrailTripwireTriggered, not MaxTurnsExceeded.

Verified RED before GREEN:

  • With the fix reverted (git stash on just src/agents/result.py), the new test fails:
    FAILED tests/test_stream_input_guardrail_timing.py::test_max_turns_does_not_clobber_input_guardrail_tripwire
    AssertionError: Expected InputGuardrailTripwireTriggered, got MaxTurnsExceeded. The tripped guardrail result was silently clobbered by a freshly-minted MaxTurnsExceeded.
    1 failed in 0.29s
    
  • With the fix restored, it passes:
    tests/test_stream_input_guardrail_timing.py::test_max_turns_does_not_clobber_input_guardrail_tripwire PASSED
    1 passed in 0.26s
    
  • Did this revert/restore cycle twice to make sure it wasn't a fluke.

Ran the targeted suites with the fix in place:

python -m pytest tests/test_max_turns.py tests/test_stream_input_guardrail_timing.py tests/test_guardrails.py tests/test_run_error_details.py
123 passed in 1.34s

Also ran ruff check and ruff format --check on both changed files (clean), and mypy directly on src/agents/result.py (clean, no issues). I did try the repo's full make typecheck via .agents/skills/code-change-verification/scripts/run.sh, but it fails in my environment on unrelated optional-dependency modules (litellm, temporalio, sqlalchemy, the vercel/runloop sandbox extras) that aren't installed in a plain pip install -e . venv — none of those errors are in files this PR touches.

Issue number

None filed; found by local repro while testing guardrail + max_turns interaction in a streamed run.

…aming runs

RunResultStreaming._check_errors() re-creates MaxTurnsExceeded and overwrites
self._stored_exception on every call once current_turn > max_turns, because
_max_turns_handled is only ever set True by the opt-in max_turns error-handler
path in run_internal/run_loop.py. In the default (no custom handler) path it
stays False forever, so _check_errors() re-fires on each of its three call
sites in stream_events(), including the unconditional call in the finally
block.

Input guardrails default to run_in_parallel=True, so a tripped guardrail can
already be drained into _stored_exception as InputGuardrailTripwireTriggered
by an earlier _check_errors() call, then get silently overwritten by a
freshly-minted MaxTurnsExceeded on a later one. The documented
`except InputGuardrailTripwireTriggered` pattern never sees the tripwire.

Set _max_turns_handled = True in the default branch too, mirroring what the
handler path already does.

Added a regression test in test_stream_input_guardrail_timing.py that trips
an input guardrail on a run that also exceeds max_turns=1 and asserts the
caller gets InputGuardrailTripwireTriggered, not MaxTurnsExceeded.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e91a42c513

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/test_stream_input_guardrail_timing.py Outdated
test_max_turns_does_not_clobber_input_guardrail_tripwire ordered its race
with asyncio.sleep(0.05) inside the tripping guardrail. Under CI load or
slower model instrumentation, the guardrail could finish before
current_turn > max_turns was ever reached, in which case the test would
observe the correct exception even against the unpatched implementation
and silently stop being a regression test.

Replace the sleep with an asyncio.Event set by a max_turns error handler
that declines (returns None, so it falls through to the exact same
default raise path as no handler at all). The handler fires at the exact
moment the run loop establishes current_turn > max_turns, and the
guardrail now awaits that event before returning its tripwire, so it can
only ever resolve after the max-turns condition genuinely holds.

Verified 20/20 pass with the fix in place and 20/20 fail (on the correct
assertion) with src/agents/result.py's fix reverted.
@seratch seratch added this to the 0.22.x milestone Aug 23, 2026
@seratch seratch changed the title fix: max_turns no longer clobbers a tripped input guardrail exception in streaming fix(core): max_turns no longer clobbers a tripped input guardrail exception in streaming Aug 23, 2026
@seratch
seratch merged commit 1a55d70 into openai:main Aug 23, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants