Skip to content

test: wait for an approval dialog to compose before reading it - #479

Closed
hellices wants to merge 1 commit into
mainfrom
fix/478-eval-dialog-composed
Closed

hellices wants to merge 1 commit into
mainfrom
fix/478-eval-dialog-composed

Conversation

@hellices

Copy link
Copy Markdown
Owner

Closes #478

Summary

Two operation journeys failed on main's test (3.11) (run 39980913476) with NoMatches: No nodes match '.confirm-title' on ConfirmScreen() from the eval driver's handle(). The second NoMatches in that log, 'AgentPanel' on Screen(id='_default'), came from the journey tearing down after the first. #477 fixes that teardown error on its own.

The driver's readiness check returns as soon as app.screen is a ConfirmScreen, and handle() read the dialog's title straight away. Textual makes a pushed screen current before it composes the screen's widgets, so on a loaded runner the poll can land in between.

Verification

tests/evals/test_operation_journeys.py::test_a_dialog_still_composing_is_read_once_it_has_its_widgets delays ConfirmScreen's compose by 0.3 seconds. That holds the dialog in the pushed-but-empty state the runner hit. The test expects one matched dialog_observed and no hard failures.

  • RED: without the wait it failed three runs out of three with CI's exact pair of errors.
  • GREEN: with the wait it passes three runs out of three.

Run locally on macOS with Python 3.13 and the frozen lock. Each step ran directly with the project's environment rather than through make, because this machine cannot sync from PyPI:

Command Result
pytest -p no:tach tests/evals/ 556 passed
python scripts/check_source_size.py passed
ruff check src/ tests/ and ruff format --check src/ tests/ passed; 659 files already formatted
SKIP=mypy,source-size pre-commit run --files <changed files> passed
mypy no issues in 659 source files
tach check all modules validated
pytest -n 8 -q 13,750 passed, 25 skipped

Checklist

  • This closes an accepted issue in this repository that is assigned to me.
  • Tests cover the change, written to fail first where behavior changed.
  • make check passes locally.
  • User-facing docs and the README match the change, or nothing user-facing changed.
  • No security invariant is weakened: writes still need a keystroke approval
    and a fail-closed audit entry, and sensitive reads are still masked.

The operation-journey driver read `.confirm-title` as soon as app.screen
was a ConfirmScreen. Textual makes a pushed screen current before it
composes the screen's widgets, so on a loaded runner the poll landed in
between and the read raised "No nodes match '.confirm-title' on
ConfirmScreen()". That failed two journeys on main's 3.11 (#478); the
second NoMatches in that log came from the journey tearing down after
the first.

handle() now first waits, through _await_dialog_composed, until the
current dialog has its title or is no longer current; a dialog that
closes first falls through to the existing closed-before-a-decision
path. The new test delays ConfirmScreen's compose by 0.3 seconds and
still expects one matched dialog_observed and no hard failures; without
the wait it fails every run with CI's exact pair of errors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 23, 2026 17:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused test-only fix addresses the composition race, preserves expiry handling, and has targeted regression coverage with no identified blocking issues.

No open issues 🎉

What changed in this PR

This test-only change addresses #478 by preventing the operation-journey driver from reading an approval dialog before Textual has composed its widgets.

Changes:

  • Wait for the dialog’s title to exist or for the dialog to close, preserving existing closed-before-decision handling.
  • Add a regression test that delays composition and verifies successful dialog observation without hard failures.
File Description
tests/​evals/​test_operation_journeys.py Adds regression coverage for delayed dialog composition.
tests/​evals/​operation_app.py Waits for dialog composition before reading its contents.

Review effort: Balanced

@my-reviewer-agent my-reviewer-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

검토 범위에서 보고할 이슈가 없습니다.

@hellices

Copy link
Copy Markdown
Owner Author

Superseded by #482, which carries this pull request's commits unchanged (b72b03a) together with the other open fixes and the dependency security bumps, so all of them need one CI run and one merge instead of a rebase after each merge. Closing in favour of #482; the branch stays, so this can be reopened if needed.

@hellices hellices closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eval driver reads an approval dialog's title before Textual has composed it

2 participants