Keep higher-scoped fixtures alive across re-runs of failed subtests - #357
Merged
icemac merged 2 commits intoSep 1, 2026
Merged
Conversation
teddytennant
force-pushed
the
fix-subtest-premature-teardown
branch
from
August 27, 2026 12:45
a3b028e to
5e87576
Compare
icemac
reviewed
Aug 31, 2026
icemac
left a comment
Contributor
There was a problem hiding this comment.
Reviewed with a full diff pass plus runtime verification against pytest 9.1.1. The change does what it targets — subtest-only failures now keep module/class/session fixtures alive, and the pytest.skip()-after-failed-subtest path still re-runs without leaking. All 178 existing tests plus the 5 new ones pass, and the new tests themselves look right (module ordering is alphabetically stable; temporary_failure's test.res marker is per-tmpdir, so the scope parametrization does not cross-contaminate).
One regression, in the comment below.
Two things I looked at and am not filing:
--only-rerun/--rerun-exceptare bypassed for subtest-only failures, because_should_hard_fail_on_errorreturns early when the call report's outcome ispassed. Teardown and the re-run decision agree there, so no finalizer leak — pre-existing behaviour from the subtests support, not this PR.- The
failed_subtestsstash entry is never deleted on the final, non-re-run attempt. Harmless today: teardown only reads the current item's nodeid, and each nodeid runs one protocol per session.
— Comment created by Claude
teddytennant
force-pushed
the
fix-subtest-premature-teardown
branch
from
August 31, 2026 12:06
5e87576 to
71806f6
Compare
icemac
enabled auto-merge (squash)
September 1, 2026 05:57
icemac
approved these changes
Sep 1, 2026
lukasmasuch
pushed a commit
to streamlit/streamlit
that referenced
this pull request
Sep 7, 2026
…roup (#16852) Bumps the python-testing group with 1 update: [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures). Updates `pytest-rerunfailures` from 16.6 to 16.6.1 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst">pytest-rerunfailures's changelog</a>.</em></p> <blockquote> <h2>16.6.1 (2026-09-03)</h2> <p>Bug Fixes +++++++++</p> <ul> <li>Ensure teardown reports are passed to <code>pytest_runtest_logreport</code> for rerun attempts. (<code>[#237](pytest-dev/pytest-rerunfailures#237) <https://github.com/pytest-dev/pytest-rerunfailures/issues/237></code>_)</li> <li>Prevent superseded built-in subtest failures from remaining in the final test result when rerunning tests with pytest-xdist. (<code>[#350](pytest-dev/pytest-rerunfailures#350) <https://github.com/pytest-dev/pytest-rerunfailures/issues/350></code>_)</li> <li>Restore module, class, and session scoped fixture teardown when a <code>flaky</code> marker condition is falsy. (<code>[#351](pytest-dev/pytest-rerunfailures#351) <https://github.com/pytest-dev/pytest-rerunfailures/issues/351></code>_)</li> <li>Restore module, class, and session scoped fixture teardown when an error raised during teardown rules out a re-run. (<code>[#356](pytest-dev/pytest-rerunfailures#356) <https://github.com/pytest-dev/pytest-rerunfailures/issues/356></code>_)</li> <li>Keep module, class, and session scoped fixtures alive across re-runs of a test whose call phase failed through subtests only. (<code>[#357](pytest-dev/pytest-rerunfailures#357) <https://github.com/pytest-dev/pytest-rerunfailures/issues/357></code>_)</li> <li>Authenticate xdist StatusDB connections with a per-session token. (<code>[#358](pytest-dev/pytest-rerunfailures#358) <https://github.com/pytest-dev/pytest-rerunfailures/issues/358></code>_)</li> <li>Ensure negative rerun counts do not skip the initial test execution. (<code>[#359](pytest-dev/pytest-rerunfailures#359) <https://github.com/pytest-dev/pytest-rerunfailures/issues/359></code>_)</li> </ul> <p>Misc ++++</p> <ul> <li>Replace deprecated <code>Config.getvalue()</code> calls with <code>Config.getoption()</code>. (<code>[#345](pytest-dev/pytest-rerunfailures#345) <https://github.com/pytest-dev/pytest-rerunfailures/issues/345></code>_)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/dad07a7f360e96c0cbbc4301e36cdc439eb9a36b"><code>dad07a7</code></a> Preparing release 16.6.1</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/968fb6c1d3982ea35f041ecb62024e79016324bb"><code>968fb6c</code></a> Prevent negative reruns from skipping initial execution (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/360">#360</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/367df9044cbcf3bbf153269bdab995e1f9eb113c"><code>367df90</code></a> Keep higher-scoped fixtures alive across re-runs of failed subtests (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/357">#357</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/69ac0a7e66f1f55c1e94a2f237e2659250795ef7"><code>69ac0a7</code></a> Authenticate xdist StatusDB connections (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/358">#358</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/20decd37b22736b1a92ecb7a6ed598e7fd70cff1"><code>20decd3</code></a> Restore higher-scoped teardown when a teardown error rules out a re-run (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/356">#356</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/84af73d63c1ac3eeeeba73a5e3ac91a44078c7bb"><code>84af73d</code></a> Fix subtest rerun reporting with xdist (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/352">#352</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/d784cdda0b946c8788939255b6b72df67da82a19"><code>d784cdd</code></a> Test with pytest-xdist on CI (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/354">#354</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/9f85e063dac5387cb0b17ffcb8aadeca545292a8"><code>9f85e06</code></a> Restore higher-scoped teardown when a flaky condition is falsy (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/351">#351</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/eab650fb2a2665d61539aafda81ae549440c2834"><code>eab650f</code></a> Test rerun-except for setup errors (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/349">#349</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/0b0843a83fb7e79c37e626092a77c5f7cd49f4df"><code>0b0843a</code></a> Fix only_rerun marker precedence test (<a href="https://redirect.github.com/pytest-dev/pytest-rerunfailures/issues/348">#348</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-rerunfailures/compare/16.6...16.6.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Lockfile-only test dependency patch with bug fixes; low impact outside pytest rerun/xdist paths. > > **Overview** > Updates the locked **`pytest-rerunfailures`** dependency from **16.6** to **16.6.1** in `uv.lock` (sdist/wheel URLs and hashes only). No `pyproject.toml` or application code changes. > > The new patch release mainly fixes rerun/teardown behavior (higher-scoped fixtures, subtests, xdist reporting) and adds a per-session token for xdist StatusDB connections. CI that uses `--reruns` or the `flaky` marker should see more reliable fixture lifecycle and reporting. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 4ad93d6. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #351, the second of the two leaks I mentioned there. Independent of #356, though both touch the same condition in
pytest_runtest_teardown, so whichever lands second wants a one line rebase. Happy to do it.A call phase that fails through subtests only leaves the call report itself passing, so every entry in
_test_failed_statusesis false.pytest_runtest_teardownreads that as a finished test and lets the module, class and session finalizers run, while_should_not_rerundoes count the failed subtests and re-runs the test anyway. The higher-scoped fixtures are torn down and set up again on every attempt:With
--reruns 3that prints three setups and three teardowns. The same test failing normally, without subtests, prints one of each.The fix counts the failed subtests in the teardown check, the same way the re-run decision already does.
_get_num_failed_subteststakes a nodeid now instead of a report, since the teardown hook has no report to hand it.Tests cover class, module and session scope and the cross-module case. One is a guard rather than a reproduction: subtests that never pass exhaust the re-runs and still tear the module fixture down exactly once.