Repository navigation
Use pytest only and run PhysioNet checks as separate tests - #588
Merged
Merged
Conversation
Remove the unittest test framework from the test suite: - Test classes are plain pytest classes instead of unittest.TestCase subclasses. - self.assert*() calls are plain asserts (pytest.raises for assertRaises). - The temporary directory that used to be created in setUpClass() is a class-scoped fixture (temp_path) in conftest.py, and the CSV conversion test uses tmp_path instead of a "test_output" directory in the working directory. - assertAxesMatchSignal() in test_plot.py is a plain function. - The unittest.main() footers are removed. The tests are unchanged otherwise: the same 82 tests are collected. (unittest.mock is still used in test_processing.py.)
Tests that stream or download data from PhysioNet are now marked with "network". If such a test fails with a connection error, a timeout or an interrupted transfer, it is reported as skipped instead of failing (the reason is shown in the summary), so a slow or unreachable PhysioNet no longer breaks CI. Other failures, including failed assertions, still fail. 15 tests compared local sample data with the same data streamed from PhysioNet. They now take a "source" fixture that is parametrized with "local" and "physionet" (the latter marked "network"), so test_X[local] only uses local files and test_X[physionet] additionally streams from PhysioNet. The sequence of operations within each test is unchanged. A PhysioNet outage therefore only skips the [physionet] variants, and the local reading and writing checks always run (80 tests work offline instead of 65). Use `pytest -m "not network"` to run only these tests.
Use the same width (72 columns) and spacing ("# --- title --- #") for all
section banners.
PhysioNet regularly answers with "502 Bad Gateway", which is raised as a NetFileError and was not covered by the skip hook (only connection errors, timeouts and interrupted transfers were), so the affected [physionet] tests still failed in CI. Treat NetFileError with a 5xx status code or 429 (too many requests) like the other network errors. Client errors such as 404 or 403 still fail.
Member
|
Thanks, this looks good! One case the hook can't cover yet is a hang: wfdb doesn't set a timeout on its requests ( |
tompollard
approved these changes
Oct 28, 2026
wfdb did not set a timeout for its HTTP requests, so a server that accepts the connection but does not respond (for example, PhysioNet when it is overloaded) blocked forever: scripts hung, and in the tests a [physionet] variant waited until the CI job timeout instead of being skipped. Use a timeout of 10 seconds to connect and 60 seconds to wait for data (not for the whole transfer, so slow but steady downloads still work). A requests.exceptions.Timeout is raised if the server does not respond, which the test suite already reports as a skip for tests marked "network". Add a delay option to the dummy HTTP server in the tests to test this.
tompollard
approved these changes
Oct 28, 2026
tompollard
left a comment
Member
There was a problem hiding this comment.
Looks good to me, thanks again!
Collaborator
Author
|
Now let's wait for the tests to come back green. This time they should! |
tompollard
added a commit
that referenced
this pull request
Oct 28, 2026
#589) This is the last part of the cleanup that started with #578, #580, #581, #582 and #588. It builds the documentation without warnings, applies one docstring style everywhere, and moves the package to a `src` layout. Each commit can be reviewed on its own. 1. **Fix all Sphinx documentation warnings.** The docs build reported 94 warnings. They are fixed, and the docs workflow now builds with `-W --keep-going` so they stay fixed. - Duplicate object descriptions: repeated `automodule` directives and the classes documented on both the `wfdb` and `io` pages are marked `:no-index:`, so each object is indexed once. - `numpydoc_class_members_toctree = False`, because the autosummary stubs it links to do not exist. - Too short title underlines, and reStructuredText errors in docstrings (lists, literal blocks for the EDF/MATLAB/WAV format descriptions and tables, doctest continuation lines). - The `:no-index:` option needs Sphinx 7.2, so the pinned Sphinx version of the `docs` group goes from 7.0.0 to 7.2.6. 2. **Wrap docstrings and comments at 88 columns**, the same width as the code, and enable W505 in Ruff so it stays that way. This is a mechanical change, and the commit is listed in `.git-blame-ignore-revs`. 3. **Start docstrings on the opening quote line with a one-line summary.** Summaries that were wrapped over several lines or too long for one line are shortened, and the rest of the text moves into the description below. No code changes: comparing the ASTs with docstrings removed shows no difference. A few docstrings were reworded to fit, and the descriptions of `csv_to_wfdb` and `csv2ann` (which described reading a WFDB header) now match what the functions do. Also listed in `.git-blame-ignore-revs`. 4. **Move the package to `src/wfdb`.** This is the default module root of `uv_build`, so the `module-root` setting is removed. The tests now always run against the installed package, and `docs/conf.py` adds `src` to `sys.path`. Files are moved without changes, so `git blame` and `git log --follow` keep working. The wheel and the sdist build as expected. 5. **Ignore the two reformatting commits in git blame**, and add a note about the docstring style to `DEVELOPING.md`. The hashes stay valid with a merge commit, as in #582. 6. **Add `AGENTS.md` and `CLAUDE.md`** with short guidance for coding agents (commands and conventions). `CLAUDE.md` only points to `AGENTS.md`. Checked locally: Ruff lint and format, the docs build with `-W --keep-going`, and the tests that work offline on Python 3.11 with the lowest dependencies and on 3.15. @tompollard this concludes the tooling and infrastructure refresh. Any remaining changes are minor, such as fixing the ignored Ruff rules, which can be done later in follow-up PRs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth PR of the tooling series. It moves the test suite to plain pytest and makes tests that depend on PhysioNet independent from the local checks, so a slow or unreachable PhysioNet no longer fails (or hangs) CI. It consists of five commits (the last two were added after CI showed
502 Bad Gatewayerrors, and after a review comment about hangs):Convert tests from unittest to pytest. Test classes are plain pytest classes,
self.assert*()calls are plain asserts (pytest.raisesforassertRaises),setUpClass()is replaced by a class-scopedtemp_pathfixture inconftest.py(tmp_pathfor the CSV conversion test, which used to write totest_output/), and theunittest.main()footers are gone. Nothing else changes: the same 82 tests are collected with the same IDs. (unittest.mockis still used intest_processing.py.)Run local and PhysioNet checks as separate tests. See below.
Format section comments in
test_record.pyconsistently. The four section banners had different widths and spacing.Skip network tests on HTTP server errors, too. PhysioNet regularly answers with
502 Bad Gateway, which wfdb raises asNetFileError. The skip hook now also treats aNetFileErrorwith a 5xx status or 429 as a network error (404 and 403 still fail).Add timeouts to requests for remote files. This is the only change to library code.
wfdb/io/_url.pydid not set a timeout, so a server that accepts the connection but does not respond blocked forever (scripts hung, and CI jobs waited until the job timeout). Requests now time out after 10 seconds to connect and 60 seconds to wait for data (not for the whole transfer, so slow but steady downloads still work), and arequests.exceptions.Timeoutis raised, which the hook above reports as a skip for tests markednetwork. The constant iswfdb.io._url.TIMEOUT. A new test uses a dummy server with a delayed response. There is also a changelog entry.PhysioNet tests
Most tests that use PhysioNet first check the local sample data and then compare it with the same data streamed from PhysioNet. If PhysioNet is slow or down, the whole test failed (or hung), including the local checks.
sourcefixture that is parametrized withlocalandphysionet, e.g.test_1a[local]andtest_1a[physionet]. Thelocalvariant never uses the network. The sequence of operations within each test is unchanged (the PhysioNet read and its comparison are just conditional).physionetvariants and the twodl_databasetests are marked withnetwork(registered inpyproject.toml).tests/conftest.pyreports anetworktest as skipped (with the reason) if it fails with a connection error, a timeout or an interrupted transfer. Failed assertions, and the same errors in tests that are not markednetwork, still fail. The skip reasons are shown in the summary (-r s), so an outage is visible.pytest -m "not network"runs only the tests that work offline (documented inDEVELOPING.md). 80 of the 97 tests work offline, compared to 65 of 82 before.Testing
physionetvariants against the real PhysioNet, and they all pass. I did not run the twodl_databasetests (about 27 MB each).-n 2), and with a failed assertion in a marked test and a connection error in an unmarked test (both fail).ruff checkandruff format --checkpass.