feat(testserver): add test_server() for in-memory server testing

[karangattu]

Overview

Adds test_server() — the Python counterpart to R Shiny's testServer(). It runs an app's server function against a mock connection, so inputs can be set and outputs asserted in process: no browser, no network server, no subprocess.

from shiny.testserver import test_server


def test_app():
    with test_server("app.py") as ts:
        ts.set_inputs(a=1, b=2)
        assert ts.get_output("name") == "foo"

        ts.set_inputs(a=3, b=4)
        assert ts.get_output("name") == "bar"

Exported from shiny.testserver: test_server, test_server_async, TestServerSession, AsyncTestServerSession, TestServerScope, AsyncTestServerScope, TestServerValue, TestServerValues, DEFAULT_CLIENT_DATA.

It lives beside shiny.run and shiny.playwright as a capability package rather than under shiny.pytest, which is reserved for pytest integration (create_app_fixture, ScopeName) and raises ImportError without pytest installed. test_server needs no pytest — it imports only stdlib and shiny internals — so it stays usable from unittest, a script, or a REPL.

API

Naming what to test

app is the single positional, defaulting to app.py beside the test file — the same convention as the local_app fixture:

test_server()                 # app.py beside the test file
test_server("myapp.py")       # another file beside the test file
test_server(path_to_app)      # absolute Path, used as-is
test_server(my_mod_server)    # server function, or a shiny.App

A str, or a Path that is not already a file, resolves against the caller's directory rather than the working directory (pytest runs from the rootdir, so a bare relative path would otherwise be unusable). This mirrors create_app_fixture, including its rule that passing a str guarantees a path stays relative.

Express apps are module-level code, so they are loaded from a file like any other app.

Context manager only

with is the only way to run a session — start()/close() are private. Teardown is therefore not something a caller can skip, so SHINY_TESTMODE, sys.path, sys.modules, and the app object's patched server attribute are always restored, including when an assertion fails. Every member that needs a running session raises a RuntimeError naming the fix.

test_server_async() is the same thing for async tests; the sync version drives its own event loop and raises a pointed error if one is already running.

set_inputs takes only **kwargs. An id that is not a valid Python identifier — a module's namespaced "counter-n" — goes through an unpacked dictionary, which CPython allows: set_inputs(**{"counter-n": 7}). Both it and flush() return what they were called on, so a sequence of interactions can be one chain:

assert ts.set_inputs(name="Ada").set_inputs(n=10).get_output("doubled") == "20"

Values

get_input / get_output / get_export return a TestServerValue, which compares equal to the underlying value so the common assertion needs no unwrapping:

assert ts.get_output("name") == "foo"        # compares .value
assert ts.get_output("name").status == "ok"  # or go rich

status is ok, error, or silent — the last meaning the item never rendered because a dependency was unavailable.

A TestServerValue never pretends to hold a value it does not have:

• value is MISSING, not None, unless status is "ok". A renderer can legitimately return None, so the two must be distinguishable. __post_init__ rejects any instance whose status, value, error, and traceback disagree. This is shiny.types.MISSING, not a second sentinel; MISSING_TYPE gained the __copy__/__deepcopy__ identity guard that requires (a deep-copied sentinel stops comparing equal) and a __repr__ naming itself.
• Comparing a valueless item raises ValueError. Comparing presumes a value exists, so returning "not equal" would answer a different question than the test asked — and would let assert ts.get_output("txt") != "hi" pass for an output that never rendered. Comparing two TestServerValues still compares structurally and never raises.
• An unknown name raises KeyError naming what is available, rather than returning a placeholder. There is no "missing" status: a TestServerValue always describes something real.
• error and traceback travel together, both None when nothing raised. "" was a value pretending to be a traceback.

to_values() captures a TestServerValues snapshot of copies that outlives the block. dict(values) and dict(session) are the plain-data form, keyed on status — an item that produced nothing has no value key at all — and is JSON-serializable. TestServerValues.is_ok folds every output and export plus any fatal error into one answer; there is no per-item is_ok, since that would restate status == "ok".

Modules

A module's ids are namespaced, so they can be read as the session sees them:

ts.set_inputs(**{"counter-n": 7})
assert ts.get_output("counter-label") == "n=7"

…or through a scope, which is the test-side counterpart of the SessionProxy a module's server function receives — bare ids in, bare ids out:

with ts.make_scope("counter") as counter:
    counter.set_inputs(n=7)
    assert counter.get_output("label") == "n=7"

assert ts.get_output("counter-label") == "n=7"   # still visible on the session

make_scope, root_scope, and the public ns: ResolvedId mirror shiny.Session, and id resolution reuses ResolvedId, so the - separator and the rules for a legal id are shared with the rest of shiny rather than restated. Scopes nest (outer-inner). to_values() / dict() / is_ok / error are all scoped, so a scope reports exactly what a test of that module server on its own would: a sibling module's failure leaves is_ok true, while a fatal session error counts against every scope since it belongs to no namespace. A scope's __enter__/__exit__ are deliberate no-ops — the session owns the app's lifetime — and exist only so a module's assertions can be indented under the module they belong to.

Setting a session-wide .clientdata_* id through a scope raises rather than quietly building an id nothing reads; client data is read from the root in real Shiny too.

Client-side stand-ins

A browser reports things the server reads back: each output's size, the pixel ratio, the URL parts. Without them, render.plot raises a silent exception and produces nothing, and session.clientdata.url_pathname() never resolves — invisibly, with a passing is_ok.

The session sends defaults at startup, overridable per test:

with test_server(server, client_data={"output_width": 300}) as ts:
    ...

Keys are named after the readers on shiny.session.ClientData: an output_* key applies to every output, everything else is session-wide. None means the same as {}.

Changes outside shiny/testserver/

shiny/session/_session.py gains per-item error tracebacks. The output error handler called traceback.print_exc() and discarded the result, so a failed output reported a message with no origin. It now passes the formatted traceback to set_error(), which stores it in a new test_tracebacks dict only when _record_test_values is on.

Deliberately kept out of both the outbound error message and _build_test_snapshot(): the former is sent to the browser and the latter is served over HTTP at /session/{id}/dataobj/shinytest, and neither should carry a stack trace — that is what sanitize_errors exists to prevent. test_server reads it in process.

_build_test_snapshot() was also split out of the HTTP handler so the session can build a snapshot without going over the wire.

shiny/types.py: MISSING_TYPE gains __copy__/__deepcopy__ returning self, and a __repr__ of "MISSING" / "DEPRECATED". A sentinel is identified by identity, so copy.deepcopy minting a second one is a bug wherever it happens — dataclasses.asdict/astuple deep-copy their fields.

Testing

• tests/pytest/test_test_server.py — target resolution, Express apps, the context-manager contract, repeated and chained set_inputs, value statuses, equality and repr, construction-time validation, dict conversion, per-item tracebacks, client data, Core and Express modules, and scopes.
• test_test_server_flavors_present_the_same_api guards against drift across TestServerSession / AsyncTestServerSession / TestServerScope / AsyncTestServerScope: the public name sets must match, only set_inputs/flush may differ (in being awaitable), the read half must take the same arguments, dict() must yield the same keys, and each flavor must have the context-manager protocol it advertises.
• tests/pytest/test_types.py — the MISSING/DEPRECATED sentinels survive copying and repr by name.
• tests/pytest/test_playwright_simulation_equivalents.py — the same assertions as existing Playwright tests, run in memory.
• tests/pytest/test_controller_documentation.py — new parametrized documentation-coverage test. test_quartodoc_configs only rejected duplicates and the existing completeness test was scoped to shiny.playwright.controller, so an export could ship with no published docs. test_all_testing_exports_are_documented closes that for both shiny.pytest and shiny.testserver.

Full unit suite passes. make format check-lint check-types, make check-pyright, and make docs are all clean.

Note for review

Every new name is registered in docs/_quartodoc-testing.yml under a new "Test server" section, with @no_example() on the two functions to match create_app_fixture.