From 140202c5febec6460da00e33d5d14ab69eeaca11 Mon Sep 17 00:00:00 2001 From: Barret Schloerke Date: Fri, 11 Sep 2026 16:55:04 -0400 Subject: [PATCH 1/3] test: drive servers in memory with test_server() / testServer() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit py-shiny#2470 added `shiny.testserver.test_server()`, the Python counterpart to R's `testServer()`. A ui.tsx server is only reactive computation, so "input X produces output Y" *is* the server, and both languages can now assert it with no browser. Package tests: - `pkg-py/tests/test_in_memory_server.py` — `reactive_output` publishing through a real session (`test_reactive_output.py` only reaches `Renderer.transform()`), module namespace isolation via `make_scope()`, and output error statuses. Reuses the Playwright fixture apps; needs no e2e extras, so it lives in the unit suite. - `pkg-r/tests/testthat/test-render.R` — the R mirrors of those two, with the divergence stated: R's `testServer()` re-raises where Python reports `.status`. Example tests, one per app with a Python server, plus R counterparts for the two that ship an `app.R` (07-plotly had no R test directory): - two `FEATURES.md` leaves were wrong and are now correct and tested: 05-temperature and 08-input-handler both claimed `input.x()` is `None` before the client's first message, so the server returns `None` / `"—"`. It raises a silent exception, so those branches are unreachable from a real client. - `(test)` markers added across seven example trees for the wire shapes, thresholds, pluralization, event handling, and `AsIs` wrapping these pin. Docs and skills: - the Quarto article and its R vignette mirror are retitled "Testing" and gain a server-testing half ahead of the wire-tap material. - `shinyreact-build-app/references/testing.md` said "`[py]` has no `testServer()` equivalent … This is a real gap"; it now documents `test_server()`, and `shinyreact-convert-app`'s verify phase goes from three layers to four. Shipped copies refreshed with `make update-skills`. - documented, because each cost time to find: pass an absolute `Path`; an untyped id needs no `:shinyreact.default` suffix but a typed id does; an event input needs two `set_inputs` calls; an unset input is `status == "silent"`, not a `None` value. `test_server()` is newer than shiny 1.7.0, which the docs and skills say explicitly. Two upstream issues filed from this work: posit-dev/py-shiny#2492 (a silent `req()` failure is invisible in memory — `get_output()` keeps the stale value while the browser blanks the output) and posit-dev/py-shiny#2493 (`status == "silent"` means "never rendered", inverting Shiny's own meaning). Drive-by: `test-examples.R` read every file in an example directory before filtering to `app.R` + `www/`, so `__pycache__` produced 208 `readLines()` warnings. --- .claude/skills/shinyreact-build-app/SKILL.md | 23 +++-- .../references/testing.md | 86 ++++++++++++++--- .../skills/shinyreact-convert-app/SKILL.md | 16 ++-- examples/01-hello/FEATURES.md | 15 ++- examples/01-hello/tests/test_outputs.py | 62 +++++++++++++ .../01-hello/tests/testthat/test-outputs.R | 66 +++++++++++++ examples/02-columns/FEATURES.md | 14 ++- examples/02-columns/tests/test_moves.py | 64 +++++++++++++ examples/05-temperature/FEATURES.md | 16 ++-- examples/05-temperature/tests/test_display.py | 66 +++++++++++++ examples/06-data-frame/FEATURES.md | 12 ++- examples/06-data-frame/tests/test_table.py | 42 +++++++++ examples/07-plotly/FEATURES.md | 14 ++- examples/07-plotly/tests/test_scatter.py | 43 +++++++++ examples/07-plotly/tests/testthat.R | 10 ++ .../07-plotly/tests/testthat/test-outputs.R | 54 +++++++++++ examples/08-input-handler/FEATURES.md | 11 ++- examples/08-input-handler/tests/test_when.py | 46 ++++++++++ examples/10-bookmarking/FEATURES.md | 10 +- .../10-bookmarking/tests/test_greeting.py | 51 ++++++++++ examples/README.md | 20 ++++ pkg-py/README.md | 2 +- pkg-py/docs/_quarto.yml | 3 +- pkg-py/docs/articles/testing.qmd | 84 +++++++++++++++-- .../skills/shinyreact-build-app/SKILL.md | 23 +++-- .../references/testing.md | 86 ++++++++++++++--- .../skills/shinyreact-convert-app/SKILL.md | 16 ++-- pkg-py/tests/test_in_memory_server.py | 92 +++++++++++++++++++ pkg-r/README.md | 2 +- .../inst/skills/shinyreact-build-app/SKILL.md | 23 +++-- .../references/testing.md | 86 ++++++++++++++--- .../skills/shinyreact-convert-app/SKILL.md | 16 ++-- pkg-r/tests/testthat/test-examples.R | 15 ++- pkg-r/tests/testthat/test-render.R | 67 ++++++++++++++ pkg-r/vignettes/articles/testing.Rmd | 74 +++++++++++++-- pkg-r/vignettes/shinyreact.Rmd | 5 +- 36 files changed, 1200 insertions(+), 135 deletions(-) create mode 100644 examples/01-hello/tests/test_outputs.py create mode 100644 examples/01-hello/tests/testthat/test-outputs.R create mode 100644 examples/02-columns/tests/test_moves.py create mode 100644 examples/05-temperature/tests/test_display.py create mode 100644 examples/06-data-frame/tests/test_table.py create mode 100644 examples/07-plotly/tests/test_scatter.py create mode 100644 examples/07-plotly/tests/testthat.R create mode 100644 examples/07-plotly/tests/testthat/test-outputs.R create mode 100644 examples/08-input-handler/tests/test_when.py create mode 100644 examples/10-bookmarking/tests/test_greeting.py create mode 100644 pkg-py/tests/test_in_memory_server.py diff --git a/.claude/skills/shinyreact-build-app/SKILL.md b/.claude/skills/shinyreact-build-app/SKILL.md index 0a922ace..0a225bee 100644 --- a/.claude/skills/shinyreact-build-app/SKILL.md +++ b/.claude/skills/shinyreact-build-app/SKILL.md @@ -383,8 +383,8 @@ a list. Use `type: "shinyreact.asis"` for the parsed value untouched. Do not stop at "the code is written". Three steps, in order: 1. **Factor pure logic out of the app file** — binning, formatting, - conversions go in a module beside the app. Logic inside `app.py` / - `app.R` next to the page call cannot be reached by a test at all. + conversions go in a module beside the app, where a test can import them + without starting a session at all. 2. **Write down what the app does, in plain English, before the tests.** An agent that writes the client and then writes the client's tests is agreeing with itself — both encode the same misunderstanding. A @@ -394,16 +394,19 @@ Do not stop at "the code is written". Three steps, in order: directory** — `pytest`, or `[r]` `shiny::runTests()`, which needs the `tests/testthat.R` + `tests/testthat/` layout. -`[r]` `shiny::testServer()` drives the reactive graph with no browser: -`session$setInputs(bins = 9)` then assert on `output$dist_data`, which is the -JSON value the client would have received. **`[py]` has no equivalent**, so -factoring logic into an importable module matters more there. +Both languages drive the reactive graph with no browser, which for a `ui.tsx` +app is most of the server: `[r]` `shiny::testServer()` (`session$setInputs(bins += 9)`, then assert on `output$dist_data`) and `[py]` +`shiny.testserver.test_server()` (`ts.set_inputs(bins=9)`, then +`ts.get_output("dist_data")`). Either way the value you assert is the JSON the +client would have received. [`references/testing.md`](references/testing.md) has the four layers, the test -layout for each language, `testServer()` for plain and module servers, how to -mount the real `www/ui.js` against a fake Shiny, and the traps that cost time -(React ignores raw `change` events; debounce coalesces within a tick even at -`debounceMs: 0`). +layout for each language, `testServer()` / `test_server()` for plain and module +servers — including the input ids that need a `:type` suffix and the event +inputs that need two `set_inputs` calls — how to mount the real `www/ui.js` +against a fake Shiny, and the traps that cost time (React ignores raw `change` +events; debounce coalesces within a tick even at `debounceMs: 0`). ## Debugging diff --git a/.claude/skills/shinyreact-build-app/references/testing.md b/.claude/skills/shinyreact-build-app/references/testing.md index 8fa6e978..245ede5e 100644 --- a/.claude/skills/shinyreact-build-app/references/testing.md +++ b/.claude/skills/shinyreact-build-app/references/testing.md @@ -3,8 +3,11 @@ Do not stop at "the code is written". At minimum: 1. **Factor pure logic out of the app file.** Binning, formatting, conversions - go in a module beside the app so a test can import them; logic sitting - inside `app.py` / `app.R` next to the page call cannot be reached at all. + go in a module beside the app so a test can import them directly, with no + session at all. Logic left inside `app.py` / `app.R` next to the page call + is still reachable — `testServer()` / `test_server()` below drive the app + itself — but only through an input, which is a slower and blunter tool than + calling a function. 2. **Write down what the app does, in plain English, before the tests** — a behavior file beside the app, one atomically checkable claim per line ("the caption reads `N eruptions in M bins`, singular `bin` at M=1"). An @@ -16,7 +19,7 @@ Do not stop at "the code is written". At minimum: | Layer | Proves | Cost | |---|---|---| | pure functions in their own module | binning, formatting, conversions | trivial — always do this | -| `[r]` `shiny::testServer()` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | +| `shiny::testServer()` `[r]` / `shiny.testserver.test_server()` `[py]` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | | the client mounted in jsdom against a fake Shiny | rendering, input wiring, wire ids, status handling | low, and it exercises the file the app ships | | Playwright | layout, real Shiny, real bindings | high; reserve for what the others cannot see | @@ -31,7 +34,8 @@ myapp/ faithful.py the factored logic www/ui.js tests/ - test_faithful.py [py] pytest + test_faithful.py [py] pytest — the factored logic, called directly + test_outputs.py [py] pytest — the app, via test_server() testthat.R [r] runner: library(testthat); test_dir("testthat") testthat/ test-histogram.R [r] @@ -58,11 +62,17 @@ sys.path.insert(0, str(EXAMPLE)) from faithful import histogram, waiting # noqa: E402 ``` -## `[r]` Testing the server with `testServer()` +## Testing the server: `testServer()` `[r]`, `test_server()` `[py]` -`reactive_output` is an ordinary Shiny render function, so `shiny::testServer()` -drives it with no browser and no client: set inputs, read outputs, and get the -JSON value the client would have received. +**This is the highest-value layer for a `ui.tsx` app.** The server contains +only reactive computation, so "input X produces output Y" *is* the server, and +both languages can assert it with no browser and no client: set inputs, read +outputs, and get the JSON value the client would have received. + +### `[r]` `shiny::testServer()` + +`reactive_output` is an ordinary Shiny render function, so `testServer()` +drives it directly. ```r test_that("the histogram recomputes when bins changes", { @@ -90,10 +100,62 @@ which is most of what a shinyreact server does. Module servers work the same way: `testServer(card_server, args = list(id = "left"), { ... })`. -**`[py]` has no `testServer()` equivalent.** Cover the Python server by -factoring its logic into a module and testing that with pytest, and let the -jsdom and Playwright layers cover the wiring. This is a real gap, not an -oversight in your app. +### `[py]` `shiny.testserver.test_server()` + +The Python counterpart (py-shiny#2470, so newer than shiny 1.7.0). It loads the +app file — Express or Core, `shiny.App` or `shinyreact.ReactApp` — and runs its +server against a mock connection: + +```python +from pathlib import Path +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_the_histogram_recomputes_when_bins_changes(): + with test_server(APP) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_data").value["counts"] == [16, 37, 30, 16, 14, 57, 67, 29, 6] + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" +``` + +`get_output()` returns a value that compares equal to the underlying one, so +assert on it directly; use `.value` when you need to index into it, and +`.status` (`"ok"` / `"error"` / `"silent"`) or `.error` to assert the +non-value outcomes. Traditional renderers are readable too, so +`@render.data_frame` / `@render_plotly` outputs mounted through `ShinyOutput` +can be checked at the wire level. + +Pass an **absolute `Path`**: a relative one resolves against the test file's +directory, and the app is a directory up from `tests/`. + +Four things to know, all of them shinyreact-specific: + +- **An untyped input id needs no `:shinyreact.default` suffix.** The hook + appends it on the wire, but both Python handlers are no-ops, so + `set_inputs(bins=9)` is equivalent. +- **A typed one does.** `set_inputs(**{"when:shiny.datetime": 1756382400})` is + what makes the handler run and `input.when()` a `datetime`; without the + suffix you are testing a different app than the one the client drives. +- **An event input needs two calls.** `useShinyInput` registers its default at + mount and sends the event after, so an output behind + `@reactive.event(..., ignore_init=True)` only fires on the *second* + `set_inputs` for that id. +- **An unset input means `status == "silent"`, not a `None` value.** + `input.x()` raises a silent exception while unset, so a `if x is None:` + branch in your server is unreachable from a real client — assert the status + instead. (A *later* `req()` failure is not visible in memory at all: + py-shiny#2492.) + +Module ids can be read as the session sees them (`ts.get_output("counter-n")`) +or through a scope, which strips the namespace on the way in and out: + +```python +with ts.make_scope("counter") as counter: + counter.set_inputs(n=7) + assert counter.get_output("label") == "n=7" +``` ## The jsdom layer — mount the client the app ships diff --git a/.claude/skills/shinyreact-convert-app/SKILL.md b/.claude/skills/shinyreact-convert-app/SKILL.md index ed308117..c8184922 100644 --- a/.claude/skills/shinyreact-convert-app/SKILL.md +++ b/.claude/skills/shinyreact-convert-app/SKILL.md @@ -149,12 +149,16 @@ rendered), then the rest of the outputs, then layout and polish. ## Phase 5 — verify -Three layers, cheapest first: factor pure logic out of the app file so it is -importable and test it directly; test the client by evaluating the real -`www/ui.js` against a fake `window.Shiny` in jsdom (not by importing the -component — that tests a copy the app does not ship); and reserve Playwright -for layout and real bindings, which the other two structurally cannot see. The -`shinyreact-build-app` skill's `references/testing.md` has the traps. +Four layers, cheapest first: factor pure logic out of the app file so it is +importable and test it directly; drive the ported server with no browser — +`[r]` `shiny::testServer()`, `[py]` `shiny.testserver.test_server()` — which +for a `ui.tsx` app covers most of it, since the server is only reactive +computation; test the client by evaluating the real `www/ui.js` against a fake +`window.Shiny` in jsdom (not by importing the component — that tests a copy the +app does not ship); and reserve Playwright for layout and real bindings, which +the others structurally cannot see. The `shinyreact-build-app` skill's +[`references/testing.md`](../shinyreact-build-app/references/testing.md) has +the details and the traps. A port has one advantage a new app does not: **the original still runs.** Where the logic is a pure transform, capture its output from the original app and diff --git a/examples/01-hello/FEATURES.md b/examples/01-hello/FEATURES.md index a634ab18..14cd7ef8 100644 --- a/examples/01-hello/FEATURES.md +++ b/examples/01-hello/FEATURES.md @@ -20,7 +20,8 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server -- output `dist_data` → `{breaks: number[], counts: number[]}` +- output `dist_data` → `{breaks: number[], counts: number[]}` `(test)` + - the two keys, in that order, and nothing else `(test)` - equal-width bins over `[min, max]`: `breaks` has `bins + 1` entries, `counts` has `bins` `(test)` - binning matches R's `hist()`: half-open `(lo, hi]` with the first bin @@ -35,16 +36,20 @@ a unit test; `(verify)` marks a claim not yet checked against the code. - `bins = 9` → `counts == [16, 37, 30, 16, 14, 57, 67, 29, 6]`, identical in R and Python `(test)` - `[r]` vectors are wrapped in `I()` so a one-bin result serializes as a JSON - array, not a scalar -- output `dist_caption` → `"272 eruptions in N bins"` + array, not a scalar — the value the output delivers carries the `AsIs` + class `(test)` +- output `dist_caption` → `"272 eruptions in N bins"` `(test)` - singular `"bin"` when `N == 1` `(test)` - the count is the dataset length, not the bin count +- `[py]` `app.py` (Express) and `app-core.py` (Core) produce identical values + for both outputs — the same `tests/test_outputs.py` runs against each + `(test)` - before the client's first `bins` message - `[py]` `input.bins()` raises a silent exception, so neither output produces - a value + a value — not an error, and not a `None` value `(test)` - `[r]` `input$bins` is `NULL` and both outputs return `NULL` explicitly — `req()` is deliberately not used, because its silent error still reaches - the client console + the client console `(test)` - the server never renders an image: no plotting library, no `plotOutput` placeholder diff --git a/examples/01-hello/tests/test_outputs.py b/examples/01-hello/tests/test_outputs.py new file mode 100644 index 00000000..1d1db9b3 --- /dev/null +++ b/examples/01-hello/tests/test_outputs.py @@ -0,0 +1,62 @@ +"""Pins this example's server outputs as the client sees them. + +`test_faithful.py` next to this file tests the binner as a pure function; +this file drives the *app* — `shiny.testserver.test_server()` runs the server +against a mock connection, so `dist_data` and `dist_caption` can be asserted +exactly as they arrive at `useShinyOutputValue()`. No browser, no subprocess. + +Both Python servers are covered, because `app.py` (Express) and `app-core.py` +(Core) are claimed to be interchangeable over one `www/` client. + +Run it from the app directory, the way a user of the app would:: + + pytest + +An input id needs no `:shinyreact.default` suffix here: the React hook appends +it on the wire, but Python's handler is a no-op, so `set_inputs(bins=9)` +reaches the server identically. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from shiny.testserver import test_server + +EXAMPLE = Path(__file__).resolve().parents[1] +APPS = [EXAMPLE / "app.py", EXAMPLE / "app-core.py"] + +# Mirrored in test_faithful.py, tests/test-histogram.R and tests/ui.test.ts. +COUNTS_9 = [16, 37, 30, 16, 14, 57, 67, 29, 6] + + +@pytest.mark.parametrize("app", APPS, ids=lambda p: p.name) +def test_dist_data_wire_shape(app: Path) -> None: + with test_server(app) as ts: + ts.set_inputs(bins=9) + data = ts.get_output("dist_data").value + assert list(data) == ["breaks", "counts"] + assert data["counts"] == COUNTS_9 + assert data["breaks"][0] == 43.0 + assert data["breaks"][-1] == pytest.approx(96.0) + + +@pytest.mark.parametrize("app", APPS, ids=lambda p: p.name) +def test_dist_caption_pluralizes(app: Path) -> None: + with test_server(app) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" + + ts.set_inputs(bins=1) + assert ts.get_output("dist_caption") == "272 eruptions in 1 bin" + assert ts.get_output("dist_data").value["counts"] == [272] + + +@pytest.mark.parametrize("app", APPS, ids=lambda p: p.name) +def test_neither_output_renders_before_the_first_bins_message(app: Path) -> None: + # `input.bins()` raises a silent exception while unset, so both outputs + # have no value at all — not an error, and not a `None` value. + with test_server(app) as ts: + assert ts.get_output("dist_data").status == "silent" + assert ts.get_output("dist_caption").status == "silent" diff --git a/examples/01-hello/tests/testthat/test-outputs.R b/examples/01-hello/tests/testthat/test-outputs.R new file mode 100644 index 00000000..817ea0b6 --- /dev/null +++ b/examples/01-hello/tests/testthat/test-outputs.R @@ -0,0 +1,66 @@ +# Pins app.R's outputs as the client sees them, using shiny::testServer() -- +# inputs in, output values out, with no browser and no client. +# +# test-histogram.R next door pins the *math* by reimplementing app.R's two +# lines, because that logic lives inside server(). This file drives server() +# itself, so it pins what the app actually answers with. The Python +# counterpart is test_outputs.py beside them, which does the same thing with +# shiny.testserver.test_server(). +# +# Run it from the app directory, the way a user of the app would: +# +# Rscript -e 'shiny::runTests()' +# Rscript -e 'testthat::test_dir("tests/testthat")' +# +# The shinyreact package's own suite also runs it, via +# pkg-r/tests/testthat/test-examples.R. +# +# app_dir is "../../" from tests/testthat/, the same default AppDriver uses. +app_dir <- testthat::test_path("..", "..") + +test_that("dist_data is the {breaks, counts} the client draws", { + shiny::testServer(app_dir, { + session$setInputs(bins = 9) + + expect_named(output$dist_data, c("breaks", "counts")) + # Mirrors COUNTS_9 in test_faithful.py and test-histogram.R. + expect_equal( + unclass(output$dist_data$counts), + c(16L, 37L, 30L, 16L, 14L, 57L, 67L, 29L, 6L) + ) + expect_equal(unclass(output$dist_data$breaks)[[1]], 43) + expect_equal(unclass(output$dist_data$breaks)[[10]], 96) + }) +}) + +test_that("dist_caption pluralizes on the bin count", { + shiny::testServer(app_dir, { + session$setInputs(bins = 9) + expect_equal(output$dist_caption, "272 eruptions in 9 bins") + + session$setInputs(bins = 1) + expect_equal(output$dist_caption, "272 eruptions in 1 bin") + expect_equal(unclass(output$dist_data$counts), 272L) + expect_equal(unclass(output$dist_data$breaks), c(43, 96)) + }) +}) + +test_that("the vectors carry I(), so a one-bin result is a JSON array", { + # The class is the behavior: without it toJSON() emits `272` where the + # client expects `[272]`. test-histogram.R pins the serialization; this + # pins that app.R really applies it. + shiny::testServer(app_dir, { + session$setInputs(bins = 1) + expect_s3_class(output$dist_data$counts, "AsIs") + expect_s3_class(output$dist_data$breaks, "AsIs") + }) +}) + +test_that("both outputs are NULL before the client's first bins message", { + # app.R returns NULL explicitly rather than using req(), so there is no + # silent error to reach the client console. + shiny::testServer(app_dir, { + expect_null(output$dist_data) + expect_null(output$dist_caption) + }) +}) diff --git a/examples/02-columns/FEATURES.md b/examples/02-columns/FEATURES.md index 30afd28b..45ab0a61 100644 --- a/examples/02-columns/FEATURES.md +++ b/examples/02-columns/FEATURES.md @@ -17,18 +17,22 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server (`app.py`, Express) -- output `column_data` → the whole `{col: item[]}` dict +- output `column_data` → the whole `{col: item[]}` dict `(test)` + - it renders from the initial contents before any input arrives `(test)` - input `move_item` → `{item, from, to}` - handled by a `@reactive.effect` + `@reactive.event(input.move_item, - ignore_init=True)`, so the initial `null` from mount is ignored + ignore_init=True)`, so the initial `null` from mount is ignored `(test)` - the move is applied to a copy of the dict, then `columns.set(...)` — the reactive value is replaced, not mutated in place - - if `item` is not in `data[from]`, nothing changes (no error) + - if `item` is not in `data[from]`, nothing changes (no error) `(test)` - the item is appended to the end of the destination column, never inserted + `(test)` + - successive moves accumulate `(test)` - `[py]` only — this example has no R server - the server logic lives inside `app.py` next to `set_react_page()`, so it is - not importable and no unit test covers it — logic in a module beside `app.py` - would be testable + not importable — `tests/test_moves.py` drives the app itself with + `shiny.testserver.test_server()` instead, sending the mount-time `null` + first so the move is not swallowed as the init ## Client (`www/ui.js`) diff --git a/examples/02-columns/tests/test_moves.py b/examples/02-columns/tests/test_moves.py new file mode 100644 index 00000000..9b8b79b3 --- /dev/null +++ b/examples/02-columns/tests/test_moves.py @@ -0,0 +1,64 @@ +"""Pins the server's move handling as the client drives it. + +`shiny.testserver.test_server()` runs `app.py`'s server against a mock +connection, so the `move_item` → `column_data` round trip can be asserted +without a browser. Run it from the app directory:: + + pytest + +`move_item` is an **event** input (`@reactive.event(..., ignore_init=True)`), +so a test has to mirror what the client does: `useShinyInput("move_item", +null)` registers the `null` default at mount, and the drop sends the move +after. With a single `set_inputs` the move *is* the init and is ignored. +""" + +from __future__ import annotations + +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + +INITIAL = { + "A": ["Apple", "Apricot"], + "B": ["Banana", "Blueberry"], + "C": ["Cherry", "Cranberry"], +} + + +def test_column_data_starts_at_the_initial_three_columns() -> None: + with test_server(APP) as ts: + assert ts.get_output("column_data") == INITIAL + + +def test_a_move_removes_from_the_source_and_appends_to_the_target() -> None: + with test_server(APP) as ts: + ts.set_inputs(move_item=None) # the hook's default, sent at mount + ts.set_inputs(move_item={"item": "Apple", "from": "A", "to": "C"}) + + assert ts.get_output("column_data") == { + "A": ["Apricot"], + "B": ["Banana", "Blueberry"], + # Appended, not inserted in sorted position. + "C": ["Cherry", "Cranberry", "Apple"], + } + + +def test_a_move_of_an_item_the_source_does_not_hold_is_ignored() -> None: + with test_server(APP) as ts: + ts.set_inputs(move_item=None) + ts.set_inputs(move_item={"item": "Cherry", "from": "A", "to": "B"}) + assert ts.get_output("column_data") == INITIAL + + +def test_moves_accumulate() -> None: + with test_server(APP) as ts: + ts.set_inputs(move_item=None) + ts.set_inputs(move_item={"item": "Apple", "from": "A", "to": "B"}) + ts.set_inputs(move_item={"item": "Apple", "from": "B", "to": "C"}) + + data = ts.get_output("column_data").value + assert data["A"] == ["Apricot"] + assert data["B"] == ["Banana", "Blueberry"] + assert data["C"] == ["Cherry", "Cranberry", "Apple"] diff --git a/examples/05-temperature/FEATURES.md b/examples/05-temperature/FEATURES.md index f73b1972..6203b3bc 100644 --- a/examples/05-temperature/FEATURES.md +++ b/examples/05-temperature/FEATURES.md @@ -9,19 +9,23 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server (`app.py`, Express) -- one output, `display` → `{celsius, fahrenheit, zone}` +- one output, `display` → `{celsius, fahrenheit, zone}` `(test)` - `fahrenheit` is `round(c * 9 / 5 + 32, 1)` — one decimal, so 20 °C → 68.0 -- `zone` thresholds, all inclusive upper bounds + `(test)` + - the client rounds to whole degrees, so the two disagree by design: 37 °C is + `98.6` from the server and `99` on the client `(test)` +- `zone` thresholds, all inclusive upper bounds `(test)` - `c <= 0` → `"Freezing"` - `c <= 15` → `"Cold"` - `c <= 30` → `"Comfortable"` - otherwise → `"Hot"` -- `input.celsius()` is `None` before the client's first message → returns - `None`, so the echo line does not render +- before the client's first message, `input.celsius()` raises a silent + exception, so `display` never renders — the `c is None` guard in `app.py` is + unreachable from a real client `(test)` - `[py]` only — this example has no R server - the logic lives inside `app.py` next to `set_react_page()`, so it is not - importable and no unit test covers it — logic in a module beside `app.py` - would be testable + importable — `tests/test_display.py` drives the app itself with + `shiny.testserver.test_server()` instead ## Client (`www/ui.js`) diff --git a/examples/05-temperature/tests/test_display.py b/examples/05-temperature/tests/test_display.py new file mode 100644 index 00000000..b49b2ae9 --- /dev/null +++ b/examples/05-temperature/tests/test_display.py @@ -0,0 +1,66 @@ +"""Pins the server echo — the half of this example the client does not compute. + +The client converts locally and the server converts again; a threshold or +rounding change on one side without the other is a visible divergence, so both +sides need a test. `ui.test.ts` next to this file covers the client; +`shiny.testserver.test_server()` covers the server, in memory. Run it from the +app directory:: + + pytest + +The client rounds to whole degrees while the server keeps one decimal — the +`98.6` below is the server's answer for the same 37 °C the client shows as +`99°F`, which is exactly the divergence worth pinning. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_display_wire_shape() -> None: + with test_server(APP) as ts: + ts.set_inputs(celsius=20) + assert ts.get_output("display") == { + "celsius": 20, + "fahrenheit": 68.0, + "zone": "Comfortable", + } + + +def test_fahrenheit_keeps_one_decimal() -> None: + with test_server(APP) as ts: + ts.set_inputs(celsius=37) + assert ts.get_output("display").value["fahrenheit"] == 98.6 + + +@pytest.mark.parametrize( + "celsius, zone", + [ + (-40, "Freezing"), + (0, "Freezing"), # inclusive upper bound + (1, "Cold"), + (15, "Cold"), + (16, "Comfortable"), + (30, "Comfortable"), + (31, "Hot"), + (60, "Hot"), + ], +) +def test_zone_thresholds(celsius: int, zone: str) -> None: + with test_server(APP) as ts: + ts.set_inputs(celsius=celsius) + assert ts.get_output("display").value["zone"] == zone + + +def test_no_echo_before_the_clients_first_message() -> None: + # `input.celsius()` raises a silent exception while unset, so `display` + # never renders at all — the `c is None` guard in `app.py` is unreachable + # from a real client. + with test_server(APP) as ts: + assert ts.get_output("display").status == "silent" diff --git a/examples/06-data-frame/FEATURES.md b/examples/06-data-frame/FEATURES.md index e3bd1a59..0069cfab 100644 --- a/examples/06-data-frame/FEATURES.md +++ b/examples/06-data-frame/FEATURES.md @@ -9,13 +9,17 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server (`app.py`, Express) -- output `greeting` (`reactive_output`) → `"Showing N rows"` +- output `greeting` (`reactive_output`) → `"Showing N rows"` `(test)` - no singular special case: `N = 1` reads `"Showing 1 rows"` - output `my_table` (`@render.data_frame`) → a pandas DataFrame with `N` rows - and columns `Name` / `Value` / `Category` - - `Name` is `"Item 1" … "Item N"`, 1-based - - `Value` is `10, 20, … 10N` + and columns `Name` / `Value` / `Category` `(test)` + - `Name` is `"Item 1" … "Item N"`, 1-based `(test)` + - `Value` is `10, 20, … 10N` `(test)` - `Category` is `"A"` on even `i`, `"B"` on odd `i` — so row 1 is `B` + `(test)` + - on the wire it is `{payload: {columns, data, typeHints, …}}` — row-major + `data`, one array per row `(test)` +- `row_count` drives both outputs `(test)` - there is no `ui.output_data_frame("my_table")` anywhere; the element the binding attaches to is created by the client - `set_react_page()` discovers the data-frame renderer's `HTMLDependency` and diff --git a/examples/06-data-frame/tests/test_table.py b/examples/06-data-frame/tests/test_table.py new file mode 100644 index 00000000..ad0f4937 --- /dev/null +++ b/examples/06-data-frame/tests/test_table.py @@ -0,0 +1,42 @@ +"""Pins both outputs, including the traditional `@render.data_frame` one. + +`shiny.testserver.test_server()` reads a `reactive_output` and a traditional +renderer the same way, so the payload `ShinyOutput` hands to +`shiny-data-frame` can be asserted in memory. Run it from the app directory:: + + pytest +""" + +from __future__ import annotations + +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_greeting_counts_the_rows() -> None: + with test_server(APP) as ts: + ts.set_inputs(row_count=3) + assert ts.get_output("greeting") == "Showing 3 rows" + + +def test_data_frame_payload() -> None: + with test_server(APP) as ts: + ts.set_inputs(row_count=3) + payload = ts.get_output("my_table").value["payload"] + + assert payload["columns"] == ["Name", "Value", "Category"] + assert payload["data"] == [ + ["Item 1", 10, "B"], + ["Item 2", 20, "A"], + ["Item 3", 30, "B"], + ] + + +def test_row_count_drives_both_outputs() -> None: + with test_server(APP) as ts: + ts.set_inputs(row_count=5) + assert ts.get_output("greeting") == "Showing 5 rows" + assert len(ts.get_output("my_table").value["payload"]["data"]) == 5 diff --git a/examples/07-plotly/FEATURES.md b/examples/07-plotly/FEATURES.md index ffcdbd59..efb895a2 100644 --- a/examples/07-plotly/FEATURES.md +++ b/examples/07-plotly/FEATURES.md @@ -10,21 +10,31 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server -- output `greeting` (`reactive_output`) → `"Showing N random points"` +- output `greeting` (`reactive_output`) → `"Showing N random points"` `(test)` + - no singular special case: `N = 1` reads `"Showing 1 random points"` + `(test)` - `[r]` returns `NULL` while `input$num_points` is `NULL`; `req()` is deliberately avoided so the silent error does not reach the client console + `(test)` - `[py]` `input.num_points()` raises a silent exception before the first - message + message, so neither output renders `(test)` - output `scatter` — a Plotly figure of `N` standard-normal `(x, y)` points, seeded 42, title `"Random Scatter (N points)"`, margins l 40 / r 20 / t 40 / b 40 - `[py]` `@render_plotly` from shinywidgets, `px.scatter` over `np.random.default_rng(42)` - `[r]` `plotly::renderPlotly` with `set.seed(42)` and `rnorm` + - on the wire it is a `json` string carrying the figure, with the widget's + own dependencies (`plotly-main`, `crosstalk`, …) attached to it — but + **not** `plotly-binding`, which arrives separately `(test)` - the two servers draw *different* points — the RNG streams differ — so this is not a cross-language parity claim - `[r]` uses `req(input$num_points)` here, unlike `greeting` + - `[py]` on the wire it is a widget reference, not a figure: + `{model_id, fill, widget_pkg: "plotly"}` — the client's `ShinyOutput` is + what turns it into a plot `(test)` - there is no `output_widget()` / `plotlyOutput()` placeholder in either server + — the value is produced all the same `(test)` - the render function's binding JS is discovered and delivered automatically - `[py]` `set_react_page()` inlines the dependency into `` - `[r]` `page_react()` cannot inline it (the UI is built before `server()` diff --git a/examples/07-plotly/tests/test_scatter.py b/examples/07-plotly/tests/test_scatter.py new file mode 100644 index 00000000..ffd9340d --- /dev/null +++ b/examples/07-plotly/tests/test_scatter.py @@ -0,0 +1,43 @@ +"""Pins both outputs: the JSON echo and the plotly widget. + +`shiny.testserver.test_server()` reads a `reactive_output` and a +`@render_plotly` widget the same way, so this example's central claim — a +traditional widget renderer needs no `*Output()` placeholder to produce its +value — is checkable without a browser. Run it from the app directory:: + + pytest +""" + +from __future__ import annotations + +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_greeting_counts_the_points() -> None: + with test_server(APP) as ts: + ts.set_inputs(num_points=50) + assert ts.get_output("greeting") == "Showing 50 random points" + + ts.set_inputs(num_points=1) + assert ts.get_output("greeting") == "Showing 1 random points" + + +def test_scatter_renders_a_widget_with_no_placeholder() -> None: + with test_server(APP) as ts: + ts.set_inputs(num_points=50) + widget = ts.get_output("scatter").value + + # What shinywidgets puts on the wire: a widget reference, not a figure. + # The client's `ShinyOutput` is what turns it into a plot. + assert widget["widget_pkg"] == "plotly" + assert widget["model_id"] + + +def test_neither_output_renders_before_the_first_message() -> None: + with test_server(APP) as ts: + assert ts.get_output("greeting").status == "silent" + assert ts.get_output("scatter").status == "silent" diff --git a/examples/07-plotly/tests/testthat.R b/examples/07-plotly/tests/testthat.R new file mode 100644 index 00000000..ee81e5d0 --- /dev/null +++ b/examples/07-plotly/tests/testthat.R @@ -0,0 +1,10 @@ +# Runner for this example app's own tests, in the layout `shiny::runTests()` +# and `shinytest2::test_app()` expect. From the app directory: +# +# Rscript -e 'shiny::runTests()' +# +# The Python and JS tests for this app live beside testthat/ and are run with +# pytest and vitest; see examples/README.md. +library(testthat) + +test_dir("testthat") diff --git a/examples/07-plotly/tests/testthat/test-outputs.R b/examples/07-plotly/tests/testthat/test-outputs.R new file mode 100644 index 00000000..a1f198d2 --- /dev/null +++ b/examples/07-plotly/tests/testthat/test-outputs.R @@ -0,0 +1,54 @@ +# Pins app.R's outputs with shiny::testServer() -- no browser, no client. +# +# The Python counterpart is test_scatter.py beside it, which does the same +# thing with shiny.testserver.test_server(). Both exist to pin this example's +# central claim from the server side: a widget renderer produces its value +# with no *Output() placeholder anywhere. +# +# Run it from the app directory, the way a user of the app would: +# +# Rscript -e 'shiny::runTests()' +# +# The shinyreact package's own suite also runs it, via +# pkg-r/tests/testthat/test-examples.R. +app_dir <- testthat::test_path("..", "..") + +test_that("greeting counts the points", { + shiny::testServer(app_dir, { + session$setInputs(num_points = 50) + expect_equal(output$greeting, "Showing 50 random points") + + # No singular special case. + session$setInputs(num_points = 1) + expect_equal(output$greeting, "Showing 1 random points") + }) +}) + +test_that("greeting is NULL before the client's first message", { + # app.R returns NULL explicitly rather than using req(), so no silent error + # reaches the client console -- unlike `scatter`, which does use req(). + shiny::testServer(app_dir, { + expect_null(output$greeting) + }) +}) + +test_that("scatter renders a plotly widget with no plotlyOutput() anywhere", { + skip_if_not_installed("plotly") + + shiny::testServer(app_dir, { + session$setInputs(num_points = 50) + + # renderPlotly() hands the client a JSON string carrying the figure, with + # the widget's HTML dependencies attached as an attribute -- the React + # side's ShinyOutput is what mounts it, and page_react() pushes those deps + # after the flush. + expect_s3_class(output$scatter, "json") + expect_match(output$scatter, '"type":"scatter"', fixed = TRUE) + dep_names <- vapply(attr(output$scatter, "deps"), `[[`, character(1), "name") + expect_true("plotly-main" %in% dep_names) + # The *binding* JS is not in here: page_react() discovers it from the + # render function and pushes it as a shinyreact-deps message instead, + # which is what pkg-r/tests/testthat/test-dep-discovery.R covers. + expect_false("plotly-binding" %in% dep_names) + }) +}) diff --git a/examples/08-input-handler/FEATURES.md b/examples/08-input-handler/FEATURES.md index 0053c3a5..ccab0e53 100644 --- a/examples/08-input-handler/FEATURES.md +++ b/examples/08-input-handler/FEATURES.md @@ -12,7 +12,9 @@ a unit test; `(verify)` marks a claim not yet checked against the code. - the hook's `type` appends `:shiny.datetime` to the wire id, so the value arrives as `when:shiny.datetime` and Shiny's built-in handler coerces it - before `input.when()` resolves + before `input.when()` resolves `(test)` + - the suffix is what buys the coercion: the same number sent as plain `when` + stays an `int` `(test)` - opting into a `type` bypasses shinyreact's own `shinyreact.default` handler — the value is handled by Shiny's registry, not by shinyreact - the id/type pairing is a per-id contract: a second mount of `"when"` without @@ -21,9 +23,10 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server (`app.py`, Express) - one output, `when_info` → `""`, e.g. - `"datetime → datetime.datetime(2026, 8, 28, 12, 0, tzinfo=...)"` -- `input.when()` is `None` before the first client message → returns `"—"` - (em dash) + `"datetime → datetime.datetime(2026, 8, 28, 12, 0, tzinfo=...)"` `(test)` +- before the first client message, `input.when()` raises a silent exception, so + `when_info` never renders — the `"—"` (em dash) branch in `app.py` is + unreachable from a real client `(test)` - the example asserts nothing about the timezone the handler attaches - `[py]` only — this example has no R server; R's handler registry has no `shiny.datetime` equivalent, so the client is not portable as written diff --git a/examples/08-input-handler/tests/test_when.py b/examples/08-input-handler/tests/test_when.py new file mode 100644 index 00000000..fafe63aa --- /dev/null +++ b/examples/08-input-handler/tests/test_when.py @@ -0,0 +1,46 @@ +"""Pins the input handler's coercion at the server, in memory. + +This example exists to show that `useShinyInput(..., {type: "shiny.datetime"})` +makes the server read a `datetime`. `shiny.testserver.test_server()` can assert +exactly that, because `set_inputs` takes the **wire** id — so the `:type` +suffix the hook appends is part of the test, and the handler really runs. Run +it from the app directory:: + + pytest +""" + +from __future__ import annotations + +import datetime +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_the_handler_turns_unix_seconds_into_a_datetime() -> None: + with test_server(APP) as ts: + # What the client sends: unix *seconds*, under the suffixed wire id. + ts.set_inputs(**{"when:shiny.datetime": 1756382400}) + echoed = ts.get_output("when_info").value + + assert echoed.startswith("datetime → datetime.datetime(") + # The example asserts nothing about the timezone, only the type. + assert datetime.datetime.fromtimestamp(1756382400).year == 2025 + + +def test_an_unsuffixed_value_is_not_coerced() -> None: + # The suffix is what buys the coercion: without it the number arrives as a + # number. This is the failure mode the per-id type contract exists to + # prevent, and the reason to write the suffix out in the test above. + with test_server(APP) as ts: + ts.set_inputs(when=1756382400) + assert ts.get_output("when_info") == "int → 1756382400" + + +def test_nothing_renders_before_the_clients_first_message() -> None: + # `input.when()` raises a silent exception while unset, so the `None` + # branch in `app.py` (the em dash) is unreachable from a real client. + with test_server(APP) as ts: + assert ts.get_output("when_info").status == "silent" diff --git a/examples/10-bookmarking/FEATURES.md b/examples/10-bookmarking/FEATURES.md index 0b123520..125b78d0 100644 --- a/examples/10-bookmarking/FEATURES.md +++ b/examples/10-bookmarking/FEATURES.md @@ -15,12 +15,16 @@ a unit test; `(verify)` marks a claim not yet checked against the code. further wiring — a UI object built once could not carry a per-request restore payload - output `greeting` (`reactive_output`) → `text='' num= - checked=` - - `checked` is the string `"yes"` / `"no"`, not a boolean - - `txt` is `repr`'d, so it is quoted + checked=` `(test)` + - `checked` is the string `"yes"` / `"no"`, not a boolean `(test)` + - `txt` is `repr`'d, so it is quoted `(test)` + - it renders from the hook defaults (`""` / `0` / `false`), since the client + does not gate on `useShinyInitialized()` `(test)` - `@reactive.effect` + `@reactive.event(input.bookmark_clicks, ignore_init=True)` → `await session.bookmark()`, which rewrites the browser URL + - the mount-time `0` does not bookmark; the URL rewrite itself needs a + browser, so in-memory tests only assert the effect runs cleanly `(test)` - `[py]` only — this example has no R server ## Restore path diff --git a/examples/10-bookmarking/tests/test_greeting.py b/examples/10-bookmarking/tests/test_greeting.py new file mode 100644 index 00000000..2fcd31e8 --- /dev/null +++ b/examples/10-bookmarking/tests/test_greeting.py @@ -0,0 +1,51 @@ +"""Pins the echo output and the bookmark effect's inputs. + +`shiny.testserver.test_server()` loads this app through `ReactApp`, so the +server half of the example runs in memory. The *restore* half does not: it is a +property of the rendered page and the browser URL, and stays pinned by +`pkg-py/tests/test_bookmark_restore.py` and its Playwright counterpart. Run it +from the app directory:: + + pytest +""" + +from __future__ import annotations + +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_greeting_wire_shape() -> None: + with test_server(APP) as ts: + ts.set_inputs(txt="hi", num=5, chk=True) + # `checked` is the string "yes"/"no", and `txt` is repr'd, so quoted. + assert ts.get_output("greeting") == "text='hi' num=5 checked=yes" + + ts.set_inputs(chk=False) + assert ts.get_output("greeting") == "text='hi' num=5 checked=no" + + +def test_greeting_renders_from_the_hook_defaults() -> None: + # The client renders immediately rather than gating on + # `useShinyInitialized()`, so the first values the server sees are the + # hook defaults (or the restored ones). + with test_server(APP) as ts: + ts.set_inputs(txt="", num=0, chk=False) + assert ts.get_output("greeting") == "text='' num=0 checked=no" + + +def test_a_bookmark_click_is_an_event_input() -> None: + # `bookmark_clicks` is write-only and `priority: "event"`, with the count + # incremented per click; `ignore_init=True` means the mount-time 0 does + # not bookmark. The URL rewrite itself needs a browser — see the + # Playwright suite — so all this asserts is that the effect runs cleanly. + with test_server(APP) as ts: + ts.set_inputs(txt="hi", num=5, chk=True) + ts.set_inputs(bookmark_clicks=0) + ts.set_inputs(bookmark_clicks=1) + + assert ts.to_values().is_ok + assert ts.get_output("greeting") == "text='hi' num=5 checked=yes" diff --git a/examples/README.md b/examples/README.md index aa888896..937fd571 100644 --- a/examples/README.md +++ b/examples/README.md @@ -110,6 +110,26 @@ Rscript -e 'shiny::runTests()' # the app's R tests (also shinytest2::t npx vitest run --root .. 01-hello # the app's UI tests ``` +Most of the Python tests drive the app itself, with +[`shiny.testserver.test_server()`](https://shiny.posit.co/py/api/testing/): +inputs in, output values out, in memory. That is a good fit for these apps +because a `ui.tsx` server contains only reactive computation, so the JSON a +test asserts is exactly what `useShinyOutputValue()` receives: + +```python +with test_server(APP) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" +``` + +Pass an absolute `Path` (`Path(__file__).resolve().parents[1] / "app.py"`), and +mind two shinyreact-specific details: an **untyped** input id needs no +`:shinyreact.default` suffix (the hook adds it on the wire, but Python's +handler is a no-op), while a **typed** one does, because the suffix is what +runs the handler; and an input read through `@reactive.event(..., +ignore_init=True)` needs two `set_inputs` calls, mirroring the client's +register-at-mount then send-the-event. + The UI tests need a JS toolchain, which the no-build examples deliberately do not carry. `examples/package.json` provides one for the whole examples tree, so `npm install` there once covers every example: diff --git a/pkg-py/README.md b/pkg-py/README.md index 650ae5e6..9b6b6eea 100644 --- a/pkg-py/README.md +++ b/pkg-py/README.md @@ -78,6 +78,6 @@ shiny run shinyreact/examples/01-hello/app.py - [Get started](https://posit-dev.github.io/shinyreact/) walks through the `ui.tsx` pattern: inputs, outputs, messages, and embedding traditional Shiny renderers. - [TSX files and JavaScript build tools](https://posit-dev.github.io/shinyreact/articles/tsx-and-build-tools.html) explains `.tsx`, JSX, TypeScript, and what `npm run build` does. - [Client hooks](https://posit-dev.github.io/shinyreact/articles/hooks.html) lists everything at `window.shinyreact`. -- [Testing wire payloads](https://posit-dev.github.io/shinyreact/articles/testing.html) covers `WireTap` for Playwright tests. +- [Testing](https://posit-dev.github.io/shinyreact/articles/testing.html) covers `shiny.testserver.test_server()` for driving a server with no browser, and `WireTap` for wire payloads in Playwright tests. - [Agent Skills](https://posit-dev.github.io/shinyreact/articles/agent-skills.html) explains the skills that ship with the package for coding agents. - The [examples catalog](https://github.com/posit-dev/shinyreact/blob/main/examples/README.md) lists runnable apps from no-build to Vite + HMR. diff --git a/pkg-py/docs/_quarto.yml b/pkg-py/docs/_quarto.yml index 807c41ba..1238896b 100644 --- a/pkg-py/docs/_quarto.yml +++ b/pkg-py/docs/_quarto.yml @@ -97,7 +97,8 @@ quartodoc: contents: - send_message - title: Testing - desc: Record the wire payloads a Playwright test observes. + desc: Record the wire payloads a Playwright test observes. For testing a + server with no browser, see `shiny.testserver.test_server()`. contents: - playwright.WireTap diff --git a/pkg-py/docs/articles/testing.qmd b/pkg-py/docs/articles/testing.qmd index a1ef63ad..fc7527cb 100644 --- a/pkg-py/docs/articles/testing.qmd +++ b/pkg-py/docs/articles/testing.qmd @@ -1,13 +1,85 @@ --- -title: "Testing wire payloads" +title: "Testing" --- In a shinyreact app the contract between server and client is the JSON that crosses the Shiny websocket: the values `reactive_output` delivers, the payloads `send_message()` pushes, and the values `useShinyInput()` sends back. -A wire tap records those payloads in a browser test so you can assert on them directly, without inspecting the rendered DOM. +Both halves of this page assert that JSON — the first without a browser at all, the second inside a real one. + +## Testing the server without a browser + +A `ui.tsx` server contains only reactive computation, so "this input produces that output value" *is* the server. +Shiny can run one against a mock connection — no browser, no subprocess — and hand you the value the client would have received. + +::: {.panel-tabset} + +### Python + +`shiny.testserver.test_server()` loads the app file (Express or Core, `shiny.App` or `shinyreact.ReactApp`) and runs its server: + +```python +from pathlib import Path + +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_dist_outputs(): + with test_server(APP) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" + assert ts.get_output("dist_data").value["counts"][0] == 16 +``` + +`get_output()` compares equal to the underlying value, so assert on it directly; reach for `.value` to index into it, and `.status` (`"ok"`, `"error"`, `"silent"`) or `.error` for the non-value outcomes. +Traditional renderers embedded with [`ShinyOutput`](hooks.qmd) are readable the same way, so a `@render.data_frame` payload can be checked here too. + +Three details are specific to shinyreact: + +- An **untyped** input id needs no `:shinyreact.default` suffix — the hook appends it on the wire, but Python's handler is a no-op, so `set_inputs(bins=9)` is equivalent. +- A **typed** one does: `set_inputs(**{"when:shiny.datetime": 1756382400})` is what runs the handler and makes `input.when()` a `datetime`. +- An **unset** input means `status == "silent"`, not a `None` value, because `input.x()` raises a silent exception while unset. + +An input read through `@reactive.event(..., ignore_init=True)` needs two `set_inputs` calls, mirroring the client: `useShinyInput` registers its default at mount and sends the event after. + +`test_server()` is newer than shiny 1.7.0. + +### R + +`reactive_output()` is an ordinary render function, so `shiny::testServer()` drives it directly and `output$id` is the JSON value itself — no spec wrapper, no coercion: + +```r +test_that("dist_data bins the waiting column", { + shiny::testServer(app_dir, { + session$setInputs(bins = 9) + expect_named(output$dist_data, c("breaks", "counts")) + expect_equal( + unclass(output$dist_data$counts), + c(16L, 37L, 30L, 16L, 14L, 57L, 67L, 29L, 6L) + ) + expect_equal(output$dist_caption, "272 eruptions in 9 bins") + }) +}) +``` + +`unclass()` is there because the app wraps its vectors in `I()` so a one-bin result serializes as `[272]` rather than `272`; the `AsIs` class rides along on the value the test sees. + +Pass a directory containing the app, or the `server` function itself. +Module ids are namespaced as the session sees them, so a scoped output is read as `` output$`counter-label` ``, or reachable through `session$makeScope("counter")`; a module server can also be driven on its own with `testServer(card_server, args = list(id = "left"), { ... })`. + +`testServer()` **raises** rather than reporting a status: reading an output whose render function failed `req()` throws a `shiny.silent.error`, so `expect_error(output$answer, class = "shiny.silent.error")` is how you assert an output produced nothing. +Python's `test_server()` reports that through `.status` instead. + +::: + +## Testing wire payloads + +A wire tap records the payloads crossing the websocket in a browser test, so you can assert on them directly instead of inspecting the rendered DOM. +Reserve it for what the section above structurally cannot see: the values the *client* chooses to send, real bindings, and real rendering. ::: {.panel-tabset} -## Python +### Python `shinyreact.playwright.WireTap` needs the `playwright` package. Construct it before `page.goto()` so it sees every frame: @@ -23,7 +95,7 @@ def test_dist_data(page, app): tap.expect_output_value("dist_data", lambda d: d["breaks"][0] == 43.0) ``` -## R +### R `wire_tap()` needs the shinytest2 package. Start the `AppDriver` with `shiny.trace = TRUE` so every websocket frame is recorded in the app's logs: @@ -44,7 +116,7 @@ test_that("dist_data bins the waiting column", { ::: -## Matchers +### Matchers Each `expect_*` method takes a matcher and retries until it matches or a timeout (10 seconds by default) elapses: @@ -61,7 +133,7 @@ There is one `expect_*` per channel: Successive expectations on one channel assert an ordered subsequence: each scans from just past the previous match, so a value that arrives between two checks is never missed. -## Full histories +### Full histories `all_output_values(id)`, `all_messages(id)`, and `all_input_values(id)` return everything that crossed a channel, in order. Input ids match the bare id or any `id:type` wire id, so use the id you wrote in `useShinyInput()`. diff --git a/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/SKILL.md b/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/SKILL.md index 0a922ace..0a225bee 100644 --- a/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/SKILL.md +++ b/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/SKILL.md @@ -383,8 +383,8 @@ a list. Use `type: "shinyreact.asis"` for the parsed value untouched. Do not stop at "the code is written". Three steps, in order: 1. **Factor pure logic out of the app file** — binning, formatting, - conversions go in a module beside the app. Logic inside `app.py` / - `app.R` next to the page call cannot be reached by a test at all. + conversions go in a module beside the app, where a test can import them + without starting a session at all. 2. **Write down what the app does, in plain English, before the tests.** An agent that writes the client and then writes the client's tests is agreeing with itself — both encode the same misunderstanding. A @@ -394,16 +394,19 @@ Do not stop at "the code is written". Three steps, in order: directory** — `pytest`, or `[r]` `shiny::runTests()`, which needs the `tests/testthat.R` + `tests/testthat/` layout. -`[r]` `shiny::testServer()` drives the reactive graph with no browser: -`session$setInputs(bins = 9)` then assert on `output$dist_data`, which is the -JSON value the client would have received. **`[py]` has no equivalent**, so -factoring logic into an importable module matters more there. +Both languages drive the reactive graph with no browser, which for a `ui.tsx` +app is most of the server: `[r]` `shiny::testServer()` (`session$setInputs(bins += 9)`, then assert on `output$dist_data`) and `[py]` +`shiny.testserver.test_server()` (`ts.set_inputs(bins=9)`, then +`ts.get_output("dist_data")`). Either way the value you assert is the JSON the +client would have received. [`references/testing.md`](references/testing.md) has the four layers, the test -layout for each language, `testServer()` for plain and module servers, how to -mount the real `www/ui.js` against a fake Shiny, and the traps that cost time -(React ignores raw `change` events; debounce coalesces within a tick even at -`debounceMs: 0`). +layout for each language, `testServer()` / `test_server()` for plain and module +servers — including the input ids that need a `:type` suffix and the event +inputs that need two `set_inputs` calls — how to mount the real `www/ui.js` +against a fake Shiny, and the traps that cost time (React ignores raw `change` +events; debounce coalesces within a tick even at `debounceMs: 0`). ## Debugging diff --git a/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/references/testing.md b/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/references/testing.md index 8fa6e978..245ede5e 100644 --- a/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/references/testing.md +++ b/pkg-py/src/shinyreact/.agents/skills/shinyreact-build-app/references/testing.md @@ -3,8 +3,11 @@ Do not stop at "the code is written". At minimum: 1. **Factor pure logic out of the app file.** Binning, formatting, conversions - go in a module beside the app so a test can import them; logic sitting - inside `app.py` / `app.R` next to the page call cannot be reached at all. + go in a module beside the app so a test can import them directly, with no + session at all. Logic left inside `app.py` / `app.R` next to the page call + is still reachable — `testServer()` / `test_server()` below drive the app + itself — but only through an input, which is a slower and blunter tool than + calling a function. 2. **Write down what the app does, in plain English, before the tests** — a behavior file beside the app, one atomically checkable claim per line ("the caption reads `N eruptions in M bins`, singular `bin` at M=1"). An @@ -16,7 +19,7 @@ Do not stop at "the code is written". At minimum: | Layer | Proves | Cost | |---|---|---| | pure functions in their own module | binning, formatting, conversions | trivial — always do this | -| `[r]` `shiny::testServer()` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | +| `shiny::testServer()` `[r]` / `shiny.testserver.test_server()` `[py]` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | | the client mounted in jsdom against a fake Shiny | rendering, input wiring, wire ids, status handling | low, and it exercises the file the app ships | | Playwright | layout, real Shiny, real bindings | high; reserve for what the others cannot see | @@ -31,7 +34,8 @@ myapp/ faithful.py the factored logic www/ui.js tests/ - test_faithful.py [py] pytest + test_faithful.py [py] pytest — the factored logic, called directly + test_outputs.py [py] pytest — the app, via test_server() testthat.R [r] runner: library(testthat); test_dir("testthat") testthat/ test-histogram.R [r] @@ -58,11 +62,17 @@ sys.path.insert(0, str(EXAMPLE)) from faithful import histogram, waiting # noqa: E402 ``` -## `[r]` Testing the server with `testServer()` +## Testing the server: `testServer()` `[r]`, `test_server()` `[py]` -`reactive_output` is an ordinary Shiny render function, so `shiny::testServer()` -drives it with no browser and no client: set inputs, read outputs, and get the -JSON value the client would have received. +**This is the highest-value layer for a `ui.tsx` app.** The server contains +only reactive computation, so "input X produces output Y" *is* the server, and +both languages can assert it with no browser and no client: set inputs, read +outputs, and get the JSON value the client would have received. + +### `[r]` `shiny::testServer()` + +`reactive_output` is an ordinary Shiny render function, so `testServer()` +drives it directly. ```r test_that("the histogram recomputes when bins changes", { @@ -90,10 +100,62 @@ which is most of what a shinyreact server does. Module servers work the same way: `testServer(card_server, args = list(id = "left"), { ... })`. -**`[py]` has no `testServer()` equivalent.** Cover the Python server by -factoring its logic into a module and testing that with pytest, and let the -jsdom and Playwright layers cover the wiring. This is a real gap, not an -oversight in your app. +### `[py]` `shiny.testserver.test_server()` + +The Python counterpart (py-shiny#2470, so newer than shiny 1.7.0). It loads the +app file — Express or Core, `shiny.App` or `shinyreact.ReactApp` — and runs its +server against a mock connection: + +```python +from pathlib import Path +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_the_histogram_recomputes_when_bins_changes(): + with test_server(APP) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_data").value["counts"] == [16, 37, 30, 16, 14, 57, 67, 29, 6] + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" +``` + +`get_output()` returns a value that compares equal to the underlying one, so +assert on it directly; use `.value` when you need to index into it, and +`.status` (`"ok"` / `"error"` / `"silent"`) or `.error` to assert the +non-value outcomes. Traditional renderers are readable too, so +`@render.data_frame` / `@render_plotly` outputs mounted through `ShinyOutput` +can be checked at the wire level. + +Pass an **absolute `Path`**: a relative one resolves against the test file's +directory, and the app is a directory up from `tests/`. + +Four things to know, all of them shinyreact-specific: + +- **An untyped input id needs no `:shinyreact.default` suffix.** The hook + appends it on the wire, but both Python handlers are no-ops, so + `set_inputs(bins=9)` is equivalent. +- **A typed one does.** `set_inputs(**{"when:shiny.datetime": 1756382400})` is + what makes the handler run and `input.when()` a `datetime`; without the + suffix you are testing a different app than the one the client drives. +- **An event input needs two calls.** `useShinyInput` registers its default at + mount and sends the event after, so an output behind + `@reactive.event(..., ignore_init=True)` only fires on the *second* + `set_inputs` for that id. +- **An unset input means `status == "silent"`, not a `None` value.** + `input.x()` raises a silent exception while unset, so a `if x is None:` + branch in your server is unreachable from a real client — assert the status + instead. (A *later* `req()` failure is not visible in memory at all: + py-shiny#2492.) + +Module ids can be read as the session sees them (`ts.get_output("counter-n")`) +or through a scope, which strips the namespace on the way in and out: + +```python +with ts.make_scope("counter") as counter: + counter.set_inputs(n=7) + assert counter.get_output("label") == "n=7" +``` ## The jsdom layer — mount the client the app ships diff --git a/pkg-py/src/shinyreact/.agents/skills/shinyreact-convert-app/SKILL.md b/pkg-py/src/shinyreact/.agents/skills/shinyreact-convert-app/SKILL.md index ed308117..c8184922 100644 --- a/pkg-py/src/shinyreact/.agents/skills/shinyreact-convert-app/SKILL.md +++ b/pkg-py/src/shinyreact/.agents/skills/shinyreact-convert-app/SKILL.md @@ -149,12 +149,16 @@ rendered), then the rest of the outputs, then layout and polish. ## Phase 5 — verify -Three layers, cheapest first: factor pure logic out of the app file so it is -importable and test it directly; test the client by evaluating the real -`www/ui.js` against a fake `window.Shiny` in jsdom (not by importing the -component — that tests a copy the app does not ship); and reserve Playwright -for layout and real bindings, which the other two structurally cannot see. The -`shinyreact-build-app` skill's `references/testing.md` has the traps. +Four layers, cheapest first: factor pure logic out of the app file so it is +importable and test it directly; drive the ported server with no browser — +`[r]` `shiny::testServer()`, `[py]` `shiny.testserver.test_server()` — which +for a `ui.tsx` app covers most of it, since the server is only reactive +computation; test the client by evaluating the real `www/ui.js` against a fake +`window.Shiny` in jsdom (not by importing the component — that tests a copy the +app does not ship); and reserve Playwright for layout and real bindings, which +the others structurally cannot see. The `shinyreact-build-app` skill's +[`references/testing.md`](../shinyreact-build-app/references/testing.md) has +the details and the traps. A port has one advantage a new app does not: **the original still runs.** Where the logic is a pure transform, capture its output from the original app and diff --git a/pkg-py/tests/test_in_memory_server.py b/pkg-py/tests/test_in_memory_server.py new file mode 100644 index 00000000..595efe28 --- /dev/null +++ b/pkg-py/tests/test_in_memory_server.py @@ -0,0 +1,92 @@ +"""Server-side behavior of the ui.tsx pattern, driven in memory. + +`shiny.testserver.test_server()` (py-shiny#2470) runs an app's server function +against a mock connection: inputs in, output values out, no browser and no +subprocess. That suits shinyreact particularly well — a ui.tsx server is *only* +reactive computation, so what a test wants to assert is the JSON that reaches +`useShinyOutputValue()`, which is exactly what `get_output()` hands back. + +The apps here are the Playwright fixture apps, reused as-is: these tests pin +the server half of claims whose client half still needs a browser (see +`pkg-py/tests/playwright/`). `test_server` needs no `tests-e2e` extras, so this +file lives in the unit suite. + +Two shinyreact-specific notes for anyone adding to this file: + +- **Untyped input ids need no `:shinyreact.default` suffix.** The React hook + appends it on the wire, but both Python handlers are no-ops, so + `set_inputs(n=1)` and `set_inputs(**{"n:shinyreact.default": 1})` reach the + server identically. Use the suffixed form only when the handler itself is + what you are testing (see `test_input_handler_dispatch.py`). +- **Event inputs need two calls.** A client registers a `useShinyInput` + default at mount and sends the event after, so an output guarded by + `@reactive.event(..., ignore_init=True)` only fires on the *second* + `set_inputs` for that id. +""" + +from __future__ import annotations + +from pathlib import Path + +from shiny.testserver import test_server + +APPS = Path(__file__).parent / "playwright" / "apps" + + +def test_reactive_output_publishes_raw_json() -> None: + """The value a `reactive_output` returns reaches the client unwrapped. + + `test_reactive_output.py` asserts this through `Renderer.transform()`; + here it goes through a real session, so nothing between the render function + and the wire can re-wrap it. + """ + with test_server(APPS / "module_counter" / "app.py") as ts: + ts.set_inputs(**{"a-count": 3}) + assert ts.get_output("a-serverCount") == 3 + + +def test_module_namespaces_stay_isolated() -> None: + """In-memory twin of `playwright/test_module_namespaces.py`. + + A scope is the test-side counterpart of the `SessionProxy` a module server + receives, so the module's ids can be read bare. + """ + with test_server(APPS / "module_counter" / "app.py") as ts: + with ts.make_scope("a") as a: + a.set_inputs(count=2) + assert a.get_output("serverCount") == 2 + + # Namespace isolation: counter "b" is untouched, and its output has not + # rendered at all. + assert ts.get_output("b-serverCount").status == "silent" + assert ts.get_output("a-serverCount") == 2 + + +def test_output_error_statuses() -> None: + """Server half of `playwright/test_output_error.py`. + + Which of the three outcomes an input produces — a value, a silent + `req()`, or an error message — is decided server-side; only the rendering + of each needs the browser. + """ + with test_server(APPS / "output-error" / "app.py") as ts: + ts.set_inputs(n=1) + assert ts.get_output("answer") == "ok: 1" + assert ts.get_output("answer").status == "ok" + + # A *silent* req() failure is invisible in memory: real Shiny sends a + # null value that blanks the output (asserted in the e2e test), but + # `test_server` leaves the previous value recorded, so `status` stays + # "ok" with the stale "ok: 1". "silent" means "never rendered", not + # "rendered nothing this time" — don't assert silence here. + ts.set_inputs(n=-1) + assert ts.get_output("answer") == "ok: 1" + + ts.set_inputs(n=0) + assert ts.get_output("answer").status == "error" + assert ts.get_output("answer").error == "invalid number of 'breaks'" + + # Recovering clears the error. + ts.set_inputs(n=2) + assert ts.get_output("answer") == "ok: 2" + assert ts.get_output("answer").error is None diff --git a/pkg-r/README.md b/pkg-r/README.md index 0914a16f..2df1456b 100644 --- a/pkg-r/README.md +++ b/pkg-r/README.md @@ -63,6 +63,6 @@ shiny::runGitHub("posit-dev/shinyreact", subdir = "examples/01-hello") - `vignette("shinyreact")` walks through the `ui.tsx` pattern: inputs, outputs, messages, and embedding traditional Shiny renderers. - [TSX files and JavaScript build tools](https://posit-dev.github.io/shinyreact/articles/tsx-and-build-tools.html) explains `.tsx`, JSX, TypeScript, and what `npm run build` does. -- [Testing wire payloads](https://posit-dev.github.io/shinyreact/r/articles/testing.html) covers `wire_tap()` for shinytest2 tests. +- [Testing](https://posit-dev.github.io/shinyreact/r/articles/testing.html) covers `shiny::testServer()` for driving a server with no browser, and `wire_tap()` for wire payloads in shinytest2 tests. - [Agent Skills](https://posit-dev.github.io/shinyreact/r/articles/agent-skills.html) explains the skills that ship with the package for coding agents. - The [examples catalog](https://github.com/posit-dev/shinyreact/blob/main/examples/README.md) lists runnable apps from no-build to Vite + HMR. diff --git a/pkg-r/inst/skills/shinyreact-build-app/SKILL.md b/pkg-r/inst/skills/shinyreact-build-app/SKILL.md index 0a922ace..0a225bee 100644 --- a/pkg-r/inst/skills/shinyreact-build-app/SKILL.md +++ b/pkg-r/inst/skills/shinyreact-build-app/SKILL.md @@ -383,8 +383,8 @@ a list. Use `type: "shinyreact.asis"` for the parsed value untouched. Do not stop at "the code is written". Three steps, in order: 1. **Factor pure logic out of the app file** — binning, formatting, - conversions go in a module beside the app. Logic inside `app.py` / - `app.R` next to the page call cannot be reached by a test at all. + conversions go in a module beside the app, where a test can import them + without starting a session at all. 2. **Write down what the app does, in plain English, before the tests.** An agent that writes the client and then writes the client's tests is agreeing with itself — both encode the same misunderstanding. A @@ -394,16 +394,19 @@ Do not stop at "the code is written". Three steps, in order: directory** — `pytest`, or `[r]` `shiny::runTests()`, which needs the `tests/testthat.R` + `tests/testthat/` layout. -`[r]` `shiny::testServer()` drives the reactive graph with no browser: -`session$setInputs(bins = 9)` then assert on `output$dist_data`, which is the -JSON value the client would have received. **`[py]` has no equivalent**, so -factoring logic into an importable module matters more there. +Both languages drive the reactive graph with no browser, which for a `ui.tsx` +app is most of the server: `[r]` `shiny::testServer()` (`session$setInputs(bins += 9)`, then assert on `output$dist_data`) and `[py]` +`shiny.testserver.test_server()` (`ts.set_inputs(bins=9)`, then +`ts.get_output("dist_data")`). Either way the value you assert is the JSON the +client would have received. [`references/testing.md`](references/testing.md) has the four layers, the test -layout for each language, `testServer()` for plain and module servers, how to -mount the real `www/ui.js` against a fake Shiny, and the traps that cost time -(React ignores raw `change` events; debounce coalesces within a tick even at -`debounceMs: 0`). +layout for each language, `testServer()` / `test_server()` for plain and module +servers — including the input ids that need a `:type` suffix and the event +inputs that need two `set_inputs` calls — how to mount the real `www/ui.js` +against a fake Shiny, and the traps that cost time (React ignores raw `change` +events; debounce coalesces within a tick even at `debounceMs: 0`). ## Debugging diff --git a/pkg-r/inst/skills/shinyreact-build-app/references/testing.md b/pkg-r/inst/skills/shinyreact-build-app/references/testing.md index 8fa6e978..245ede5e 100644 --- a/pkg-r/inst/skills/shinyreact-build-app/references/testing.md +++ b/pkg-r/inst/skills/shinyreact-build-app/references/testing.md @@ -3,8 +3,11 @@ Do not stop at "the code is written". At minimum: 1. **Factor pure logic out of the app file.** Binning, formatting, conversions - go in a module beside the app so a test can import them; logic sitting - inside `app.py` / `app.R` next to the page call cannot be reached at all. + go in a module beside the app so a test can import them directly, with no + session at all. Logic left inside `app.py` / `app.R` next to the page call + is still reachable — `testServer()` / `test_server()` below drive the app + itself — but only through an input, which is a slower and blunter tool than + calling a function. 2. **Write down what the app does, in plain English, before the tests** — a behavior file beside the app, one atomically checkable claim per line ("the caption reads `N eruptions in M bins`, singular `bin` at M=1"). An @@ -16,7 +19,7 @@ Do not stop at "the code is written". At minimum: | Layer | Proves | Cost | |---|---|---| | pure functions in their own module | binning, formatting, conversions | trivial — always do this | -| `[r]` `shiny::testServer()` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | +| `shiny::testServer()` `[r]` / `shiny.testserver.test_server()` `[py]` | the reactive graph: inputs in, `reactive_output` values out | low, and no browser | | the client mounted in jsdom against a fake Shiny | rendering, input wiring, wire ids, status handling | low, and it exercises the file the app ships | | Playwright | layout, real Shiny, real bindings | high; reserve for what the others cannot see | @@ -31,7 +34,8 @@ myapp/ faithful.py the factored logic www/ui.js tests/ - test_faithful.py [py] pytest + test_faithful.py [py] pytest — the factored logic, called directly + test_outputs.py [py] pytest — the app, via test_server() testthat.R [r] runner: library(testthat); test_dir("testthat") testthat/ test-histogram.R [r] @@ -58,11 +62,17 @@ sys.path.insert(0, str(EXAMPLE)) from faithful import histogram, waiting # noqa: E402 ``` -## `[r]` Testing the server with `testServer()` +## Testing the server: `testServer()` `[r]`, `test_server()` `[py]` -`reactive_output` is an ordinary Shiny render function, so `shiny::testServer()` -drives it with no browser and no client: set inputs, read outputs, and get the -JSON value the client would have received. +**This is the highest-value layer for a `ui.tsx` app.** The server contains +only reactive computation, so "input X produces output Y" *is* the server, and +both languages can assert it with no browser and no client: set inputs, read +outputs, and get the JSON value the client would have received. + +### `[r]` `shiny::testServer()` + +`reactive_output` is an ordinary Shiny render function, so `testServer()` +drives it directly. ```r test_that("the histogram recomputes when bins changes", { @@ -90,10 +100,62 @@ which is most of what a shinyreact server does. Module servers work the same way: `testServer(card_server, args = list(id = "left"), { ... })`. -**`[py]` has no `testServer()` equivalent.** Cover the Python server by -factoring its logic into a module and testing that with pytest, and let the -jsdom and Playwright layers cover the wiring. This is a real gap, not an -oversight in your app. +### `[py]` `shiny.testserver.test_server()` + +The Python counterpart (py-shiny#2470, so newer than shiny 1.7.0). It loads the +app file — Express or Core, `shiny.App` or `shinyreact.ReactApp` — and runs its +server against a mock connection: + +```python +from pathlib import Path +from shiny.testserver import test_server + +APP = Path(__file__).resolve().parents[1] / "app.py" + + +def test_the_histogram_recomputes_when_bins_changes(): + with test_server(APP) as ts: + ts.set_inputs(bins=9) + assert ts.get_output("dist_data").value["counts"] == [16, 37, 30, 16, 14, 57, 67, 29, 6] + assert ts.get_output("dist_caption") == "272 eruptions in 9 bins" +``` + +`get_output()` returns a value that compares equal to the underlying one, so +assert on it directly; use `.value` when you need to index into it, and +`.status` (`"ok"` / `"error"` / `"silent"`) or `.error` to assert the +non-value outcomes. Traditional renderers are readable too, so +`@render.data_frame` / `@render_plotly` outputs mounted through `ShinyOutput` +can be checked at the wire level. + +Pass an **absolute `Path`**: a relative one resolves against the test file's +directory, and the app is a directory up from `tests/`. + +Four things to know, all of them shinyreact-specific: + +- **An untyped input id needs no `:shinyreact.default` suffix.** The hook + appends it on the wire, but both Python handlers are no-ops, so + `set_inputs(bins=9)` is equivalent. +- **A typed one does.** `set_inputs(**{"when:shiny.datetime": 1756382400})` is + what makes the handler run and `input.when()` a `datetime`; without the + suffix you are testing a different app than the one the client drives. +- **An event input needs two calls.** `useShinyInput` registers its default at + mount and sends the event after, so an output behind + `@reactive.event(..., ignore_init=True)` only fires on the *second* + `set_inputs` for that id. +- **An unset input means `status == "silent"`, not a `None` value.** + `input.x()` raises a silent exception while unset, so a `if x is None:` + branch in your server is unreachable from a real client — assert the status + instead. (A *later* `req()` failure is not visible in memory at all: + py-shiny#2492.) + +Module ids can be read as the session sees them (`ts.get_output("counter-n")`) +or through a scope, which strips the namespace on the way in and out: + +```python +with ts.make_scope("counter") as counter: + counter.set_inputs(n=7) + assert counter.get_output("label") == "n=7" +``` ## The jsdom layer — mount the client the app ships diff --git a/pkg-r/inst/skills/shinyreact-convert-app/SKILL.md b/pkg-r/inst/skills/shinyreact-convert-app/SKILL.md index ed308117..c8184922 100644 --- a/pkg-r/inst/skills/shinyreact-convert-app/SKILL.md +++ b/pkg-r/inst/skills/shinyreact-convert-app/SKILL.md @@ -149,12 +149,16 @@ rendered), then the rest of the outputs, then layout and polish. ## Phase 5 — verify -Three layers, cheapest first: factor pure logic out of the app file so it is -importable and test it directly; test the client by evaluating the real -`www/ui.js` against a fake `window.Shiny` in jsdom (not by importing the -component — that tests a copy the app does not ship); and reserve Playwright -for layout and real bindings, which the other two structurally cannot see. The -`shinyreact-build-app` skill's `references/testing.md` has the traps. +Four layers, cheapest first: factor pure logic out of the app file so it is +importable and test it directly; drive the ported server with no browser — +`[r]` `shiny::testServer()`, `[py]` `shiny.testserver.test_server()` — which +for a `ui.tsx` app covers most of it, since the server is only reactive +computation; test the client by evaluating the real `www/ui.js` against a fake +`window.Shiny` in jsdom (not by importing the component — that tests a copy the +app does not ship); and reserve Playwright for layout and real bindings, which +the others structurally cannot see. The `shinyreact-build-app` skill's +[`references/testing.md`](../shinyreact-build-app/references/testing.md) has +the details and the traps. A port has one advantage a new app does not: **the original still runs.** Where the logic is a pure transform, capture its output from the original app and diff --git a/pkg-r/tests/testthat/test-examples.R b/pkg-r/tests/testthat/test-examples.R index 4d4e3498..c57bb728 100644 --- a/pkg-r/tests/testthat/test-examples.R +++ b/pkg-r/tests/testthat/test-examples.R @@ -44,8 +44,18 @@ test_that("R example apps are installed at examples-shiny/", { test_that("shipped R example apps match examples/", { skip_if_not(dir.exists(examples_dir), "not running from the repo") + # Only app.R + www/ are copied; the example's README/FEATURES/tests and its + # Python siblings stay in examples/. Filter before reading, not after: the + # example directories also hold binary files (__pycache__, node_modules) + # that readLines() warns about. + copied <- "^(app\\.R|www/)" + tree <- function(root) { - files <- sort(list.files(root, recursive = TRUE)) + files <- sort(grep( + copied, + list.files(root, recursive = TRUE), + value = TRUE + )) stats::setNames( lapply(files, function(f) readLines(file.path(root, f))), files @@ -53,10 +63,7 @@ test_that("shipped R example apps match examples/", { } for (name in shipped_r_examples) { - # Only app.R + www/ are copied; the example's README/FEATURES/tests and its - # Python siblings stay in examples/. src <- tree(file.path(examples_dir, name)) - src <- src[grepl("^(app\\.R|www/)", names(src))] expect_equal( tree(system.file("examples-shiny", name, package = "shinyreact")), src, diff --git a/pkg-r/tests/testthat/test-render.R b/pkg-r/tests/testthat/test-render.R index c96bc645..0bffdcc6 100644 --- a/pkg-r/tests/testthat/test-render.R +++ b/pkg-r/tests/testthat/test-render.R @@ -49,3 +49,70 @@ test_that("reactive_output attaches no UI placeholder or dependency", { expect_no_match(placeholder$html, "my_id", fixed = TRUE) expect_length(placeholder$dependencies, 0L) }) + +test_that("reactive_output is namespaced inside a module server", { + # Mirrors test_module_namespaces_stay_isolated in + # pkg-py/tests/test_in_memory_server.py, which drives the same app shape + # through shiny.testserver.test_server(). The browser-level counterpart is + # pkg-py/tests/playwright/test_module_namespaces.py; R has no e2e suite yet + # (#194), so this is R's coverage of the claim. + counter_server <- function(id) { + moduleServer(id, function(input, output, session) { + output$serverCount <- reactive_output( + if (is.null(input$count)) 0L else input$count + ) + }) + } + server <- function(input, output, session) { + counter_server("a") + counter_server("b") + } + + shiny::testServer(server, { + session$setInputs(`a-count` = 3L) + expect_identical(output$`a-serverCount`, 3L) + # Namespace isolation: counter "b" is untouched. + expect_identical(output$`b-serverCount`, 0L) + }) + + # The same ids through a scope, which is how a module's own test reads them. + shiny::testServer(server, { + a <- session$makeScope("a") + a$setInputs(count = 2L) + expect_identical(output$`a-serverCount`, 2L) + }) +}) + +test_that("reactive_output surfaces errors and silent errors to the test", { + # Mirrors test_output_error_statuses in pkg-py/tests/test_in_memory_server.py + # -- with a deliberate divergence in the *testing* API, not in shinyreact: + # R's testServer() re-raises, so reading the output is the assertion, while + # Python's test_server() reports `.status` / `.error` instead. Python also + # keeps the previous value after a silent error where R does not + # (posit-dev/py-shiny#2492). + server <- function(input, output, session) { + output$answer <- reactive_output({ + n <- input$n + req(!is.null(n)) + if (n == 0) { + stop("invalid number of 'breaks'") + } + paste0("ok: ", n) + }) + } + + shiny::testServer(server, { + # req() fails: a silent error, with no message. + expect_error(output$answer, class = "shiny.silent.error") + + session$setInputs(n = 1) + expect_identical(output$answer, "ok: 1") + + session$setInputs(n = 0) + expect_error(output$answer, "invalid number of 'breaks'") + + # Recovering clears the error. + session$setInputs(n = 2) + expect_identical(output$answer, "ok: 2") + }) +}) diff --git a/pkg-r/vignettes/articles/testing.Rmd b/pkg-r/vignettes/articles/testing.Rmd index d3c51918..1f72857e 100644 --- a/pkg-r/vignettes/articles/testing.Rmd +++ b/pkg-r/vignettes/articles/testing.Rmd @@ -1,5 +1,5 @@ --- -title: "Testing wire payloads" +title: "Testing" --- ```{r, include = FALSE} @@ -9,11 +9,71 @@ knitr::opts_chunk$set(collapse = TRUE, comment = "#>", eval = FALSE) In a shinyreact app the contract between server and client is the JSON that crosses the Shiny websocket: the values `reactive_output()` delivers, the payloads `send_message()` pushes, and the values `useShinyInput()` sends back. -`wire_tap()` records those payloads in a +Both halves of this article assert that JSON — the first with no browser at +all, the second inside a real one. + +## Testing the server with `testServer()` + +A `ui.tsx` server contains only reactive computation, so "this input produces +that output value" *is* the server. `reactive_output()` is an ordinary render +function, so `shiny::testServer()` drives it with no browser and no client, and +`output$id` is the JSON value itself — no spec wrapper, no coercion: + +```{r} +test_that("dist_data bins the waiting column", { + shiny::testServer(app_dir, { + session$setInputs(bins = 9) + + expect_named(output$dist_data, c("breaks", "counts")) + expect_equal( + unclass(output$dist_data$counts), + c(16L, 37L, 30L, 16L, 14L, 57L, 67L, 29L, 6L) + ) + expect_equal(output$dist_caption, "272 eruptions in 9 bins") + }) +}) +``` + +`unclass()` is there because the app wraps its vectors in `I()` so a one-bin +result serializes as `[272]` rather than `272` — the `AsIs` class rides along +on the value the test sees. + +Pass a directory containing the app, or the `server` function itself. Module +ids are namespaced as the session sees them (``output$`counter-label` ``), or +reachable through `session$makeScope("counter")`; a module server can also be +driven on its own: + +```{r} +shiny::testServer(card_server, args = list(id = "left"), { + session$setInputs(n = 7) + expect_equal(output$label, "n=7") +}) +``` + +`testServer()` **raises** rather than reporting a status. Reading an output +whose render function failed `req()` throws a silent error, which is how you +assert that an output produced nothing: + +```{r} +expect_error(output$answer, class = "shiny.silent.error") +``` + +`examples/01-hello` and `examples/07-plotly` both carry a +`tests/testthat/test-outputs.R` written this way, runnable from the app +directory with `shiny::runTests()`. + +The Python counterpart is `shiny.testserver.test_server()`, which reports +`.status` (`"ok"`, `"error"`, `"silent"`) instead of raising. + +## Testing wire payloads with `wire_tap()` + +`wire_tap()` records the payloads crossing the websocket in a [shinytest2](https://rstudio.github.io/shinytest2/) test so you can assert on -them directly, without inspecting the rendered DOM. +them directly, without inspecting the rendered DOM. Reserve it for what +`testServer()` structurally cannot see: the values the *client* chooses to +send, real bindings, and real rendering. -## Setup +### Setup `wire_tap()` needs the shinytest2 package. Start the `AppDriver` with `shiny.trace = TRUE` so every websocket frame is recorded in the app's logs: @@ -32,7 +92,7 @@ test_that("dist_data bins the waiting column", { }) ``` -## Matchers +### Matchers Each `expect_*` function takes a matcher and retries until it matches or a timeout (10 seconds by default) elapses: @@ -54,7 +114,7 @@ Successive expectations on one channel assert an ordered subsequence: each scans from just past the previous match, so a value that arrives between two checks is never missed. -## Full histories +### Full histories To inspect everything that crossed a channel, use the `all_*` functions: @@ -67,7 +127,7 @@ tap$all_input_values("bins") Input ids match the bare id or any `id:type` wire id, so use the id you wrote in `useShinyInput()`. -## Python counterpart +### Python counterpart `shinyreact.playwright.WireTap` in the Python package has the same methods and semantics for Playwright tests. One divergence: `jsonlite::fromJSON()` maps a diff --git a/pkg-r/vignettes/shinyreact.Rmd b/pkg-r/vignettes/shinyreact.Rmd index bd2a3943..af07f8be 100644 --- a/pkg-r/vignettes/shinyreact.Rmd +++ b/pkg-r/vignettes/shinyreact.Rmd @@ -189,8 +189,9 @@ them as initial values instead of its default. - [TSX files and JavaScript build tools](https://posit-dev.github.io/shinyreact/articles/tsx-and-build-tools.html) explains `.tsx`, JSX, TypeScript, and what `npm run build` does, for readers new to JavaScript tooling. -- [Testing wire payloads](https://posit-dev.github.io/shinyreact/r/articles/testing.html) - shows how to assert the JSON that crosses the websocket with `wire_tap()`. +- [Testing](https://posit-dev.github.io/shinyreact/r/articles/testing.html) + shows how to assert the JSON a server produces with `shiny::testServer()`, + and the JSON that crosses the websocket with `wire_tap()`. - The [JS reference](https://posit-dev.github.io/shinyreact/js/) documents every hook and component at `window.shinyreact`. - [`DESIGN.md`](https://github.com/posit-dev/shinyreact/blob/main/DESIGN.md) From 9dfa818f0fd925cb4e7ffc3075627523b30701a7 Mon Sep 17 00:00:00 2001 From: Barret Schloerke Date: Fri, 11 Sep 2026 17:02:05 -0400 Subject: [PATCH 2/3] docs(examples): say why 11-npm-local's server leaves carry no (test) Merging main brought in a new example. Its tests deliberately never load the app -- in Python that would trigger its build-on-first-run -- so test_server() is out of reach there, which is worth a leaf rather than a silent gap. --- examples/11-npm-local/FEATURES.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/examples/11-npm-local/FEATURES.md b/examples/11-npm-local/FEATURES.md index 135a535b..6c401f5f 100644 --- a/examples/11-npm-local/FEATURES.md +++ b/examples/11-npm-local/FEATURES.md @@ -57,6 +57,11 @@ a unit test; `(verify)` marks a claim not yet checked against the code. - `[py]` `input.bins()` raises a silent exception, so neither output produces a value - `[r]` `input$bins` is `NULL` and both outputs return `NULL` explicitly +- alone among the examples, the outputs are **not** driven in memory: + `test_server()` / `testServer()` load the app, which is the one thing this + example's tests avoid — `[py]` loading `app.py` triggers its + build-on-first-run. `examples/01-hello` pins the same binning and the same + two output shapes against three servers ## Client (`src/ui.jsx`) From dd12576aa2f15509a90c0cd18c2227b00fd2d0e3 Mon Sep 17 00:00:00 2001 From: Barret Schloerke Date: Fri, 11 Sep 2026 17:34:31 -0400 Subject: [PATCH 3/3] test(examples): fix review findings in the in-memory example tests - 07-plotly: hoist skip_if_not_installed("plotly") to file scope, since app.R calls plotly::renderPlotly() in server() and every testServer() call needs it, not just the scatter test - 08-input-handler: replace a tautological stdlib assertion with an exact check on the echoed datetime; the handler decodes as UTC and strips tzinfo, so the repr is machine-independent. Fix the FEATURES.md leaf that claimed a tzinfo= in the repr. --- examples/07-plotly/tests/testthat/test-outputs.R | 13 ++++++++++--- examples/08-input-handler/FEATURES.md | 5 +++-- examples/08-input-handler/tests/test_when.py | 7 +++---- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/examples/07-plotly/tests/testthat/test-outputs.R b/examples/07-plotly/tests/testthat/test-outputs.R index a1f198d2..924fc1b7 100644 --- a/examples/07-plotly/tests/testthat/test-outputs.R +++ b/examples/07-plotly/tests/testthat/test-outputs.R @@ -13,6 +13,10 @@ # pkg-r/tests/testthat/test-examples.R. app_dir <- testthat::test_path("..", "..") +# app.R calls plotly::renderPlotly() inside server(), so every testServer() +# call here -- not just the scatter test -- needs plotly installed. +testthat::skip_if_not_installed("plotly") + test_that("greeting counts the points", { shiny::testServer(app_dir, { session$setInputs(num_points = 50) @@ -33,8 +37,6 @@ test_that("greeting is NULL before the client's first message", { }) test_that("scatter renders a plotly widget with no plotlyOutput() anywhere", { - skip_if_not_installed("plotly") - shiny::testServer(app_dir, { session$setInputs(num_points = 50) @@ -44,7 +46,12 @@ test_that("scatter renders a plotly widget with no plotlyOutput() anywhere", { # after the flush. expect_s3_class(output$scatter, "json") expect_match(output$scatter, '"type":"scatter"', fixed = TRUE) - dep_names <- vapply(attr(output$scatter, "deps"), `[[`, character(1), "name") + dep_names <- vapply( + attr(output$scatter, "deps"), + `[[`, + character(1), + "name" + ) expect_true("plotly-main" %in% dep_names) # The *binding* JS is not in here: page_react() discovers it from the # render function and pushes it as a shinyreact-deps message instead, diff --git a/examples/08-input-handler/FEATURES.md b/examples/08-input-handler/FEATURES.md index ccab0e53..baaf3051 100644 --- a/examples/08-input-handler/FEATURES.md +++ b/examples/08-input-handler/FEATURES.md @@ -23,11 +23,12 @@ a unit test; `(verify)` marks a claim not yet checked against the code. ## Server (`app.py`, Express) - one output, `when_info` → `""`, e.g. - `"datetime → datetime.datetime(2026, 8, 28, 12, 0, tzinfo=...)"` `(test)` + `1756382400` → `"datetime → datetime.datetime(2025, 8, 28, 12, 0)"` `(test)` + - Shiny's handler decodes as UTC and strips the `tzinfo`, so the repr is + naive and machine-independent `(test)` - before the first client message, `input.when()` raises a silent exception, so `when_info` never renders — the `"—"` (em dash) branch in `app.py` is unreachable from a real client `(test)` -- the example asserts nothing about the timezone the handler attaches - `[py]` only — this example has no R server; R's handler registry has no `shiny.datetime` equivalent, so the client is not portable as written diff --git a/examples/08-input-handler/tests/test_when.py b/examples/08-input-handler/tests/test_when.py index fafe63aa..c1adee84 100644 --- a/examples/08-input-handler/tests/test_when.py +++ b/examples/08-input-handler/tests/test_when.py @@ -11,7 +11,6 @@ from __future__ import annotations -import datetime from pathlib import Path from shiny.testserver import test_server @@ -25,9 +24,9 @@ def test_the_handler_turns_unix_seconds_into_a_datetime() -> None: ts.set_inputs(**{"when:shiny.datetime": 1756382400}) echoed = ts.get_output("when_info").value - assert echoed.startswith("datetime → datetime.datetime(") - # The example asserts nothing about the timezone, only the type. - assert datetime.datetime.fromtimestamp(1756382400).year == 2025 + # Shiny's handler decodes as UTC and strips the tzinfo, so the value + # is the same on every machine: 1756382400 is 2025-08-28T12:00:00Z. + assert echoed == "datetime → datetime.datetime(2025, 8, 28, 12, 0)" def test_an_unsuffixed_value_is_not_coerced() -> None: