Skip to content

test(session): run the gridwire grid conformance corpus in CI - #20

Merged
kingb merged 1 commit into
mainfrom
test/grid-conformance
Sep 28, 2026
Merged

kingb merged 1 commit into
mainfrom
test/grid-conformance

Conversation

@kingb

@kingb kingb commented Sep 27, 2026

Copy link
Copy Markdown
Owner

Makes Ember's agreement with the shared gridwire grid corpus permanent. Until now it rested on a single manual run.

What

  • scripts/conformance/fetch-grid-corpus.sh fetches conformance/grid/ from the public gridwire repo at the revision pinned in conformance/GRIDWIRE_REV (currently e977304). Bump the pin on purpose, since new corpus cases are new obligations.
  • crates/ember-session/tests/grid_conformance.rs runs Ember's real AlacrittyProjection over every case and compares the decoded deltas.
  • CI: a new step in the check job fetches the corpus, runs the test with --ignored, and greps for its pass line, so a renamed or filtered-out test can't pass the step by running nothing.

Why it's shaped this way

This is a conformance test, not a unit test. The corpus isn't Ember's, and what it protects, agreement with other producers of the grid, can't be seen from Ember's own unit coverage. So it:

  • fails, never skips, on a missing, empty or unpaired corpus
  • fails when the corpus has a field Ember's GridDelta can't represent, instead of silently dropping it on decode
  • names the case and the first differing field when it fails, for someone who has never seen this PR

It's #[ignore]d so a plain cargo test doesn't need the network, and it shows up as ignored with a reason that says how to run it.

One real drift fixed

The first run caught a mismatch. gridwire decodes bracketed_paste, mouse_reporting and marks with #[serde(default)]; ember-core's GridDelta didn't, so it couldn't decode a delta that leaves them at their defaults. The attributes now match gridwire's exactly. There's no runtime change, because the local path passes deltas in-process.

Checked that it can fail

Sabotage Result
Projection stops reshipping the full style table on reset fails reset-style-carry ([1].new_styles: produced 0 entries, corpus expects 2) and reset-mid-stream
Corpus missing fails: "grid corpus not found … fails rather than skipping"
A case with no .expect.json fails, naming the case
An expected delta with a field Ember can't represent fails: "corpus field .focus_reporting has no counterpart in Ember's GridDelta"

Validation

cargo fmt --check, cargo clippy --all-targets --all-features -D warnings and cargo test --all --all-features are clean. The CI step, simulated locally: all 7 corpus cases agree.

🤖 Generated with Claude Code

https://claude.ai/code/session_014xVjnf8UuQPRQtv2QqgpPK

gridwire publishes a cross-producer corpus (conformance/grid/): byte inputs
and the GridDelta every producer of the neutral grid must turn them into.
Ember's local projection matched it in a one-time manual run; this makes that
agreement permanent by checking it on every CI run.

It is a conformance test, not a unit test. The corpus isn't Ember's, and the
property it protects (agreement with other producers) is invisible to Ember's
own unit coverage, so it lives on its own and fails rather than skipping:

- scripts/conformance/fetch-grid-corpus.sh fetches the corpus from the public
  gridwire repo at the revision pinned in conformance/GRIDWIRE_REV.
- tests/grid_conformance.rs drives the real projection through each case and
  compares decoded deltas. A missing, empty, or unpaired corpus fails, and so
  does a corpus field Ember's GridDelta can't represent (otherwise it would be
  dropped on decode and never compared). A failure names the case and the
  first differing field.
- The test is #[ignore]d so a plain cargo test doesn't need the network. The
  CI step runs it with --ignored and greps for its pass line, so a filtered-out
  test can't pass the step by running nothing.

Running it surfaced one real drift: gridwire reads bracketed_paste,
mouse_reporting and marks with #[serde(default)], and ember-core's GridDelta
didn't, so it couldn't decode a delta that omits them at their defaults. The
serde attributes now match gridwire's exactly. There's no runtime change, since
the local path passes deltas in-process.

Checked that it can fail: dropping the full style table on reset fails
reset-style-carry and reset-mid-stream with the field named, and a missing
corpus, an unpaired case, and an unknown corpus field each fail loudly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014xVjnf8UuQPRQtv2QqgpPK
@kingb
kingb merged commit 7b7f0e2 into main Sep 28, 2026
13 checks passed
@kingb
kingb deleted the test/grid-conformance branch September 28, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant