Railing runs every major UI component library against the W3C's own accessibility specification, continuously, and publishes the results.
Pick a component library today and you are choosing on vibes. Nobody can tell you whether its combobox is actually operable by keyboard, whether its dialog traps focus correctly, or whether the version you upgraded to last week quietly broke either of those things.
Railing answers that question with evidence, in public, on every release.
railing.dev · How it works · Scoring · Decision log · Add a library
A railing is installed for the people who cannot manage the stairs without one. Everybody carrying shopping, everybody in a hurry, everybody who has ever missed a step, ends up holding it too. Accessibility work is rarely only for the people it was built for.
Testing individual websites is retail. Testing the libraries they are built from is wholesale.
One broken combobox in a popular library is a broken combobox in tens of thousands of downstream applications. Fix it once upstream and it is fixed everywhere at once, including in the apps whose teams will never run an accessibility audit of their own.
| One-off audits | Railing | |
|---|---|---|
| Cadence | Once, then stale | Every release, forever |
| Comparability | One library at a time | Same assertions across every library |
| Basis | Auditor's judgement | W3C ARIA Authoring Practices Guide |
| Reproducibility | Trust the author | Raw JSON + replayable Playwright traces |
| Regressions | Invisible | Tracked per version |
That last row is the one that compounds. "This library was compliant in v2 and broke in v3" is a finding no blog post can produce, and it becomes more valuable the longer the project runs.
The hard problem is that you cannot write one test that runs everywhere. <Dialog> in Radix is not <Modal> in MUI is not <AlertDialog> in Chakra, different props, different DOM, different everything.
So we invert it.
┌──────────────────┐ mounts ┌────────────────────────┐
│ Library under │ ────────────────► │ Adapter │
│ test │ │ (~50 lines, per lib) │
└──────────────────┘ └───────────┬────────────┘
│ serves
▼
┌────────────────────────┐
│ /harness/dialog │
│ fixed URL, fixed IDs │
└───────────┬────────────┘
│ HTTP
▼
┌──────────────────┐ asserts ┌────────────────────────┐
│ Component spec │ ────────────────► │ Runner │
│ (from W3C APG) │ │ library-agnostic │
└──────────────────┘ └───────────┬────────────┘
│
▼
┌────────────────────────┐
│ Result JSON + traces │
└────────────────────────┘
- Component specs define canonical contracts, Dialog, Combobox, Tabs, Menu, Accordion, with assertions derived directly from the ARIA Authoring Practices Guide.
- Adapters mount a given library's version of that component into a fixed harness at a fixed URL with fixed test IDs.
- The runner navigates to the URL and executes the spec. It never knows which library it is testing.
Three consequences fall out of this design, and they are the whole project:
- Grounding assertions in APG makes the rubric defensible. When a maintainer objects, they are not arguing with our opinion, they are arguing with the W3C.
- Adapters are tiny, so the community can write them. Fifty lines is a first-contribution-sized task. We scale by adapters, not by writing more tests ourselves.
- The runner speaks HTTP, not React. Vue, Svelte, Angular and Web Component libraries slot in later without touching the engine.
See docs/ARCHITECTURE.md for the full design and docs/HARNESS-PROTOCOL.md for the adapter contract.
Pre-launch. Seven libraries are measured and every result is withheld.
Nothing is published until each maintainer has had their findings privately, with the adapter used to produce them and fourteen days to reply. That is the disclosure policy, and it is enforced in code.
| Milestone | State |
|---|---|
| Harness protocol + runner engine | Working |
| Dialog spec (13 assertions) | Implemented |
| Combobox spec (14 assertions) | Implemented |
| Menu spec (13 assertions) | Implemented |
| Tabs spec (13 assertions) | Implemented |
| Accordion spec (11 assertions) | Implemented |
| React Spectrum calibration control | 5 specs, all green |
| Radix reference adapter | 4 green, combobox n/a |
| Five subject adapters | measured, results withheld |
| Broken fixture, false positives/negatives measured | Calibrated on both |
| Public site, badges, CI | Live at railing.dev |
Read docs/PLAN.md for the route from here to launch.
Requires Node 20.11+ (.nvmrc pins 22) and pnpm 10.
nvm use
pnpm install
pnpm --filter @railing-dev/runner exec playwright install chromium
# Terminal 1: serve the Radix adapter
pnpm --filter @railing-dev/adapter-radix run dev
# Terminal 2: run the Dialog spec against it
pnpm railing run --target radix --component dialog --base-url http://localhost:5180Results are written to results/ as JSON, with a human-readable summary on stdout.
Every version this index measures is pinned exactly, in one place, and cross-checked:
pnpm check:versionsThat verifies the catalog, node_modules, targets.json and any existing results all name the same versions, including catching a stale result, a plausible-looking score describing a version nobody can install any more. See docs/VERSIONING.md.
Run the tests, which enforce the project's own rules rather than only documenting them:
pnpm testBuild the public index from whatever results are on disk:
pnpm site:buildThat emits web/out: the index, the written pages generated from docs/, a shields.io
badge endpoint per published result, and the raw JSON. Nothing about a library appears
until its maintainer has been notified and fourteen days have passed, which is enforced
in the build and in the tests rather than left to whoever runs it.
To work on the site itself:
pnpm site:devTo check the runner itself rather than a library:
pnpm fixture:broken # terminal 1
pnpm calibrate # terminal 2That runs the spec against a deliberately broken dialog and compares every assertion to a catalogue of known defects. It fails loudly on a false positive or a false negative, which is what keeps the numbers on the index worth publishing.
The highest-value contribution is an adapter for a library we do not cover yet. It is about fifty lines and needs no knowledge of the test engine. Start with docs/ADAPTERS.md and copy adapters/radix.
The second highest is auditing an existing adapter. An unfair result caused by a badly written adapter is the single biggest risk to this project's credibility, so adapters get reviewed harder than test code does.
Please read CONTRIBUTING.md, in particular the rule that we notify maintainers before we publish anything about their library.
Every released component score is also a shields.io endpoint, so a library can wear its measurement in its own README:
Badges exist only for released scores: the same publication gate that governs the site governs the API, so a badge URL for an unreleased library simply does not resolve. The badge updates when the score does, and the JSON behind it names the exact library version measured.
Railing is not a naming-and-shaming project. Every maintainer gets the full results and a right of reply before publication, and their response is published alongside the score. A library that fixes an issue before we publish is a success of this project, not a story we lost.
MIT