Skip to content

docs: CLI reference with a drift test, stack matrix, measured performance, custom-rules FAQ - #258

Merged
dgesteves merged 4 commits into
mainfrom
docs/os2-53-cli-reference
Oct 29, 2026
Merged

dgesteves merged 4 commits into
mainfrom
docs/os2-53-cli-reference

Conversation

@dgesteves

@dgesteves dgesteves commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Roadmap item OS2-53.

What was wrong

  • No CLI reference. --quiet-without-design-system was in no doc at all (only src/cli.ts). --baseline, --ignore-baseline, --include-tests, --components, --docs and --no-watch were each named once, in passing. inspect --format json was in no doc and not in --help. Nothing failed when a flag shipped undocumented.
  • No "does it work on my stack" table.
  • docs/how-it-works.md and the FAQ said only that onsystem "takes milliseconds", with no numbers.
  • Nothing answered "can I add our own rule?".

What changed

  • docs/cli.md (new, rendered at /docs/cli under Reference): every command, the options they share, then one section per command with its options. Also check's exit codes (0, 1, 2), the flag pairs check refuses, NO_COLOR and ONSYSTEM_CHANGED_LINES, and all five config file names (onsystem.config.mts too).

  • test/cli-docs.test.ts (the drift test). It checks, both ways:

    • --help lists exactly the options the parser in src/cli.ts accepts (read with the TypeScript AST).
    • Each option has a row under every command --help gives it to, with the same argument name.
    • The page has no row --help does not back.
    • It names every name in CONFIG_FILES.
    • Every action.yml input, read with the YAML parser, has its row and default in docs/required-check.md, and the table has no other row.
    • docs/configuration.md names every config file extension too.

    The page's tables are written for readers, not pasted from --help, so a wording change to the help does not fail the test; a new, removed or moved flag does. The one row --help does not back, inspect --format json, is listed in the test and run there on the demo.

  • docs/quickstart.md: a link to the CLI reference, and a "What it works with" table after the steps: UI code, styling, variants, components, tokens, monorepos, agents, CI and editors, runtime, each with what is not supported yet. It does not list component folders. It says "zero config finds the design system in the common layouts" and links how. Its Agents row calls tested only the Tier 1 setups of docs/clients.md#support-tiers (the Claude Code plugin, the MCP server in Claude Code and Gemini CLI, the skill through npx skills add), and says that the hook in Codex CLI and the MCP server in Cursor, VS Code and other clients follow each client's docs but have not been run in it, with a link to the tiers. test/support-tiers-docs.test.ts checks that no Tier 2 setup is named on the tested side.

  • docs/how-it-works.md: a "Performance and footprint" section. Its table is written by the new scripts/perf.mjs (node scripts/perf.mjs --write, after pnpm build and pnpm corpus fetch --only chatbot --only midday). It measures:

    • check on two corpus apps, with the design system cached and with --no-cache, then per file.
    • The Claude Code hook on one edit, with onsystem installed in the project and through npx.
    • The MCP server: initialize, the first tool call, and then check_ui, search_components and get_component.
    • The npm package: unpacked size, installed size and provenance.

    The install size is stated plainly (15 packages, 42.2 MB, 24.3 MB of it TypeScript), not framed as an advantage. The table says that the corpus checkouts have no node_modules, and that reading the design system takes longer in a project with its dependencies installed. The script prints the load average as it starts, and warns on stderr when the 1-minute load goes above half the cores or a run from the cache is slower than one with --no-cache. "Design decisions" now says "checks a file in milliseconds once the design system is read" and links the table.

  • test/perf-docs.test.ts checks:

    • The table names the machine, Node and onsystem versions.
    • It has the rows the script writes.
    • The corpus commits it names are the ones corpus/repos.json pins, so a corpus bump fails the test until the table is measured again.
    • The hook's file exists in the demo.
    • "Signed provenance" holds only while publishConfig.provenance is on.
    • Neither page says a bare "takes milliseconds" any more.
    • The node_modules caveat is the one the script writes.
  • FAQ (site/lib/faq.ts): "Can I add our own rules?" Not to onsystem itself (no rule API is planned, so the answer does not say "yet"). The rules are a fixed set, and the config rejects an unknown rule id. A raw <a> where the design system has a Link is already a prefer-design-system-component finding, and elements maps more. A team-only rule is an ESLint rule next to the ESLint plugin (not run by the hook or check_ui). A built-in rule follows CONTRIBUTING's "Adding a lint rule". test/custom-rules-faq.test.ts runs each claim: an unknown rule id is rejected, and <a href> next to a Link component is reported. "My components are not in components/ui" now names every flat folder of primitives wrappers zero config reads: src/, and since 0.11.1 (fix: zero-config examples, and React Aria starters in an app's src/components #257) src/components/, components/ and app/components/; the test holds it to docs/configuration.md.

  • docs/configuration.md: the Config file section lists onsystem.config.mts.

  • docs/cli.md, check: a file named outright is checked unless it is one of the design system's own component files (--include-design-system) or --skip-excluded leaves it out. In the demo, check components/ui/button.tsx prints "Nothing to check: 1 design-system file skipped" and exits 0. The exit-2 list also names no paths, an unknown --format, a non-integer --max-warnings, a missing --baseline file and --prune-baseline with no baseline; each was run on the demo and exits 2.

  • Site wiring for /docs/cli: site/app/docs/cli/page.tsx, site/lib/docs.ts (the sidebar reads this list, so site/components/docs-nav.tsx needed no change), site/lib/data.ts, site/lib/site.ts, site/lib/llms.ts, site/scripts/generate.mjs. Two site tests were updated for the new page: the Reference group in docs-shell.test.ts, and where the Tools section ends in llms-full.txt in docs-agree.test.ts.

No changeset: docs, site, a script and tests only. Nothing under src/ changed.

Numbers

Every number in the table comes from node scripts/perf.mjs --write, run on an Apple M1 Max (10 cores, 32 GB), macOS 27.0.1, Node.js 24.18.0, against onsystem 0.11.0 (the release then) on 2026-10-29, as the median of 5 runs. The table names that version.

The Mac was shared with other jobs. The 1-minute load average was about 5 to 6 during the final run. In one earlier run, under a load of about 12, the npx hook row came out at 3.47 s. Every other run gave 1.12 to 1.21 s for that row, and a separate probe of six npx hook runs gave 1.13 to 1.22 s after the first. The corpus checkouts have no node_modules, and the table says so.

After the review, a one-run check of the updated script against onsystem 0.11.1, under a 1-minute load of about 7, printed the load average and the new warning ("the 1-minute load average reached 7.65 on 10 cores"). Its numbers were close to the table's, except --no-cache on midday (3.55 s against 1.68 s), so the table was not rewritten from that run.

How it was tested

  • The new tests fail on main:
    • test/cli-docs.test.ts: docs/cli.md does not exist. With the page in place but its --quiet-without-design-system row removed, it fails with --quiet-without-design-system under "check": expected undefined to be defined.
    • test/perf-docs.test.ts: no "Performance and footprint" section, and "takes milliseconds" is still there.
    • test/custom-rules-faq.test.ts: no FAQ entry.
  • All of these passed on the branch after merging main (0.11.1, with fix: zero-config examples, and React Aria starters in an app's src/components #257):
    • pnpm install --frozen-lockfile --prefer-offline
    • pnpm format:check, pnpm lint, pnpm typecheck
    • pnpm test: 110 files, 1,426 passed, 3 skipped
    • pnpm build: schema.json unchanged
  • Site: pnpm --filter site generate, test (295 passed), typecheck, lint and build all passed. Served locally after the review fixes: /docs/cli, /docs, /docs/faq, /docs/how-it-works and /docs/configuration show the new text, and the quickstart links /docs/clients#support-tiers.
  • Browser: I served the built site locally and looked at /docs/cli, /docs#what-it-works-with and /docs/how-it-works#performance-and-footprint. The sidebar entry, the on-this-page list and the tables render. /docs/cli.md, /og/docs/cli, the sitemap entry, the FAQ anchor and the llms-full.txt section are all served.
  • Corpus: node scripts/corpus.mjs --only chatbot --only midday prints "Counts match corpus/snapshot.json". No file under src/ changed, so the counts cannot move.

Follow-ups (outside this PR's files)

  • src/cli.ts: --help says onsystem.config.{json,ts,mjs,js}, which leaves out .mts, and does not list inspect --format json. When it does, drop the entry in BEYOND_HELP in test/cli-docs.test.ts.
  • docs/troubleshooting.md still has no sections on the hook staying silent, the ESLint plugin, Windows, or Node.js below 20.19.
  • docs/ci.md and docs/required-check.md: the workflow YAML is npm only (cache: npm, npm ci). A pnpm variant is still to do.
  • docs/rules.md: what allow does per rule is only on the site's rules pages.
  • README: an npm provenance badge. README is in OS2-40 in this wave.
  • FAQ, "Does it work with agents other than Claude Code?": it says the hook runs in Codex CLI, which docs/clients.md lists as Tier 2 (installed, not run).
  • Re-run node scripts/perf.mjs --write on an idle machine to measure 0.11.1.

…ance, custom-rules FAQ

docs/cli.md lists every command and option of the CLI, under the
commands --help gives them to, with check's exit codes. Before it,
--quiet-without-design-system was in no doc, and --baseline,
--ignore-baseline, --include-tests, --components, --docs and --no-watch
were each named once, in passing. test/cli-docs.test.ts holds the page
to --help and --help to the parser (src/cli.ts), both ways, checks that
it names every config file name (onsystem.config.mts too), and that
every GitHub Action input has its row and default in
docs/required-check.md. The site renders the page at /docs/cli.

The quickstart gets a "What it works with" table: UI code, styling,
variants, components, tokens, monorepos, agents, CI and runtime, with
what is not supported yet.

docs/how-it-works.md said only that onsystem "takes milliseconds". Its
new "Performance and footprint" section has a table written by
scripts/perf.mjs: check on two corpus apps (per run and per file), the
Claude Code hook on one edit with an install and through npx, the MCP
server's handshake, first call and tool calls, and the package size,
install size and provenance, with the machine named.
test/perf-docs.test.ts holds the table to what the script writes and to
the corpus commits it measured. The FAQ's "takes milliseconds" points
to the table too.

The FAQ answers "Can I add our own rules?": the rules are a fixed set,
a raw <a> next to a Link is already a finding, a team-only rule is an
ESLint rule, and CONTRIBUTING.md has the steps for a built-in one.
…are tested

- cli.md: a file named outright is checked unless it is one of the design
  system's own component files or --skip-excluded leaves it out; the exit-2
  list names no paths, an unknown --format, a non-integer --max-warnings, a
  missing --baseline file and --prune-baseline with no baseline.
- quickstart.md: the Agents row calls tested only the Tier 1 setups and
  links the support tiers for Codex CLI, Cursor, VS Code and the rest.
- FAQ: no "yet" on custom rules, since no rule API is planned; zero config
  is said to read a flat src/, src/components/, components/ or
  app/components/ of primitives wrappers.
- configuration.md lists onsystem.config.mts.
- perf.mjs prints the load average as it starts, and warns when the load is
  high or a run from the cache is slower than one without it; the table
  says that reading the design system takes longer with dependencies
  installed.
- Tests: action.yml inputs are read with the YAML parser and held to one
  table row each; the perf line accepts "1 run"; the quickstart, FAQ,
  configuration.md and perf caveat are checked.
Comment thread test/perf-docs.test.ts Fixed
@dgesteves
dgesteves merged commit 3c74d21 into main Oct 29, 2026
17 checks passed
@dgesteves
dgesteves deleted the docs/os2-53-cli-reference branch October 29, 2026 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants