Skip to content

Bring agent guidance in line with make and CI - #1376

Merged
MaxGhenis merged 3 commits into
mainfrom
docs/guidance-match-make-ci
Oct 20, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
docs/guidance-match-make-ci

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1373. That PR made make typecheck cover app/, website/ and calculator-app/, made make test cover app/ and website/, and added app's Stylelint to CI. Its review noted that several agent commands and docs still described the older, app-only setup. This PR brings them in line. Docs and agent prompts only; no code, config or CI changes.

What changed

The four gaps from the #1373 review:

  • .claude/commands/typecheck.md ran npm run typecheck in app/ only. It now runs make typecheck from the repository root (all three projects, the same checks as CI's Type checking job).
  • .claude/agents/lint-format-validator.md didn't mention Stylelint. It now says make lint runs Stylelint on app/'s CSS (config app/.stylelintrc.json), adds how to fix Stylelint problems, and checks Prettier with CI's three commands (app: bun run prettier; website and calculator-app: bun run format:check). Before, it checked app/ only, and its "final verification" ran make format, which writes files instead of checking them.
  • docs/engineering/skills/testing.md listed app/ commands without naming the directory. It now points to make test, lists the app/ and website/ commands separately, and says calculator-app/ has no suite of its own (tests of its code run in app/, e.g. app/src/tests/unit/calculator/). Two corrections found while checking: bun run vitest -- --coverage fails (MISSING DEPENDENCY Cannot find dependency '@vitest/coverage-v8'; no coverage provider is in bun.lock), and bun run check runs typecheck, ESLint and Prettier but not Stylelint or tests. A scope line says the "Writing tests" conventions are app/'s; website/ tests live in website/src/__tests__/ and use @testing-library/react directly.
  • docs/engineering/skills/repository-guidance.md step 3 now says CI runs the tests on Node 26 while make test uses whichever node is first on PATH (app/.nvmrc says v24.3.0), so a test of formatted numbers can differ locally.

Siblings with the same lag, fixed in the same pass:

  • .claude/commands/eslint.md and prettier.md ran npm scripts in app/ only. They now run make lint (ESLint in all three, plus app Stylelint) and make format.
  • .claude/commands/finalize-pr.md names Stylelint and no longer says "PR is ready" after only typecheck, lint and format. Its last line now notes CI also runs make test and make build.
  • .claude/agents/typecheck-validator.md and lint-format-validator.md hard-coded cd /Users/administrator/Documents/PolicyEngine/policyengine-app-v2. They now run from the repository root.
  • .claude/commands/write-unit-tests.md and next-phase.md told agents to read app/src/tests/CLAUDE.md, which 3025aed moved to docs/engineering/skills/testing.md (R096). They now point there.
  • README.md's development workflow listed only make test && make lint. It now lists make format, make lint, make typecheck and make test and points to "Before committing"; the Testing section also shows make typecheck.
  • .claude/commands/write-unit-tests.md step 6 runs the modified tests with bun run vitest -- <paths>, as testing.md does.

How each claim was checked

  • Makefile on dc9c469: make -n typecheck test lint format prints app/website/calculator typecheck; app/website test; app/website/calculator lint; app prettier:write, website and calculator format.
  • app/package.json: lint = eslint && stylelint; eslint uses --max-warnings 0; stylelint = stylelint '**/*.css' --cache; check = typecheck + eslint + prettier. calculator-app/package.json has no test script, and no test files are tracked under calculator-app/.
  • .github/workflows/pr.yaml and push.yaml: the Lint and format job runs app Prettier, ESLint and Stylelint, then website and calculator Prettier checks and ESLint; the Tests job runs actions/setup-node with node-version: "26", then app and website tests. 714b6b3 gives the reason (Intl compact notation differs between Node versions; ScorecardComparisons).
  • node_modules/.bin/vitest starts with #!/usr/bin/env node. A probe bin with that shebang, run through bun run, reported the PATH Node's process.version and typeof Bun === "undefined", and a fake node placed first on PATH was the one invoked. The same probe showed bun run x -- --flag passes --flag through.
  • cd app && bun run vitest -- src/tests/placeholder.test.tsx: 1 passed. Adding --coverage: exit 1, missing @vitest/coverage-v8.
  • app/.nvmrc = v24.3.0; website/ and calculator-app/ have none. No workflow or config reads .nvmrc.
  • git diff --check clean. Prettier doesn't check .md files in any of the three projects.

Review

  • Round 1, independent review on Subfleet (--task review --tier hard; served by Opus 5.5 because every Codex lane was on hold): APPROVE at 53dd9ed, no blockers, four nits.
    • README's make test && make lint workflow line: fixed in 342c31d.
    • npx in /write-unit-tests step 6: fixed in 342c31d.
    • "bun run test runs the tests on Node" assumes node is on PATH: reworded in 342c31d.
    • Whether Bun strips the -- in bun run vitest -- <args>: the reviewer had no shell to check. It does: bun run vitest -- --coverage src/tests/placeholder.test.tsx echoes $ vitest run --coverage src/tests/placeholder.test.tsx, and bun run vitest -- src/tests/placeholder.test.tsx runs one test file. No change needed.
  • d930354 rewords README's workflow line: make format writes rather than checks, and CI also builds.
  • Round 2, delta review of 53dd9ed..d930354 (same route, Opus 5.5): APPROVE at d930354. One optional nit: the README line doesn't mention the embed-URL check. That check has no make target and is documented in "Embedded sites", so it was left as is. bun run vitest -- src/tests/unit/calculator/robots.test.ts runs exactly one file.

Not changed

  • .claude/commands/next-phase.md reads .claude-plans/calculation-system-refactor.md, which doesn't exist. The command looks obsolete; deleting it is a separate call.
  • app/.nvmrc still says v24.3.0 while CI tests on 26. Changing it is a config change, not a doc fix, so this PR only documents the gap.

🤖 Generated with Claude Code

PR #1373 made `make typecheck` cover app, website and calculator-app,
made `make test` cover app and website, and added app's Stylelint to
CI. Several agent commands and docs still described the older,
app-only setup.

- /typecheck, /eslint and /prettier run `make typecheck`, `make lint`
  and `make format` from the repository root instead of npm scripts in
  app/ only. /finalize-pr names Stylelint and no longer says the PR is
  ready without the tests and build.
- The lint-format-validator agent covers Stylelint and checks Prettier
  in all three projects, as CI does. Both validator agents run from the
  repository root instead of a hard-coded /Users/administrator path.
- testing.md says which directory each command runs in, points to
  `make test`, and says calculator-app has no suite of its own. Its
  coverage command fails because @vitest/coverage-v8 isn't installed;
  the doc now says so. `bun run check` doesn't run Stylelint or tests.
- repository-guidance.md says CI runs the tests on Node 26 while
  `make test` uses the local Node (app/.nvmrc says v24.3.0), so a test
  of formatted numbers can differ locally.
- /write-unit-tests and /next-phase pointed at app/src/tests/CLAUDE.md,
  which moved to docs/engineering/skills/testing.md on 2026-09-10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 20, 2026 8:54pm UTC
policyengine-website Ready Ready Preview Oct 20, 2026 8:54pm UTC

Request Review

- README's development workflow listed only `make test && make lint`;
  it now lists all four checks CI runs and points to "Before
  committing". The Testing section also shows `make typecheck`.
- /write-unit-tests runs the modified tests with `bun run vitest --`,
  as testing.md does, instead of npx.
- repository-guidance.md says `bun run test` runs vitest on the `node`
  first on PATH, rather than just "on Node".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Admin merge audit, pinned to reviewed head d930354c57a592e6d5a7cc9ae153ed741d86d850.

  • Independent review: Subfleet --task review --tier hard. Every Codex lane was on operator hold, so Opus 5.5 served it on lane claude-19.
    • Round 1 (job 20261020-164310-pr1376-review-r1) gave APPROVE at 53dd9ed with four nits. Three were fixed in 342c31d and d930354; the fourth was settled by running the command.
    • Round 2 (job 20261020-165341-pr1376-review-r2) was a delta review of 53dd9ed..d930354 and gave APPROVE at d930354. It raised one optional nit (the README line omits the embed-URL check), left as is.
    • No CHANGES_REQUESTED reviews.
  • CI: gh pr checks 1376 exits 0 on run 39495837014 at d930354. Lint and format, Type checking, Tests, Build, Check embedded site URLs, Chromatic and both Vercel previews pass.
  • Mergeable: MERGEABLE. main moved to dc0031b (Link calculator error pages to the visitor's country's reports #1374) after CI started. Its files (app/package.json, bun.lock, calculator error pages and their test) don't overlap this PR's 11 Markdown files. It only adds fast-check as a dev dependency, so every claim in these docs still holds.
  • Why admin: the ruleset requires one approving review, which the PR author can't give themselves. This is a docs-only change (agent prompts and engineering guidance), with no code, config, CI, published numbers or outward actions.

@MaxGhenis
MaxGhenis merged commit 8f2399c into main Oct 20, 2026
10 checks passed
@MaxGhenis
MaxGhenis deleted the docs/guidance-match-make-ci branch October 20, 2026 21:12

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — d930354c Deployed Oct 20, 2026 by vercel[bot]
Preview – policyengine-calculator-next — d930354c Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant