Repository navigation
chore(agents): add review checks and a review skill #108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
scottlovegrove
merged 2 commits into
scottlovegrove/agents-3-skills
from
scottlovegrove/agents-4-review
Nov 5, 2026
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Check: architecture | ||
|
|
||
| Rules lint and types can't express. Most are written in `AGENTS.md` and `CODEBASE.md`; report where a change works around one. | ||
|
|
||
| ## Report | ||
|
|
||
| - A user-facing error thrown as a bare `Error`, or printed with `console.error()` and an exit code, instead of `CliError(code, message, hints?)` from `src/lib/errors.ts`; a new code missing from the `ErrorCode` type; `process.exit()` in a command handler. | ||
| - A new SDK call without an entry in `API_SPINNER_MESSAGES` (`src/lib/api.ts`), or a read-only one missing from `KNOWN_SAFE_API_METHODS` in `src/lib/permissions.ts` (a read-only token would block it). | ||
| - A mutating command without `--json` where it has scripting value, or `json`/`full` options added by hand instead of extending `MutationOptions` in `src/lib/options.ts`. | ||
| - Reference parsing (ids, URLs, fuzzy names) written again instead of using `src/lib/refs.ts`. | ||
| - A positional `[workspace-ref]` without the `--workspace` alias, or one that accepts both without an error. | ||
| - A new top-level command not wired into the lazy registry in `src/index.ts`. | ||
| - A barrel file, or a helper that duplicates one in `src/lib/` or `@doist/cli-core`. Where cli-core has a convention, the CLI follows it. | ||
|
|
||
| ## Fine | ||
|
|
||
| - An exception documented next to the code, with the reason. | ||
| - A new broadly reusable helper in `src/lib/`, added to the `CODEBASE.md` catalog. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Check: command surface | ||
|
|
||
| What people and agents see: command and flag names, help text, human output, `--json`, `--ndjson` and `--full` shapes, error codes, and `SKILL_CONTENT`. Scripts and installed agent skills depend on every one. | ||
|
|
||
| ## Report | ||
|
|
||
| - A renamed or removed command, flag or `--json` field, or a changed error code, without a breaking-change PR title (`feat!:`, `fix!:`). | ||
| - A command or flag change not reflected in `SKILL_CONTENT` (`src/lib/skills/content.ts`) and `skills/comms-cli/SKILL.md`, or an example there that no longer runs. | ||
| - List output that tells an id and a name apart only by colour or dimming, so the line no longer reads in `--accessible` mode. | ||
| - Meaning carried only by colour or symbols, with no `--accessible` alternative. | ||
| - Results printed to stderr or diagnostics to stdout; a spinner or prompt in `--json` or `--ndjson` mode. | ||
| - Several spinner labels for parallel fetches that build one view. They share one label. | ||
| - A flag named differently from the same flag elsewhere (`--workspace`, `--json`, `--full`, `--dry-run`, `--yes`), or a new verb where an existing one fits. | ||
| - An error message that does not name the problem, or hints without a working command. | ||
|
|
||
| ## Fine | ||
|
|
||
| - New optional flags and new `--json` fields (additive) in a `feat:` PR. | ||
| - Help text wording fixes. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Check: evidence | ||
|
|
||
| Apply to the PR description and any report from an agent. | ||
|
|
||
| ## Report | ||
|
|
||
| - A behaviour claim ("fixed", "works", "safe", "no regressions") with no test or command that ran it. | ||
| - A "Not verified" section missing when something could not be checked: the real Comms API, output in a real terminal, the other Node version CI tests on. | ||
| - A change to shared code (`src/lib/`, command registration or global flags in `src/index.ts`, an `@doist/cli-core` or `@doist/comms-sdk` upgrade) without the one fact that makes it safe. | ||
| - A PR title whose type doesn't match the change: behaviour changes need `feat:`, `fix:` or `perf:` to release (`!` for breaking); tooling and docs use `chore:`, `ci:`, `docs:` or `test:` so they don't. | ||
|
|
||
| ## Fine | ||
|
|
||
| - Claims about what the code shows, with the cited line. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Check: privacy and secrets | ||
|
|
||
| This repository is public, and the npm package ships `dist/`, `scripts/` and `CHANGELOG.md`. | ||
|
|
||
| ## Report | ||
|
|
||
| - A secret, API key, token or password, even an expired one. | ||
| - Real user data in fixtures, tests or docs: real names, emails, user or workspace IDs, message content or URLs copied from an account. | ||
| - Personal identifiers in docs and examples: real emails, usernames, home-directory paths, local repository names. Use placeholders (`alice@example.com`, `~/code`). | ||
| - Internal details: private hostnames, internal URLs, Doist-internal tools or documents. The staging host already used in tests as a stand-in base URL is fine. | ||
| - A new file under `scripts/` or in package.json `files` that should not be published. | ||
| - Untracked files the change doesn't need (`.env`, logs, `.tmp/`). | ||
|
|
||
| ## Fine | ||
|
|
||
| - Made-up fixtures and cli-core's fictional test accounts (Alan Grant, Ellie Sattler). | ||
| - Public URLs: the Comms app and API, GitHub. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Check: security | ||
|
|
||
| The CLI holds the user's Comms token and reads their messages. Apply to changes in `src/` and `scripts/`. | ||
|
|
||
| ## Report | ||
|
|
||
| - A token, `Authorization` header or `COMMS_API_TOKEN` value written to output, logs, errors, `--json` or progress events. | ||
| - A secret fetched that the command does not display. An opt-in reveal flag must gate the API call, not just the printing. | ||
| - Token storage that bypasses the keyring-backed store in `src/lib/auth.ts`, or a token written to the config file outside the documented offline fallback (`src/lib/user-records.ts`). | ||
| - A mutating SDK method added to `KNOWN_SAFE_API_METHODS` in `src/lib/permissions.ts`, which lets it run under a read-only token. | ||
| - Private channel content shown without `--include-private-channels` (or `COMMS_INCLUDE_PRIVATE_CHANNELS`): new output that skips the filtering in `src/lib/public-channels.ts`. | ||
| - A shell command built from user or API input as a string, instead of `spawn`/`execFile` with an argument list (`openEditor()` in `src/lib/input.ts` runs `$EDITOR`); a URL from API data opened in the browser without checking it is an http(s) URL. | ||
| - OAuth changes (`src/lib/auth-provider.ts`, `src/lib/scopes.ts`, `oauth/client-metadata.json`) that widen the scopes a plain `tdc auth login` asks for. | ||
| - `src/postinstall.ts` doing more than refreshing installed skills: it runs on every install. | ||
|
|
||
| ## Fine | ||
|
|
||
| - Logging command names, durations, counts and error codes. | ||
| - Test fixtures with fake tokens. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Check: tests | ||
|
|
||
| Apply to every change that touches code or tests. The quality bar is `.agents/skills/write-tests/references/test-rules.md`. | ||
|
|
||
| ## Report | ||
|
|
||
| - A test deleted, skipped (`.skip`, `.todo`, `xit`), or with an assertion loosened (`toEqual` → `toMatchObject`, exact arguments → `toHaveBeenCalled()`, an exact output line → `stringContaining`), without a behaviour change that explains it. | ||
| - A new snapshot test. The repository has none: tests assert exact output strings or parsed JSON. | ||
| - New behaviour with no test, or a test that would still pass with the change undone. | ||
| - A command, flag or `--json` change with no test that runs the command through `program.parseAsync`. | ||
| - A new `vi.mock` of an internal module that holds logic, or a mock of the function under test. | ||
| - An expected value computed by the code under test. | ||
| - Config that runs fewer tests: `vitest.config.ts` `include`/`exclude`, the scripts in `package.json`, the steps in `.github/workflows/test.yml`. | ||
|
|
||
| ## Fine | ||
|
|
||
| - `.not.toHaveBeenCalled()`, guards like `toBeDefined()` before reading a value, and the `getCommsClient` mock every command test uses, as the test rules describe. | ||
| - A test changing its assertions because the behaviour changed on purpose, when the description says so. | ||
| - A refactor changing import paths, names or call sites in tests. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| import { | ||
| chmodSync, | ||
| existsSync, | ||
| readFileSync, | ||
| readlinkSync, | ||
| rmSync, | ||
| symlinkSync, | ||
| writeFileSync, | ||
| } from 'node:fs' | ||
| import { basename, join } from 'node:path' | ||
| import { afterEach, beforeEach, expect, it } from 'vitest' | ||
| import { createReviewWorktree } from './review-worktree.ts' | ||
| import { createTempRepo, type TempRepo } from './test-repo.ts' | ||
| import { codeFingerprint, snapshot } from './tree.ts' | ||
|
|
||
| let repo: TempRepo | ||
| let target: string | ||
|
|
||
| beforeEach(() => { | ||
| repo = createTempRepo() | ||
| repo.write('.gitignore', 'ignored/\n') | ||
| repo.write('src/a.ts', 'a\n') | ||
| repo.write('src/gone.ts', 'gone\n') | ||
| repo.write('src/staged.ts', 'old\n') | ||
| writeFileSync(join(repo.dir, 'src/image.bin'), Buffer.from([0, 1, 2, 3])) | ||
| repo.git(['add', '.']) | ||
| repo.git(['commit', '--quiet', '-m', 'initial']) | ||
| target = join(repo.dir, '..', `${basename(repo.dir)} review`) | ||
| }) | ||
|
|
||
| afterEach(() => { | ||
| rmSync(target, { recursive: true, force: true }) | ||
| repo.cleanup() | ||
| }) | ||
|
|
||
| it('reproduces staged, unstaged, deleted, binary and untracked changes', () => { | ||
| repo.write('src/a.ts', 'unstaged edit\n') | ||
| repo.write('src/staged.ts', 'staged edit\n') | ||
| repo.git(['add', 'src/staged.ts']) | ||
| rmSync(join(repo.dir, 'src/gone.ts')) | ||
| repo.write('src/new/untracked.ts', 'untracked\n') | ||
| writeFileSync(join(repo.dir, 'src/image.bin'), Buffer.from([0, 1, 2, 255])) | ||
| repo.write('ignored/cache.txt', 'ignored\n') | ||
| const before = codeFingerprint(snapshot(repo.dir)) | ||
|
|
||
| createReviewWorktree(repo.dir, target) | ||
|
|
||
| expect(codeFingerprint(snapshot(target))).toBe(before) | ||
| expect(readFileSync(join(target, 'src/a.ts'), 'utf8')).toBe('unstaged edit\n') | ||
| expect(existsSync(join(target, 'src/gone.ts'))).toBe(false) | ||
| expect([...readFileSync(join(target, 'src/image.bin'))]).toEqual([0, 1, 2, 255]) | ||
| expect(existsSync(join(target, 'ignored/cache.txt'))).toBe(false) | ||
| }) | ||
|
|
||
| it('works when the user has diff.noprefix set', () => { | ||
| repo.git(['config', 'diff.noprefix', 'true']) | ||
| repo.write('src/a.ts', 'unstaged edit\n') | ||
| createReviewWorktree(repo.dir, target) | ||
| expect(readFileSync(join(target, 'src/a.ts'), 'utf8')).toBe('unstaged edit\n') | ||
| }) | ||
|
|
||
| it('works with a target inside the checkout, without copying it into itself', () => { | ||
| repo.write('src/new.ts', 'untracked\n') | ||
| const inside = join(repo.dir, 'review-copy') | ||
| createReviewWorktree(repo.dir, inside) | ||
| expect(readFileSync(join(inside, 'src/new.ts'), 'utf8')).toBe('untracked\n') | ||
| expect(existsSync(join(inside, 'review-copy'))).toBe(false) | ||
| }) | ||
|
|
||
| it('applies the real change when a textconv driver is configured', () => { | ||
| repo.write('.gitattributes', '*.txt diff=upper\n') | ||
| repo.write('notes.txt', 'hello\n') | ||
| repo.git(['add', '.']) | ||
| repo.git(['commit', '--quiet', '-m', 'notes']) | ||
| repo.git(['config', 'diff.upper.textconv', 'tr a-z A-Z <']) | ||
| repo.write('notes.txt', 'hello world\n') | ||
| createReviewWorktree(repo.dir, target) | ||
| expect(readFileSync(join(target, 'notes.txt'), 'utf8')).toBe('hello world\n') | ||
| }) | ||
|
|
||
| it('copies an untracked relative symlink as a relative symlink', () => { | ||
| symlinkSync('../src/a.ts', join(repo.dir, 'src/link.ts')) | ||
| createReviewWorktree(repo.dir, target) | ||
| expect(readlinkSync(join(target, 'src/link.ts'))).toBe('../src/a.ts') | ||
| }) | ||
|
|
||
| it('leaves the original checkout as it was', () => { | ||
| repo.write('src/a.ts', 'unstaged edit\n') | ||
| repo.write('src/new.ts', 'untracked\n') | ||
| const status = repo.git(['status', '--porcelain']).stdout | ||
| const before = codeFingerprint(snapshot(repo.dir)) | ||
|
|
||
| createReviewWorktree(repo.dir, target) | ||
| writeFileSync(join(target, 'src/a.ts'), 'reviewer broke it\n') | ||
|
|
||
| expect(repo.git(['status', '--porcelain']).stdout).toBe(status) | ||
| expect(codeFingerprint(snapshot(repo.dir))).toBe(before) | ||
| }) | ||
|
|
||
| it('refuses a target that already exists', () => { | ||
| expect(() => createReviewWorktree(repo.dir, repo.dir)).toThrow('already exists') | ||
| }) | ||
|
|
||
| // An unreadable file makes the copy fail, after the worktree exists. Root can read it anyway, and | ||
| // chmod cannot make a file unreadable on Windows. | ||
| it.skipIf(process.getuid?.() === 0 || process.platform === 'win32')( | ||
| 'cleans up when a step after creating the worktree fails', | ||
| () => { | ||
| repo.write('src/a.ts', 'unstaged edit\n') | ||
| repo.write('src/unreadable.ts', 'secret\n') | ||
| chmodSync(join(repo.dir, 'src/unreadable.ts'), 0o000) | ||
| expect(() => createReviewWorktree(repo.dir, target)).toThrow(/EACCES/) | ||
| expect(existsSync(target)).toBe(false) | ||
| expect(repo.git(['worktree', 'list']).stdout.trim().split('\n')).toHaveLength(1) | ||
| }, | ||
| ) | ||
|
|
||
| it('works from the command line, in a checkout whose path has a space', () => { | ||
| repo.copyFromCheckout('.agents/scripts/review-worktree.ts', '.agents/scripts/lib') | ||
| repo.git(['add', '.agents']) | ||
| repo.git(['commit', '--quiet', '-m', 'tools']) | ||
| repo.write('src/a.ts', 'unstaged edit\n') | ||
| const result = repo.run(process.execPath, ['.agents/scripts/review-worktree.ts', target]) | ||
| expect(result.stderr).toBe('') | ||
| expect(result.stdout).toContain('Review worktree ready') | ||
| expect(result.stdout).toContain(`cd '${target}' && npm ci`) | ||
| expect(readFileSync(join(target, 'src/a.ts'), 'utf8')).toBe('unstaged edit\n') | ||
| }) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| // A separate worktree with the current change in it, for a reviewer that needs to run or break | ||
| // code. It checks out HEAD, applies the staged and unstaged diff, and copies untracked files that | ||
| // are not ignored. It does not link node_modules: reinstalling through a symlinked node_modules | ||
| // would empty the original checkout's install. | ||
|
|
||
| import { cpSync, existsSync, mkdirSync, rmSync } from 'node:fs' | ||
| import { dirname, join } from 'node:path' | ||
| import { git } from './git.ts' | ||
|
|
||
| export function createReviewWorktree(root: string, target: string): void { | ||
| if (existsSync(target)) throw new Error(`${target} already exists`) | ||
|
|
||
| // Read the change before the worktree exists: a target inside the checkout would otherwise be | ||
| // listed as an untracked folder and copied into itself. Fixed prefixes and no external diff, | ||
| // textconv or colour, whatever the user's git config says, so `git apply` can read the patch. | ||
| const diff = git(root, [ | ||
| 'diff', | ||
| 'HEAD', | ||
| '--binary', | ||
| '--no-ext-diff', | ||
| '--no-textconv', | ||
| '--no-color', | ||
| '--src-prefix=a/', | ||
| '--dst-prefix=b/', | ||
| ]) | ||
| if (!diff.ok) throw new Error(`git diff failed: ${diff.stderr.trim()}`) | ||
| const untracked = git(root, ['ls-files', '-z', '--others', '--exclude-standard']) | ||
| if (!untracked.ok) throw new Error(`git ls-files failed: ${untracked.stderr.trim()}`) | ||
|
|
||
| const added = git(root, ['worktree', 'add', '--quiet', '--detach', target, 'HEAD']) | ||
| if (!added.ok) throw new Error(`git worktree add failed: ${added.stderr.trim()}`) | ||
|
|
||
| try { | ||
| if (diff.stdout) { | ||
| const applied = git(target, ['apply', '--whitespace=nowarn'], { input: diff.stdout }) | ||
| if (!applied.ok) throw new Error(`git apply failed: ${applied.stderr.trim()}`) | ||
| } | ||
| for (const path of untracked.stdout.split('\0').filter(Boolean)) { | ||
| mkdirSync(dirname(join(target, path)), { recursive: true }) | ||
| // verbatimSymlinks keeps a relative link relative, instead of pointing it back into | ||
| // the original checkout. | ||
| cpSync(join(root, path), join(target, path), { | ||
| recursive: true, | ||
| verbatimSymlinks: true, | ||
| }) | ||
| } | ||
| } catch (error) { | ||
| // Leave nothing behind, so a retry is not refused with "already exists". | ||
| git(root, ['worktree', 'remove', '--force', target]) | ||
| rmSync(target, { recursive: true, force: true }) | ||
| throw error | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| // `npm run review:worktree -- <dir>`: a separate worktree with the current change in it, for a | ||
| // reviewer that needs to run or break code. The shared checkout stays read-only for reviewers. | ||
| // See lib/review-worktree.ts. | ||
|
|
||
| import { resolve } from 'node:path' | ||
| import { CHECKOUT_ROOT } from './lib/git.ts' | ||
| import { createReviewWorktree } from './lib/review-worktree.ts' | ||
|
|
||
| const target = process.argv[2] | ||
| if (!target) { | ||
| console.error('usage: npm run review:worktree -- <dir>') | ||
| process.exit(2) | ||
| } | ||
| const dir = resolve(target) | ||
| try { | ||
| createReviewWorktree(CHECKOUT_ROOT, dir) | ||
| } catch (error) { | ||
| console.error(`review-worktree: ${(error as Error).message}`) | ||
| process.exit(1) | ||
| } | ||
| const quoted = `'${dir.replaceAll("'", "'\\''")}'` | ||
| console.log(`Review worktree ready: ${dir}`) | ||
| console.log(`Install dependencies there before running anything: cd ${quoted} && npm ci`) | ||
| console.log(`Remove it when done: git worktree remove --force ${quoted}`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.