The CLI has no tests: three open bugs all live in cli.py

[royalpinto007]

Problem

agentrace/cli.py has zero test coverage. tests/test_agentrace.py has 17 tests and every one of them targets parse_session or analyse directly. Nothing imports agentrace.cli.

The only thing exercising the CLI is two lines in .github/workflows/ci.yml:

agentrace --file tests/fixtures/session.jsonl list
agentrace --file tests/fixtures/session.jsonl stats --json

That checks two of the four commands exit zero. It does not check what they printed, and check and show are not run at all.

The consequences are visible in the open issue list. #5 (check runs every check twice per run, and --strict ignores --severity), #8 (--file and --dir are not mutually exclusive), and #9 (show picks the first match when a short id is ambiguous) are all bugs in cli.py, and all three would have been caught by an ordinary CLI test. The double-running of analyse in cmd_check is right there in the source: once in the loop, then again in any(f.severity == "high" for r in runs for f in analyse(r)).

Why it matters

This is a CLI. The command surface is the product. Everything under it is well tested and the part users actually touch is not tested at all, which is exactly backwards from where the bugs are.

Suggested approach

1. Add tests/test_cli.py and drive main(argv) directly. It already takes argv: list[str] | None and returns an int, so it is testable without subprocesses.
2. Use pytest's capsys for stdout assertions. Rich writes to stdout via Console, so consider making the module level console injectable or constructing it with an explicit file so tests can capture it deterministically, and set a fixed width so table wrapping does not make assertions brittle.
3. Cover per command:
   • list: run count in the title, one row per run, empty transcript prints the no-runs message and exits 0.
   • check: exit code 0 without --strict, exit code 1 with --strict and a high severity finding, exit 0 with --strict when --severity low filters the high ones out (this is check runs every check twice per run, and --strict ignores --severity #5), and that each check runs once.
   • show: exact id match, 8 char suffix match, no match returns 1, ambiguous suffix (this is show picks the first match when a short id is ambiguous #9).
   • stats: totals against the fixture, and --json parses as JSON.
4. Add argument level cases: --file with --dir (this is --file and --dir are not mutually exclusive, so --dir is silently ignored #8), a nonexistent path, and a directory passed to --file.
5. Extend the fixture, or add a second one, so it contains an error run, an empty result, and two runs whose ids share a suffix.

Done when

• Every subcommand and every flag has at least one test.
• The exit codes are asserted, not assumed.
• The bugs in check runs every check twice per run, and --strict ignores --severity #5, --file and --dir are not mutually exclusive, so --dir is silently ignored #8, and show picks the first match when a short id is ambiguous #9 each have a test that fails today.

You do not have to fix those three bugs in the same PR. Landing the failing tests as xfail and fixing them separately is fine, and it makes the fixes easy to review.

---

If you want to take this on, comment on the issue to claim it and it will be assigned. Please keep to a maximum of 2 open claims per person at a time so other contributors get a chance.