[Hackathon] feat(setup): clarify onboarding outcomes and agent installation choices #656

Verified against (commit SHA or released version)

CLI v0.15.0, commit d61e6dfc400f5e88ee34ac799c5496ac5c5a7751. Reproduced locally on Linux with Node v22.23.3, using mocked account responses and temporary directories. The original user report came from PowerShell; native Windows verification remains part of implementation testing.

Discord Username / User ID

sanaro99

What does this improvement do?

Make the human setup step easier to complete before handing TestSprite to a coding agent. A person entering an API key and choosing agent integrations needs to know what succeeded, what still needs attention, and what to do next.

Currently, testsprite setup mixes account advisories, repeated file-conflict messages, and recovery instructions. If some skill files already exist with different contents, credentials can be saved and other skills installed successfully, but setup exits with an error without showing those installation results.

In a reproduced five-agent setup, three existing files conflicted: two for Claude and one for Cursor. Antigravity, Kiro, and Codex installed successfully; additional Claude/Cursor skills were also written. The final output was:

[info] credentials saved for profile "default"; only the agent skill install failed — re-run 'testsprite agent install --target claude,antigravity,cursor,kiro,codex' after fixing the path
Error: one or more targets already exist and differ; re-run with --force to overwrite (a .bak is kept).

The paths were valid. Repeating the suggested install command without resolving the conflicting files produces the same refusal. Successful installations are difficult to distinguish from the blocked ones.

This supports the repository's agent-first direction: improve the human step that prepares credentials and skills for the agent. VISION.md includes CLI ergonomics and onboarding in scope.

Details / implementation notes

Proposed scope: setup and its skill-installation feedback.

1. Show the destination before selection. Display the resolved project directory. Setup defaults to the current directory, which can be a home folder; keep the existing --dir behavior.
2. Make agent selection interactive on supported terminals. Arrow keys navigate, Space toggles an agent, A toggles all, and Enter confirms. Show detected agents as the initial selection and allow skipping installation. Explain which bundled skills will be installed.
3. Always summarize completed and blocked steps. Report credential persistence separately from per-agent skill outcomes, including partial installation. Show one immediate next action and group file details below it.
4. Make conflicts a clear decision. Explain that existing files differ and have been preserved. Offer keeping them or explicitly replacing them with backups. Kept or unresolved files must remain visible rather than being reported as successfully updated. Recovery instructions should target the affected agents.
5. Use restrained, consistent styling. Success, required action, and blocking failure should have distinct labels and optional color. Separate informational account notes from actions needed to finish setup; keep the longer onboarding walkthrough in help/documentation.

Illustrative summary for the reproduced case:

Setup needs attention

✓ API key saved · profile: default
✓ Antigravity, Kiro, Codex ready
! Claude: 2 existing files differ
! Cursor: 1 existing file differs

The 3 conflicting files were kept.
Next: review them, then keep or replace with backups.

Integration and compatibility: Keep Commander 12 and the existing output/prompt abstractions. Preserve noninteractive flags, JSON contracts, and exit behavior; keep interactive UI on stderr. Respect terminal capability and the existing NO_COLOR policy. No backend changes or whole-CLI redesign are needed.

Basic styling can use a shared adapter around Node's formatter. A checkbox library is a separate runtime-dependency decision: Inquirer-related packages currently appear only through development tooling. Please confirm an acceptable dependency/approach during triage before implementation.

Validation completed: The current setup behavior was reproduced using the built command logic with injected HTTP responses and real temporary file writes. Six existing setup/installer/output/prompt/error/ticker test files passed, totaling 312 tests. Additional Linux terminal probes checked masked input, backspace, cancellation/raw-mode restoration, and clean JSON/help through pipes. These establish the existing behavior; they do not validate the proposed UI, which has not been implemented.

Implementation acceptance: Cover partial installation, correct destination and next action, agent selection/cancellation, explicit replacement with backups, supported-terminal and plain fallbacks, JSON/stdout purity, and Windows PowerShell. Keep informative labels usable without color.

Related work: Coordinate with #271, which changes skill installation layout, and #615, which fixes prompt escape-sequence handling. #464 already covers CRLF-related false conflicts; this proposal concerns communicating and resolving installation outcomes, regardless of why files differ. Targeted issue/PR searches on 2026-10-23 found no matching setup presentation/selection proposal.

AI assistance: Codex assisted with source inspection, local reproduction, focused testing, and drafting this proposal.

Confirmations

• I have searched existing issues and found no matching proposal.
• I have provided my Discord identity above for reward coordination.