Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ All notable changes to `@testsprite/testsprite-cli` are documented here. The for

## [Unreleased]

### Added

- `test export <test-id> [--out <file>]` and `test import <file>` put individual backend test definitions under version control. Exports combine the existing test metadata and code API shapes with deterministic JSON and saved code-version provenance. Imports create by default; `--test-id` updates code with optimistic concurrency before metadata. Frontend definitions are explicitly unsupported in this first version.

## [0.15.0] - 2026-10-17

### Added
Expand Down
20 changes: 20 additions & 0 deletions DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,26 @@ testsprite test create --plan-from ./checkout.plan.json --dry-run --output json

`--plan-from` owns the complete test definition and does not support this setting; a supplied `--step-timeout` is warned about and ignored. Set it later with `test update --step-timeout <ms>`.

#### `testsprite test export <test-id>` / `testsprite test import <file>`

Export one **backend** test definition for version control, review, backup, or migration. The JSON object contains `test` (the supported definition fields projected from `GET /tests/{id}`) and `code` (the `GET /tests/{id}/code` object with inline Python source). Keys are sorted recursively, arrays keep their order, and source whitespace is preserved. A presigned code URL is downloaded without API authentication headers before export, so the artifact contains durable source. There is no separate file-format version.

```bash
testsprite test export test_backend --out ./login.testsprite.json
testsprite test export test_backend > ./login.testsprite.json
testsprite test import ./login.testsprite.json # create a copy
testsprite test import ./login.testsprite.json --project project_new # create in another project
testsprite test import ./login.testsprite.json --test-id test_backend --idempotency-key restore-login
```

`test` includes `id`, `projectId`, `type`, `name`, and any returned `description`, `priority`, `stepTimeoutMs`, `produces`, `consumes`, or `category`. Status, run results, timestamps, project display names, and debug details are outside the definition projection. `test.id`, `code.testId`, `code.codeVersion`, and `code.etag` record provenance; creating a copy assigns a new identity and version. Null optional metadata is unset on create. Updating patches the fields present in the file, including explicit null clears; omitted fields remain unchanged. Unknown import fields are rejected before any API request rather than discarded. Frontend export and import fail explicitly: this first version does not restore frontend plans or generated code.

An import creates by default, even if the exported `test.id` is present. `--test-id` explicitly selects update, must identify a backend test in the original project, and uses the saved `code.codeVersion` as `If-Match`. A null, empty, or wildcard version cannot silently force an update. `--expected-version <version>` supplies a reviewed target version and is required when the target differs from the saved `test.id`. `--project` applies only to creation. Definition input must be a regular UTF-8 JSON file of at most 1 MB (an optional BOM is accepted); Python/pytest code must be non-empty and at most 350 KB. Unsupported legacy code languages must be converted before import.

Updates write code first, so a stale version fails with `PRECONDITION_FAILED` (exit 6) before metadata changes. Code and metadata remain two separate API writes. If metadata fails after code succeeds, the CLI reports the partial update on stderr and preserves the underlying error. Retry with the same `--idempotency-key` to replay completed stages safely; changing the key or expected version starts a different operation. The base key is printed on stderr for JSON, verbose, or debug output and accepts at most 247 ASCII characters to leave space for stage suffixes. Create uses `:create`; update uses `:code` and `:metadata`.

Export prints the JSON definition to stdout regardless of display mode unless `--out` is supplied. `--out` uses the code-download path's sibling temporary file and atomic rename: an existing regular file is replaced only after export completes, and failures preserve its bytes. Dry-run export prints a sample without reading the API or writing a file. Dry-run import validates the supplied file and previews mutation responses without credentials or network calls. Definition export is separate from JUnit **result** reports and does not define suites or Test Lists.

#### `testsprite test create-batch`

Bulk-create frontend tests from a JSONL plan-steps file (or a directory of plan files with `--plan-from-dir`). Optional `--run --max-concurrency <N>` fans out triggers. Without `--wait`, each run is dispatched (`status: "queued"`) and the command exits 0 when every trigger is accepted — mirroring single `test run` without `--wait`; a trigger error still exits non-zero. With `--wait`, it polls every run to terminal and exits non-zero if any run does not pass.
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,8 @@ On an owned tunnel, Ctrl-C or a timeout cancels unfinished runs by default, and

📚 **Full reference — every command, flag, and example:** [DOCUMENTATION.md](./DOCUMENTATION.md), including [configuration & profiles](./DOCUMENTATION.md#configuration), [scripting](./DOCUMENTATION.md#output--scripting), and [exit codes](./DOCUMENTATION.md#exit-codes).

Backend definitions can be stored in version control with `testsprite test export <test-id> --out test.testsprite.json` and restored with `testsprite test import test.testsprite.json`. Import creates a copy by default; add `--test-id <id>` to update using the saved code version. See the [definition export/import reference](DOCUMENTATION.md#testsprite-test-export-test-id--testsprite-test-import-file) for supported fields, concurrency, and retry behavior.

## Why a CLI for coding agents?

- 🧪 **Tests like a real user.** Runs against a live browser or API in the cloud — real clicks, real navigation, real assertions. Not a mock.
Expand Down
Loading
Loading