Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## v6.2.2

- The README tells how to move a pipeline from the `steadybit/run-experiment` action,
which is being deprecated, to `experiment run`.
- `execution property set` with a single `--value` sets a list property, such as a
`STRING_LIST`, to a list of that one value; the platform refused it with "must be a
list of strings". Found by the platform test, which now covers templates, run
Expand All @@ -18,7 +20,7 @@
## v6.2.0

- `experiment run` can do what the `steadybit/run-experiment` GitHub Action does, so that
the action can run on the CLI:
pipelines using the action can move to the CLI:
- `--expect-state` passes once the run reaches a state, which need not be its end, such
as `FAILED` for an experiment expected to find a weakness, or `RUNNING`, and fails when
it ends in another; `--expect-reason` also requires the run's reason.
Expand Down
47 changes: 47 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,53 @@ In GitHub Actions a summary of every run is added to the job summary.
report_paths: steadybit.xml
```

### Moving from `steadybit/run-experiment`

The `steadybit/run-experiment` action is being deprecated in favor of this CLI, which
does what it does and more: it cancels the attack when the job is canceled, runs several
experiments at once, writes a JUnit report, and works the same in any CI. A step using
the action becomes:

```yaml
# Before
- uses: steadybit/run-experiment@v1
with:
apiAccessToken: ${{ secrets.STEADYBIT_TOKEN }}
experimentKey: ADM-1
expectedState: FAILED

# After
- uses: steadybit/cli@v6
- run: steadybit experiment run -k ADM-1 --yes --busy-retries 3 --expect-state FAILED
env:
STEADYBIT_TOKEN: ${{ secrets.STEADYBIT_TOKEN }}
```

| Action input | `experiment run` |
| ----------------------------------------- | ----------------------------------------------------------- |
| `apiAccessToken` | the `STEADYBIT_TOKEN` variable |
| `baseURL` | the `STEADYBIT_URL` variable, for an on-premise platform |
| `experimentKey` | `-k ADM-1` |
| `externalId` | `--external-id shop-latency` |
| `expectedState` | `--expect-state FAILED` |
| `expectedReason`, `expectedFailureReason` | `--expect-reason "..."` |
| `parallel: true` | `--allowParallel` |
| `maxRetries` (3 by default) | `--busy-retries 3`; the CLI does not retry unless asked |
| `maxRetriesOnExpectationFailure` | `--expectation-retries 2` |
| `delayBetweenRetriesOnExpectationFailure` | `--expectation-retry-interval 1m` (a duration, not ms) |
| `maxRetriesOnValidationFailure` | `--retries 2` |
| `delayBetweenRetriesOnValidationFailure` | `--retryInterval 15` (seconds) |

The action's outputs are in the JSON report: `--report run.json` writes each run's `id`,
`state`, `reason` and `apiLocation` (the action's `executionUrl`).

```yaml
- id: chaos
run: |
steadybit experiment run -k ADM-1 --yes --report run.json
echo "state=$(jq -r '.[0].state' run.json)" >> "$GITHUB_OUTPUT"
```

### GitLab CI

```yaml
Expand Down
5 changes: 3 additions & 2 deletions e2e/platform.sh
Original file line number Diff line number Diff line change
Expand Up @@ -198,8 +198,9 @@ until_run_is "$LONG" RUNNING
check "--no-wait fails on a run the platform refused" exits_with 1 steadybit experiment run -k "$A" --yes --no-wait
check "execution list --fail-on-match gates on that canceled run" exits_with 1 \
steadybit execution list --key "$A" --state CANCELED --ended-from "$(date -u +%F)" --limit 1 --fail-on-match
# What the run-experiment action relies on: the experiment found by its external id, an
# expected state reached before the end, and waiting while another experiment runs.
# What pipelines moving from the run-experiment action rely on: the experiment found by
# its external id, an expected state reached before the end, and waiting while another
# experiment runs.
check "the experiment is found by its external id and passes at the expected state" exits_with 0 \
steadybit experiment run --external-id "$MARK-$RUN-a" --yes --allowParallel --expect-state RUNNING --report expect.json
check "the report has the state reached and the run's API location" sh -c '
Expand Down
Loading