Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions .github/workflows/sync-permutation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ on:
description: "Test this published @agility/cli version instead of building this branch"
required: false
default: ""
seed:
description: "Replay a run's random choices (printed in its summary); blank = new seed"
required: false
default: ""

concurrency:
group: sync-permutation-${{ github.event.pull_request.number || github.ref }}-${{ github.event_name }}
Expand Down Expand Up @@ -97,15 +101,15 @@ jobs:
- name: Lease source and target
id: lease
run: |
case "$TIER" in quick) TTL=60 ;; beta) TTL=120 ;; *) TTL=240 ;; esac
case "$TIER" in quick) TTL=90 ;; beta) TTL=150 ;; *) TTL=240 ;; esac
node ci/leases.mjs create "$LEASE_PREFIX" source:blog target:blank --ttl "$TTL" --timeout 900

test:
name: Sync and verify
needs: lease
runs-on: ubuntu-latest
environment: cli-dev-live
timeout-minutes: 50
timeout-minutes: ${{ (github.event_name == 'schedule' || inputs.tier == 'nightly') && 180 || 120 }}
steps:
- uses: actions/checkout@v4

Expand All @@ -115,10 +119,13 @@ jobs:
cache: "npm"

# The CLI under test: this checkout, built; or a published version from npm.
# Always installed: the change engine (ci/permutation/changes.mjs) uses the Management SDK.
- name: Install dependencies
run: npm ci

- name: Build this branch
if: env.CLI_VERSION == ''
run: |
npm ci
npm run build
echo "CLI=node $GITHUB_WORKSPACE/dist/index.js" >> "$GITHUB_ENV"

Expand Down Expand Up @@ -152,6 +159,7 @@ jobs:
SOURCE_GUID: ${{ needs.lease.outputs.source_guid }}
TARGET_GUID: ${{ needs.lease.outputs.target_guid }}
OUT_DIR: ${{ runner.temp }}/permutation-out
SEED: ${{ inputs.seed }}
run: node ci/permutation/run.mjs

# Only the JSON results: anyone can download a public repo's artifacts, so the CLI's raw
Expand Down
42 changes: 33 additions & 9 deletions ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,39 @@ gh run rerun <run-id> # re-ru

## What it checks

Today (the first slice):

- **First sync:** `sync` exits 0 and its `--jsonSummary` reports no failed items.
- **verify:** `agility verify` reports that the target matches the source (exit 0).
- **Nothing left to sync:** an immediate `sync --preflight` plans no creates, updates or conflicts.

Each check is a row in the run's job summary. The JSON results are kept as an artifact for 30 days.
Later slices add generated sources, rounds of changes on both sides, more target types, and
reverse, round-trip and chained syncs.
A run first syncs source **A** to target **B**, then runs rounds of changes in alternating
directions:

- **Odd rounds** edit A and `sync` A → B.
- **Even rounds** edit B and `reverse-sync` B → A.
- **Every round** also edits the other instance (drift) and one item on both sides (a conflict).

The changes, in `ci/permutation/changes.mjs`, are discovered from whatever each instance holds:

| Area | Changes |
| ---------- | -------------------------------------------------------------------------------------------------- |
| Content | edit, add, add a linked item together with an item that links to it, delete, publish and unpublish |
| Ordering | swap two items in a list; reverse the components in a page zone |
| Pages | edit title and menu text, add a page, delete it later |
| Components | remove one from a zone |
| Models | add a field, change a label, rename a field |

The `quick` tier runs a fixed set covering every area in two rounds. `beta` (4 rounds) and `nightly`
(6 rounds) pick changes at random from a seed printed in the summary. Replay one with
`gh workflow run sync-permutation.yml -f tier=nightly -f seed=<seed>`.

After every sync it checks:

- **sync:** exit 0 and no failed items in `--jsonSummary`;
- **verify:** `agility verify` finds no differences;
- **noop:** a `--preflight` in the same direction plans no creates, updates or conflicts;
- **conflict:** the item edited on both sides ends at the synced side's value, or the sync reported a
conflict;
- **drift:** an edit made only on the receiving side is overwritten, or reported, never silently kept;
- **apply:** the change itself could be made.

Nothing stops a run early: every round runs, and each check is a row in the job summary. The JSON
results are kept as an artifact for 30 days.

## Promotion to `latest`

Expand Down
Loading
Loading