Skip to content

Release 1.3.0-beta.0: --syncDeletes, linked content levels, list order sync, PROD-3042/3043 fixes - #254

Merged
5PK merged 34 commits into
mainfrom
release/1.3.0-beta.0
Nov 2, 2026
Merged

5PK merged 34 commits into
mainfrom
release/1.3.0-beta.0

Conversation

@5PK

@5PK 5PK commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Combines #252 and #253 into one PR for the 1.3.0 beta, adds list order sync (PROD-2992) and the fixes from #255 (PROD-3042, PROD-3043), and bumps the version to 1.3.0-beta.0. Each branch is merged in as-is (no rewritten commits), so their own descriptions and reviews still apply. It's a minor version because --syncDeletes is a new feature.

What's in it

  • Add --syncDeletes: delete on the target what was deleted in the source #252 --syncDeletes: an opt-in final phase for sync / push / reverse-sync. It deletes on the target what was deleted in the source since the last sync, for all eight entity types, in dependency order.
    • Only entities reached through a mapping are considered.
    • Each candidate is decided as delete, conflict, blocked, unmap or keep.
    • Content and page deletes are confirmed against the source API.
    • There's a mass-delete guard (--allowMassDeletes lifts it).
    • The flag is refused with --pages / --containers / --models / --models-with-deps, and it's never read from .env.
    • Planned deletes show in --preflight and --jsonSummary.
  • PROD-3025, PROD-3076: push linked content in dependency order, two-pass for reference cycles #253 PROD-3025, linked content in dependency levels: linked items are pushed in dependency levels, leaves first, one batch per level. Before, they went as one batch, so in a chain like FeaturedPost → Post → Category, Post failed on an unresolved reference to Category, which was created later in the same batch. Works for chains of any depth.
  • PROD-3025, PROD-3076: push linked content in dependency order, two-pass for reference cycles #253 PROD-3076, reference cycles: items that reference each other by content ID, or themselves, used to fail on every sync. They now get a two-pass push in the same run. Pass 1 creates them with the references into the cycle blanked. Pass 2 then sends them in full as updates, to the target items pass 1 created (trustSameRunMappings). Each item is counted and published once. A failed pass 2 is reported and pushed again on the next sync. The server accepts a required linked field saved blank, so required fields work too.
  • PROD-3025, PROD-3076: push linked content in dependency order, two-pass for reference cycles #253 page titles: a page's source title is sent as-is, including an empty one (the Fetch API returns it as null), instead of falling back to the menu text or page name. The Management API's title is optional. This replaces Fixes from the sync permutation test: PROD-3042, PROD-3043, reverse-sync deletes #255's PROD-3043 fix, which did the same for dynamic pages only.
  • PROD-2992 list order (new in this PR, on by default, no flag): after content and --syncDeletes, and before auto-publish, each synced content list is put in the source's item order on the target.
    • Only lists whose order differs from the target's pull are sent, or ones that gained an item this run. That includes a reorder-only change in the source, which never creates a new version.
    • It calls the Management API's list item-order endpoint (setContentListItemOrder, skipUnknownIds), which renumbers the staging and live list in place. No new versions, so nothing to publish.
    • Target-only items stay after the synced ones.
    • A scoped run orders only its lists, but sends every mapped item of those lists.
    • Preflight shows a List order phase, and --jsonSummary a listOrder phase. Skipped IDs are warnings; API errors are failures.
  • Fixes from the sync permutation test: PROD-3042, PROD-3043, reverse-sync deletes #255, fixes from the two-direction sync permutation test:
    • PROD-3043: a dynamic page's empty title stays empty. Superseded in the merge by PROD-3025, PROD-3076: push linked content in dependency order, two-pass for reference cycles #253's version, which covers every page; PROD-3043's static-page test now expects the same.
    • PROD-3042: a page save realigns the requires-approval setting of its components' containers and bumps their modified date. After the page push, those containers' mappings now take the new date, so the next run doesn't report false "target changed" conflicts. Containers changed on the target before the run keep their conflict.
    • PROD-2992: a test for --syncDeletes on reverse-sync, and a reverse switch on the sync harness.
  • Target-only content edits are conflicts (behaviour change): an item edited on the target since the last sync, with the source unchanged, used to be skipped as "up to date", even with --overwrite, and the target silently kept a version the source never had. It's now reported as a conflict and skipped, or replaced with the source under --overwrite, the same as pages, models and containers already did. Mappings with no recorded target version (older mapping files) are left alone. Syncs that passed before can now report conflicts where someone edited the target directly; --overwrite gives the old outcome plus the source's values. Assets and galleries now follow the same rule: assets overwrote a target-only edit without --overwrite, and galleries had the checks crossed (a both-sides edit was overwritten, a source-only change never synced, and --overwrite updated even unchanged galleries). Both now also log the conflict in a real run, not only in preflight. Templates, URL redirections and list order record no last-synced state, so they still let the source win; that's PROD-3078.
  • verify: a link to an item deleted on both sides (by --syncDeletes, which also drops the mapping) now compares equal instead of unmapped:<id> against the target ID.
  • CI: live sync test on throwaway instances, gating promotion to latest #251 CI sync permutation test is on main (with a reference-cycle fixture: A ↔ B and a self-reference through a required linked field, checked after the first sync by cycles). This branch changes it:
    • It runs only for a release, and gates the publish. It leases two instances and runs 10 to 15 minutes, so it no longer runs on PRs or nightly. A v* tag push checks the tag against package.json, runs the test on the tag's build, and publishes to npm under beta only if it passed. The promote job no longer looks up a test run, since every version published from now on has passed. Run it by hand on a branch with gh workflow run sync-permutation.yml --ref <branch> -f tier=beta.
    • The harness grades correctly. Every sync and preflight passes --syncDeletes. When a sync keeps edits the harness made on purpose (a conflict, drift), a resolve step settles them with --overwrite before verify and the no-op check. The conflict and drift checks need their own item reported. Verify and the no-op check retry up to three times 90s apart, for Fetch API lag after edits that create no version (list reorders, field renames). field-rename only renames optional fields; renaming a required one makes the server drop every item's value.

SDK

Builds against @agility/management-sdk 0.2.0-beta.3, locked in package-lock.json. Its integrity hash matches npm. That release has:

  • urlRedirectionMethods.deleteUrlRedirections, which --syncDeletes uses for URL redirections;
  • contentMethods.setContentListItemOrder, the new list item-order endpoint (in production), which the list order phase uses.

Checked on the combined branch

Sync permutation check

A local quick-tier run of this branch (fresh dev instances, rounds in both directions, the cycle fixture, list order, --syncDeletes, and the harness changes above) passed every check, 17 of 17. Each round's verify matched and each no-op preflight planned nothing, first try. The PR no longer runs the test automatically; the tag push will.

The 4 image fields in the dev blog template that pointed at another instance's CDN referenced media that no longer existed. They now use an Agility logo, and the pool's stale copies were cycled out.

Before promoting to latest

After merging: push the v1.3.0-beta.0 tag on main. The publish workflow runs the sync permutation test on it and, if it passes, publishes under beta. Then #252 and #253 can be closed as merged through here (#251 is already merged).

🤖 Generated with Claude Code

joelvarty and others added 9 commits October 30, 2026 11:29
… (PROD-3016)

Leases throwaway instances, syncs a source to a target with the CLI under
test, runs verify, checks that a second sync would change nothing, and
always deletes the instances.

- ci/leases.mjs: dependency-free lease client (create and wait, release
  by label prefix, sweep of left-behind leases), with retries.
- ci/permutation/run.mjs: the first slice of the test. Writes a results
  grid to the job summary and a JSON artifact.
- sync-permutation.yml: quick tier on same-repo PRs, beta tier on v* tags
  (tests the published package), nightly tier with a sweep, and manual
  runs. Credentials come only from a GitHub Environment by OIDC, so fork
  PRs get none. Every identifier is an Environment secret, so nothing
  about the infrastructure appears in the repo or the public logs.
- npm-publish.yml: promote to latest requires a passing tag run.
- ci/README.md: when it runs, how to trigger it, what it checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A final sync phase, opt-in per run, that deletes the target counterpart of
every mapped entity whose source is gone: URL redirections, pages (children
first), content, containers, templates, models, assets and galleries, in
dependency order. Only entities reached through a mapping are considered.

Safety rules:
- Content and pages are confirmed deleted against the source API (404 or
  state 3); a full pull also omits unpublished items, which are kept.
- An empty source list for a type is never read as "everything deleted".
- A target edited since the last sync is a conflict unless --overwrite.
- A target that something staying on the target still uses (child pages,
  linking content or pages, container items, model usage, embedded asset
  URLs, gallery assets) is kept and reported.
- Mass-delete guard: more than 50% and more than 10 of one type deletes
  nothing and fails unless --allowMassDeletes.
- Refused with --pages, --containers, --models, --models-with-deps; not
  primed from .env.

Deleted and already-gone targets drop their mapping records. --preflight
reports planned deletes with a new "delete" action; --jsonSummary gets a
syncDeletes phase and option. URL redirections are deleted through the
Management API's bulk DELETE endpoint, called directly until the CLI moves
to an SDK release with urlRedirectionMethods.deleteUrlRedirections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--syncDeletes now deletes redirections with
apiClient.urlRedirectionMethods.deleteUrlRedirections (management SDK
0.2.0-beta.3) instead of its own fetch call, and chunks by the SDK's
MAX_URL_REDIRECTIONS_PER_REQUEST. The CLI's hand-rolled delete helper and
its result type are removed.

Bumps @agility/management-sdk to ^0.2.0-beta.3. That version is not on npm
yet, so the lockfile is regenerated once it is published.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getContentItemTypes put every referenced item into one linked tier, pushed
as a single batch in reversed insertion order. In a chain A -> B -> C, B and
C landed in the same batch, and B's references were checked before C had a
target ID, so B (and A above it) failed with "Unresolved content
reference(s) not present in the target mapping".

getContentItemTypes now also returns linkedContentLevels: the linked items
split into dependency levels (Tarjan SCCs over the reference graph, so any
depth works and cycles collapse into one level). pushContent sends each
level through its own processBatches call, so every mapping a level needs
was written by an earlier one. The per-tier result bookkeeping moves into
one helper. Cycles still fail as before; they need a two-pass push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to PROD-3025-linked-content-dependency-order
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sync permutation test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 17:26 — with GitHub Actions Active
After the first sync, the test now runs rounds that alternate direction:
odd rounds edit the source and sync forward, even rounds edit the target
and reverse-sync. Every round also edits the receiving side (drift) and
one item on both sides (a conflict).

- ci/permutation/changes.mjs: change catalogue, discovered from whatever
  each instance holds. Content edit, add, add a linked chain, delete,
  publish/unpublish; list and component reordering; page edit, add,
  delete; component removal; model field add, label change and rename.
- run.mjs: per-round sync, verify and no-op checks, plus conflict and
  drift outcomes (kept-and-reported passes, silently kept fails), and
  apply failures as findings. quick = fixed two-round set; beta and
  nightly = seeded random rounds, replayable with the printed seed.
- Workflow: always installs dependencies (the change engine uses the
  Management SDK), takes a seed input, longer time budgets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 17:30 — with GitHub Actions Active
The content save can't carry list order: an update keeps the target's
itemOrder, a new item is appended when its source position is taken, and
a reorder alone in the source makes no new version. A new phase runs
after content and --syncDeletes, before auto-publish (publishing copies
each item's order into the published version), on every sync that
includes Content.

planListOrders compares each source list, mapped through the content and
container mappings, with the target's pulled order, and only lists that
differ (or gained an item this run) are sent to the Management API's
list item-order endpoint (setContentListItemOrder, skipUnknownIds), which
renumbers the staging and live list in place. Target-only items stay
after the synced ones. A scoped run orders only its lists but sends all
of their mapped items. Preflight records a 'List order' phase; skipped
IDs are warnings, API errors failures.

The fake instance gains setContentListItemOrder; full-instance tests pin
the eight push phases' counts and assert the new phase separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 17:37 — with GitHub Actions Active
@5PK 5PK changed the title Release 1.3.0-beta.0: --syncDeletes, linked content in dependency levels Release 1.3.0-beta.0: --syncDeletes, linked content in dependency levels, list order sync Oct 30, 2026
5PK and others added 2 commits October 30, 2026 13:40
A mapped item missing from the target pull made its list 'differ' on every
run, so a mapping whose target item was deleted sent that list to the API
each sync for nothing. A missing item now only counts when this run saved
it (the orchestrator passes the target content IDs it saved per locale);
otherwise the list is compared without it.

Found on a clone pair: two lists with stale mappings re-planned on every
preflight; with this the no-change preflight plans 0 lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nc test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 17:52 — with GitHub Actions Active
5PK and others added 2 commits October 30, 2026 13:54
The page payload filled a missing title with menuText, then the page
name, so a dynamic page without a title of its own got its name on the
target (e.g. 'posts-dynamic') and verify reported pageFieldMismatch on
every run. A dynamic page's title comes from dynamic.titleFormula, and
the Management API accepts it empty (checked on a clone: saved with an
empty title, stored as ''), so a dynamic page now sends its source title
as is. Other page types keep the fallback.

A target that already took the name keeps it until its source page
changes, or a sync with --overwrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving a page (UpdatePageItem) aligns the requiresApproval setting of
the non-shared containers of its components with the page's, and stamps
lastModifiedDate on each container it changes. The container pusher
compares that date with the mapping, so on the next run those containers
looked changed on the target and every following sync wanted --overwrite.

After the page push, when it saved a page, the containers are re-read
(getContainerList) and each mapping that was in sync with the target
pull at the start of the run, and has a newer target date now, takes the
new date. A container already changed on the target before the run keeps
its conflict. Bookkeeping only: an API error is a logged warning.

On a clone pair whose containers already match their pages' setting no
date moves and nothing is advanced; the two-direction permutation test
is where the reported case shows up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 17:56 — with GitHub Actions Active
A delete made on the target is propagated to the source by
reverse-sync --syncDeletes: same pair and mapping files, deletes on the
original source, mappings removed in their forward orientation. The sync
harness gains a reverse switch that runs the CLI's own enableReverseSync.
The test fails without --syncDeletes on the reverse run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live October 30, 2026 18:44 — with GitHub Actions Active
@5PK 5PK changed the title Release 1.3.0-beta.0: --syncDeletes, linked content in dependency levels, list order sync Release 1.3.0-beta.0: --syncDeletes, linked content levels, list order sync, PROD-3042/3043 fixes Oct 30, 2026
@5PK
5PK deployed to cli-dev-live October 30, 2026 18:49 — with GitHub Actions Active
@5PK
5PK deployed to cli-dev-live November 2, 2026 18:37 — with GitHub Actions Active
@5PK
5PK deployed to cli-dev-live November 2, 2026 18:43 — with GitHub Actions Active
5PK and others added 2 commits November 2, 2026 13:44
…t sync

The first sync pulls the source through the Fetch API's preview sync
feed, which lists newly published items a minute or two after the
Management API save. In CI the fixture's items were not in the feed yet,
so round 0 synced 27 items instead of 30 and the cycles check failed on
items that were never pulled. After seeding, read the same feed (with the
source's preview key) until it lists every fixture item, up to 10 minutes,
and report how long it took.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Releasing needed a `v*` tag pushed by hand after the merge. Now a push to
main whose package.json version isn't on npm (or tagged) yet builds and
publishes it under beta, creates the v<version> tag, and starts that tag's
sync permutation run (tier beta). A tag made with GITHUB_TOKEN starts no
workflows, so the job publishes and dispatches the test itself; npm
trusted publishing keeps the publish in npm-publish.yml. The promote gate
now accepts a passing run on the tag's commit started by a tag push or by
that dispatch. Merges that don't bump the version do nothing, and a tag
pushed by hand still publishes as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK deployed to cli-dev-live November 2, 2026 19:34 — with GitHub Actions Active
@5PK
5PK deployed to cli-dev-live November 2, 2026 19:40 — with GitHub Actions Active
5PK and others added 6 commits November 2, 2026 14:56
…test

The live test failed every change round although the CLI did the right
thing each time:
- The conflict and drift it creates on purpose were kept and reported by
  the sync (which the conflict and drift checks pass on), then counted as
  differences by verify and the no-op preflight, and carried into later
  rounds. When the sync keeps such edits, a `resolve` step now settles
  them with the same sync plus --overwrite, as an editor would, before
  verify and the no-op check.
- A delete never reached the other side because no sync passed
  --syncDeletes. Every sync and preflight now does, and the no-op check
  also requires zero planned deletes.
- Verify read the Fetch API about 15s after a model field rename, which
  doesn't create a content version, and saw the old value on one side. A
  verify that finds differences is tried once more a minute later.
- The conflict check only credited conflicts the pre-sync preflight saw.
  A preflight right after the sync now also counts, so a conflict that
  only the sync itself reported isn't read as silently kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… passes

The live test leases two instances and runs for 10 to 15 minutes, and it
ran on every PR, every night and every tag (after the publish). Now
sync-permutation.yml is a reusable workflow plus the manual Run workflow
button. A v* tag push checks the tag against package.json, runs the test
on the tag's build (tier beta), and publishes to npm under beta only if
it passed, so nothing gets a dist-tag untested. The promote job drops its
lookup of a passing tag run, since every version published from now on
has passed. The lease sweep that ran nightly now runs on every run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…places it

An item edited on the target since the last sync, with the source
unchanged, fell through change detection to "Entity exists and is up to
date" and was skipped silently, even with --overwrite (which only applied
when both sides changed). The target kept a version the source never had,
preflight said up to date, and verify reported a difference.

It is now a conflict, like a change on both sides and like target-only
changes to pages, models and containers already were: reported and
skipped, or replaced with the source under --overwrite. A mapping with no
recorded target version (older mapping files) is left alone, so those
don't turn every item into a conflict. The conflict warning now says the
target changed since the last sync rather than "changes detected in both".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When --syncDeletes deletes an item and its copy (and their mapping), an
item that linked to it keeps the link on both sides. Verify compared
`unmapped:<source id>` with the bare target ID and reported a mismatch.
A reference to an item that is gone from its side and has no mapping now
canonicalizes to the same `dangling` token on both sides. A link to an
unmapped item that still exists still fails, and a mapped item missing
from the pull (unpublished) keeps its ID.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…optional fields

- verify and the no-op preflight read the Fetch API, which can take a few
  minutes to show an edit that creates no version (a list reorder, a model
  field rename). When either finds something, both run again, up to three
  times 90s apart.
- The conflict and drift checks passed on any conflict in the round. Each
  now needs its own item reported as a conflict (its URL in a preflight
  conflict entry), or the value to have been overwritten.
- model.field-rename renamed required fields too. The server drops
  existing values on a rename, leaving every item missing a required
  value, which the CLI rightly refuses to sync. It now picks an optional
  Text field.

A local quick-tier run with these and the target-only conflict change
passed every check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like content, pages, models and containers, an asset or gallery changed
on the target since the last sync is now a conflict: kept and reported,
or replaced with the source under --overwrite.

- Assets: a conflict needed both sides changed, so a target-only change
  fell through to an update and the source silently replaced it.
- Galleries had the checks crossed: a change on both sides was updated
  without --overwrite, a source-only change was reported as a conflict
  and never synced, and --overwrite updated even unchanged galleries.
  Now: source only updates, target changed is a conflict, --overwrite
  updates what changed, unchanged is skipped.

Both now log the conflict in a real run and count it as skipped; before,
only preflight recorded it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@5PK
5PK merged commit f124bca into main Nov 2, 2026
3 checks passed
@5PK
5PK deleted the release/1.3.0-beta.0 branch November 2, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants