Skip to content

feat: collections API and verified CLI installer with lockfile, sync and local MCP (#35 phase 3) - #98

Merged
mrgoonie merged 9 commits into
mainfrom
sx-phase3-install
Oct 20, 2026
Merged

mrgoonie merged 9 commits into
mainfrom
sx-phase3-install

Conversation

@mrgoonie

@mrgoonie mrgoonie commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Refs #35

Phase 3 of #35. It adds a collections web API and a verified CLI installer: the resolver, the skillx.lock.json lockfile, atomic apply, receipts, sync --locked and a local stdio MCP. Personal kits (#55) are reused as the kit source. The kit lock endpoint, overlays and kitTreeDigest are consumed as they are, with no new kit endpoints. The design is in plans/261019-1914-issues-31-88-delivery/phase3-design.md.

Checklist → evidence

#35 item Implementation Tests
Collection create/update/clone/delete apps/web/app/lib/collections/*, routes/api.v1.collections.ts apps/web/tests/collections/collections-api.test.ts
Visibility and immutable revisions private/unlisted/public; collection_revisions_immutable trigger; revisionDigest (contracts collection-revision.ts) same file (immutable digested revision 1, non-owner 404, unlisted visible)
Add/remove package members POST /:id/members (add[], remove[], CAS expectedRevision) same file
Resolve compatibility and dependencies cli/src/lib/install/resolver.ts, release-pins.ts, fetch-verify.ts (skillx.json compatible, kit runtimes). Releases declare no dependency graph yet, so the resolver detects alias collisions and shared versions install-units.test.ts, install-e2e.test.ts
Lock state with exact version/digest contracts install-lock.ts (skillx.lock/v1, validator, lockDigest); lock-file.ts contracts/src/install-lock.test.ts, install-units.test.ts
Install plan planner.ts, install-operations.ts (planDigest) install-units.test.ts, MCP test
Fetch/verify artifact fetch-verify.ts: signed manifest against trusted keys, identity/version/digest, per-file size + SHA-256, overlay digests, composition digest e2e "tampered file / untrusted key installs nothing"
First harness adapter adapters.ts, Claude Code ($CLAUDE_CONFIG_DIR/skills or ~/.claude/skills) e2e, units
Second harness adapter Codex ($CODEX_HOME/skills or ~/.codex/skills) e2e machine C, collection test, MCP test
Local install receipts install-state.ts (skillx.install-receipt/v1) e2e, recovery
Owned paths/registrations receipt files[] + owners record (skillx.install-owners/v1) e2e uninstall
Update planning/apply updateInstalled, update action with backup recovery (update at every crash point), e2e force
Uninstall with ownership proof remove-owned.ts deletes only regular files under a link-free path whose hash matches the receipt e2e uninstall, install-safety.test.ts (junction/symlink)
Preserve modified/unknown files conflict unless --force; force backs up edits and keeps unknown files e2e drift + uninstall (keptModified, keptUnknown)
Ref-count shared packages receipt requiredBy e2e "ref-counts shared skills"
skillx sync --locked sync-locked.ts, commands/install.ts e2e machine A → kit revision → machine B identical digests; drift refused
Exclude secrets/absolute paths from sync contracts sync-leaks.ts (lock write/read, collection content) sync-leaks.test.ts, install-units.test.ts, collections 422
Local stdio MCP inspect/plan/apply local-mcp.ts, skillx mcp; apply requires confirm: true and planDigest; no force over MCP install-recovery.test.ts "local MCP", install-hardening.test.ts
Interruption/retry tests apply-transaction.ts journal and recovery; CrashPoint hooks install-recovery.test.ts: 5 crash points × (update, first install), rerun converges

All installer tests run against the in-memory server (real loaders, FakeD1, FakeR2) with temp HOME/USERPROFILE/CODEX_HOME/SKILLX_HOME. None of them touches real user state.

Review fixes (isolated review of this PR)

Item Fix Tests
B1 uninstall follows links remove-owned.ts + safe-fs.ts: the skill root and every parent must be a real directory (lstat), files must be regular; links are never descended, a linked skill dir is left alone install-safety.test.ts: junction subdir, junction skill root, file symlink
B2 files changed between plan and swap apply-transaction.ts: after staging, re-prove the target against the receipt, stage and backup (tree-proof.ts); drift → conflict (local_modified), nothing swapped. Files appearing after the check are copied to ~/.skillx/backups/... and reported as displaced, never rmSync'd install-hardening.test.ts "files changed between plan and swap"
M1 planner ignores unknown: ['.'] planner.ts: link/file skill path is unowned_directory in every branch, even with --force install-safety.test.ts
M2 overlay traversal fetch-verify.ts calls validateOverlay and checks every manifest path; apply-transaction.ts writes only via containedPath install-safety.test.ts (../../evil.txt, absolute, drive, \)
M3 rename EPERM/EBUSY renameWithRetry with backoff; on failure roll back via recoverJournals, report conflict install-hardening.test.ts (busy twice → ok; always busy → old tree whole, no journal/stage)
1 collection member drift sync-locked.ts memberDrift install-safety.test.ts
2 MCP apply requires planDigest; force → usage_error install-hardening.test.ts
3 install lock race pid file hard-linked to install.lock (atomic); stale takeover via rename + re-check install-hardening.test.ts
4 lockfile scope one scope per lock: package installs merge by alias, kit/collection replace, other scope → usage_error; update refreshes, uninstall drops (--lock/--no-lock) install-safety.test.ts
5 alias isInstallAlias: no trailing ., no Windows device names install-safety.test.ts
6 API key over http assertSecureRegistry: https, or http only to loopback/*.localhost install-safety.test.ts
7 orphaned owners recover drops requiredBy owners missing from the owners record, removes skills left with none install-hardening.test.ts
Nits malformed skillx.json → incompatible; skillx.json checked before other files are fetched; migration renumbered to 0019 on top of #99's 0018 snapshot install-hardening.test.ts

Re-review fixes (head 2572985)

Item Fix Tests
MAJOR rollback loses the old tree when the target reappears journal-recovery.ts (recovery moved out of apply-transaction.ts): when the stage exists and something else now sits at the target, the .old tree is parked in ~/.skillx/backups/<runtime>/<alias>/<stamp>-previous-tree and reported in parked (also in the conflict details from a refused rename); an unjournaled .old in staging is parked, never swept install-hardening.test.ts "parks the old tree in the backups when something else takes its place" (rename hook creates demo/editor.tmp, then EPERM)
1 links inside the skill dir planner: a symlink/junction inside the skill dir is a contains_link conflict, even with --force, when the directory would be rewritten (see round 3); copyInto refuses non-regular files; retireOld removes a late link itself (target saved as <path>.link), never follows it install-hardening.test.ts "links inside a skill directory" (forced install refused; late junction removed, outside file intact, no stuck journal)
2 owner key mismatch one owner per package (packages:<name>) for multi-package installs and packages-lock syncs; uninstall/update accept several packages; "one package at a time" error removed install-e2e.test.ts "gives each package of a multi-package install or a packages-lock sync its own owner"
3 empty lock read an empty install.lock read after EEXIST retries instead of reporting busy covered by existing lock tests
Nits cli-reference.md: a missing owners file during recovery removes receipted skills (a corrupt one stops the run); uninstalling a kit/collection leaves its lock file as is —

Round-3 fixes (head 19b3966)

Item Fix Tests
1 park() copied junctions as real directories park renames the old tree into the backups (links kept as they are); across volumes, or when the rename stays refused, it falls back to retireOld, which copies regular files and records links as <path>.link without following or recreating them, so no symlink creation is needed and no journal is left stuck install-hardening.test.ts "parks across volumes without following a junction in the old tree" (EXDEV forced; savedTo holds refs.link, not the junction target's files)
2 contains_link before unchanged the link check runs only when the action would rewrite the directory (install/update); same-version reinstall, sync --locked and a new owner keep working install-hardening.test.ts "are kept while nothing is rewritten, and block an update even with --force"
3 partial multi-package uninstall uninstallRef(..., onUninstalled) runs after each owner is removed; the command drops that package from the lock right then install-e2e.test.ts "keeps the packages lock in step when a multi-package uninstall stops part way"
4 duplicate owners ownersOfRefs dedupes install-e2e.test.ts (uninstall @alpha/demo @alpha/tools @alpha/demo)
5 empty install.lock retry only on ENOENT; an empty lock keeps the "remove " hint install-hardening.test.ts lock test (empty holder)

Migrations and flags

  • 0019_collection_revision_lineage.sql is additive: two nullable columns plus the immutability trigger, with rollback SQL in its header. It was applied only to local D1.
  • SKILLX_COLLECTIONS_API is "0" in wrangler.jsonc, so the collections endpoints answer 404 until it is enabled.

Notes for reviewers

Verification

  • pnpm typecheck ✓
  • pnpm --filter web exec drizzle-kit check ✓, drizzle-kit generate: no schema changes ✓
  • pnpm test (on main + this branch): 111 files, 2210 passed, 2 skipped ✓ (installer integration tests get a 30s per-test timeout; one full run hit the 5s default under load)
  • pnpm --filter web build ✓
  • pnpm --filter skillx-sh build ✓
  • Built-CLI smoke against a temp home ✓
    • --help lists install/update/uninstall/list/sync/mcp.
    • list --json --runtime codex returns an empty envelope.
    • sync without --locked and an invalid ref both exit 2.
    • mcp answers tools/list.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor Author

This affects the verified CLI installer here. #100 adds skillx.release/v2, which is v1 plus a closed provenance member, and SkillX signs only @agentkit/* kit releases with it. verifyReleaseManifest in @skillx/contracts accepts both versions.

An installer built on it should do three things:

  • Read the provider with releaseProvenanceOf.
  • Compute trust with releaseTrust. That gives local-unverified unless a kit signature was verified separately.
  • Never map a registry signature to signed-kit.

Both PRs touch packages/contracts/src/index.ts. Whichever lands second should keep both export sets.


Generated by Claude Code

mrgoonie and others added 9 commits October 19, 2026 21:26
…d members

Add the flag-gated /api/v1/collections API (SKILLX_COLLECTIONS_API) over the
existing collections tables: create, metadata update, delete, clone with
lineage, full-replace revisions and member add/remove as compare-and-set
revisions. Revisions are digested and immutable (trigger), and content passes
a shared sync-leak scanner that refuses secrets and absolute local paths.
Contracts gain the skillx.lock/v1 lockfile schema used by the installer.

Refs #35

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… MCP

Adds skillx install, update, uninstall, list, sync --locked and mcp. Sources
(kit revision, collection revision or packages) resolve to exact versions and
digests in a skillx.lock/v1 lock; every manifest signature and file SHA-256 is
verified before a staged, journaled rename into the Claude Code or Codex skills
directory. Receipts prove ownership, so uninstall keeps edited and unknown
files and shared skills are ref-counted. Interrupted runs recover on rerun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ons API

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry publishing migration takes 0018 and merges first; the
collection revision lineage migration and its snapshot now follow it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d during install

- uninstall deletes only regular files under a link-free path; a linked
  skill directory or subdirectory is left alone
- re-check the target after staging: new or edited files are a conflict;
  files that appear after the check are saved to backups, not deleted
- retry renames refused by Windows, then roll back and report a conflict
- validate manifest and overlay paths and write only inside the target
- a skill path that is a link or file is always unowned_directory
- check skillx.json compatibility before fetching other files and refuse
  malformed declarations
- collection locks must pin exactly the revision members
- MCP apply requires planDigest and rejects force
- install lock is created atomically and taken over only from a dead pid
- lock file keeps one scope: package installs merge, update refreshes it,
  uninstall drops the package
- aliases reject a trailing dot and Windows device names
- API keys are sent only over https (http only to loopback)
- recovery drops owners no source records and removes unowned skills

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- recovery parks the old tree in the backups when the skill path has been
  taken by something else, instead of letting the staging sweep delete it;
  an unjournaled old tree in staging is parked the same way
- a symlink or junction inside a skill directory is a contains_link
  conflict, even with --force; copying refuses non-regular files
- retiring the old tree removes a late link itself and records its target,
  never following it
- explicit packages are owned one per package for multi-package installs
  and packages-lock syncs; uninstall and update accept several packages
- an install lock released between attempts is retried, not reported busy
- recovery moves into journal-recovery.ts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ewritten

- parking an old tree renames it into the backups; across volumes, or when
  the rename stays refused, regular files are copied and links recorded as
  <path>.link, so a junction is never copied as a directory
- a link inside a skill directory is a conflict only for an install or
  update that rewrites it; same-version reinstalls, sync and new owners
  keep working
- a multi-package uninstall drops each package from the lock right after
  it is removed; repeated packages are uninstalled once
- only a vanished install.lock is retried; an empty one keeps the
  "remove <path>" hint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant