Skip to content

feat(sdk,cli): install -p claude/codex writes the live bundle's skills apart from the shipped ones, and --refresh-skills rewrites them - #1104

Merged
frontegg-david merged 3 commits into
feat/skilled-model-evalsfrom
feat/cli-install-bundle-skills
Nov 4, 2026
Merged

frontegg-david merged 3 commits into
feat/skilled-model-evalsfrom
feat/cli-install-bundle-skills

Conversation

@frontegg-david

Copy link
Copy Markdown
Contributor

Stacks on #1103 (the model-eval harness), which stacks on #1102 (the generated CLI's Skilled OpenAPI commands).

What and why

<bin> install -p claude (and -p codex) now teaches a coding agent the Skilled OpenAPI bundle's skills, so the agent can use the API through the CLI. The server's own @Skills and the bundle's skills are different things, and install now keeps them apart.

Until now, a CLI built for a Skilled OpenAPI server installed a copy of the bundle skills made at build time. That copy had no actions and no CLI usage, wasn't filtered for any caller, was never refreshed, and a skill named with a space stopped the install (Invalid plugin name "Guarded action").

  • Shipped skills, the server's @Skills, are installed from the build's metadata, as before.
  • Bundle skills are read at install time from the running server, in-process or through the daemon. The CLI reads skill://index.json and calls load_skill as the logged-in caller, so only the skills and actions that caller may use are written. A server with no bundle loaded, or a caller who isn't logged in when cli.authRequired is set, fails the install and nothing is written.
  • Each bundle skill gets a SKILL.md with:
    • frontmatter: name (the display name) and description, plus metadata with source, bundle-version and generated-by;
    • a line saying which bundle version it was generated for, and the command that refreshes it;
    • the skill's instructions;
    • an ## Actions section: each action's summary, a ready <bin> execute <actionId> --input '<json>' line built from its input schema, and its optional fields;
    • how to chain several actions with <bin> run_workflow, with an example.
  • The command lines name the CLI the way it was started for install: by its name from PATH, otherwise by absolute paths. On a server with tools of its own they use skills execute and skills run. They also say that a confirmation is given with --yes.
  • Names: a bundle skill's folder is its name in kebab-case (Guarded action becomes guarded-action). A folder that a shipped skill or another bundle skill already has gets -2, -3, and so on. A shipped skill's name is still validated as before.
  • Staleness: <bin> install --refresh-skills rewrites only the bundle skills, for each install it finds or for the -p providers given. It adds new skills and removes the folder of a skill the bundle no longer has. The shipped skills, the commands and the MCP entry are left as they are. It takes --dry-run.
  • Codex: -p codex also writes both kinds of skill to .codex/skills/<bin>/ (or ~/.codex/skills/<bin>/ with --scope user). They are tracked in a .frontmcp.json that Codex skips, and uninstall -p codex removes them.
  • --dry-run and --status list the skills by kind, with the bundle version.
  • SDK: the build-time skill asset collection (collectSkillAssets()) sets bundleVersion on a skill that a plugin registered at runtime from a bundle. The CLI build uses this to leave bundle skills out of bin-meta.json and _skills/. The SDK browser bundle sizes are unchanged: every budget passes.

User-facing change

  • What changed:
    • for a Skilled OpenAPI server, <bin> install -p claude|codex writes the bundle's skills from the running server, as the caller, with CLI usage;
    • new flag: <bin> install --refresh-skills;
    • -p codex now writes skills;
    • --dry-run and --status list skills by kind;
    • SkillAssetEntry.bundleVersion is new in the SDK.
  • Before → after:
    • Install wrote a build-time copy of the bundle skills, with no actions and for no caller, and stopped on a name with a space → install writes the caller's bundle skills from the live bundle, each with runnable execute and run_workflow lines, in kebab-case folders.
    • There was no way to refresh them → install --refresh-skills rewrites the bundle skills and removes the ones that are gone.
    • -p codex wrote only the [mcp_servers] entry → it also writes the skills.
    • --no-skills was described for -p claude only → it skips both kinds for both providers.
    • With cli.authRequired and no login, install now exits 1 with Error: could not read the bundle skills from <bin>: Not logged in. ....
  • frontmcp.dev pages affected (if known):
    • the generated CLI / --target cli reference (install section);
    • the Claude Code plugin install guide;
    • the Skilled OpenAPI plugin page (a short "through the generated CLI" note).
  • Skills catalog entries updated:
    • frontmcp-deployment/references/build-for-cli.md: a new section, "Teaching a coding agent the bundle skills";
    • frontmcp-deployment/references/mcp-client-integration.md: the per-bin commands, what gets written, the Codex skills, and folder names;
    • also updated: the CLI's README.md and the plugin's README.md.

Tests

  • Unit, libs/cli: npx nx test cli passes: 133 suites, 2226 tests. New and changed tests:
    • bundle-skills.spec.ts (14 tests): composing a bundle SKILL.md (frontmatter, the bundle-version marker and refresh line, execute lines from the input schemas, optional input, the --yes note, run_workflow chaining, skills group commands, a tool left out, no actions); reading the skills from the index and load_skill in batches of 10, skipping shipped skills, failing when no bundle is loaded; shell quoting.
    • skill-tree.spec.ts (15 tests): folder names (spaces, symbols, accents, ../, non-Latin names, length), collisions with reserved folders and between skills, and the --status and refresh summaries.
    • plugin-emitter.spec.ts (15 new tests):
      • both kinds in one plugin; collisions with a shipped skill; a reinstall that removes a skill's folder;
      • refresh adds, updates and removes only bundle skills, leaving a shipped skill untouched even when edited by hand;
      • refresh keeps the manifest fields the install wrote and those the user added; keeps a folder that holds a user file; plans without writing on dry-run; refuses when nothing is installed; ignores a tampered manifest's paths;
      • Codex skills: install, refresh and uninstall.
    • generate-cli-entry.spec.ts (2 new tests): install is wired with the CLI's Skilled commands and --refresh-skills; a server without the plugin gets no bundle skills.
  • Unit, libs/sdk: the src/direct suites pass, 21 suites and 344 tests. A new test checks that collectSkillAssets() marks a bundle skill with its bundleVersion.
  • E2E, demo-e2e-cli-exec: the new cli-install-bundle-skills.e2e.spec.ts (8 tests) uses a Skilled OpenAPI server with a @Skill of its own and a bundle skill named with a space. Its CLI runs by name from PATH. It checks that:
    • bin-meta.json lists only the shipped skill;
    • --dry-run lists both kinds and writes nothing;
    • install -p claude writes both kinds, the space-named skill goes in guarded-action/, and the admin-only skill is left out;
    • the SKILL.md's execute and run_workflow lines, run with sh -c, reach the API and print the invoice;
    • --status lists both kinds;
    • after a bundle change, --refresh-skills --dry-run plans the change, and --refresh-skills adds one skill, updates one and removes one, leaving the hand-edited shipped skill alone.
  • E2E, existing specs:
    • cli-install-claude-plugin.e2e.spec.ts: the Codex tests now use --scope user and check the Codex skills.
    • cli-login-authorities.e2e.spec.ts: 2 new tests. Install before login exits 1 and writes nothing. An admin's install writes the admin skill, and a viewer's refresh removes it. This spec runs where credentials are kept in a file, so CI on Linux; it is skipped on macOS.
    • Run together, the touched and Skilled specs pass: 5 suites (1 skipped on macOS), 70 tests passed, 6 skipped.
    • The whole demo-e2e-cli-exec suite: 42 suites passed and 1 skipped on macOS; 257 tests passed, 6 skipped.
      • cli-dev-events fails here, 3 tests. That spec covers frontmcp dev and touches nothing this PR changes.
      • Without a built @frontmcp/observability, the dev server writes no stall events.
      • With it built, the dev server can't start: the built SDK's package.json maps #js-yaml to ./src/third-party/lazy-js-yaml.ts, which dist doesn't have.
  • Bundle size: demo-e2e-bundle-size passes, 12 suites and 63 tests, with the budgets unchanged.
  • Other checks: nx run-many -t typecheck -p cli,sdk, demo-e2e-cli-exec:typecheck-e2e, tsc --noEmit for the cli and sdk libraries, prettier and eslint on the changed files, fix-unused-imports, and yarn install --immutable.

…OpenAPI bundle's skills

A CLI built for a Skilled OpenAPI server installed a copy of the bundle skills
made at build time: no actions, no CLI usage, for no particular caller, never
refreshed, and a skill name with a space stopped the install.

install now keeps two kinds of skill apart. The server's own @skills come from
the build's metadata, as before. The bundle's skills are read from the running
server (in-process or the daemon) as the logged-in caller, through
skill://index.json and load_skill, so only what that caller may use is written.
Each SKILL.md has the instructions, a ready `<bin> execute <actionId> --input`
line per action, how to chain actions with run_workflow, and the bundle version
it was generated for. A bundle skill's folder is its name in kebab-case, with a
suffix when taken.

install --refresh-skills rewrites only the bundle skills after a new bundle,
removing the folders of skills that are gone. -p codex now writes both kinds to
.codex/skills/<bin>/. --dry-run and --status list both kinds.

The SDK's build-time skill asset collection marks a skill a plugin registered
from a bundle with its bundleVersion, so the build leaves it out of bin-meta.json
and _skills/.
… bundle skills

A CLI whose server has a @skill of its own beside a bundle with a skill named
with a space: install -p claude writes both kinds, leaves out the skill the
caller may not use, and the SKILL.md execute and run_workflow lines run as
written. --refresh-skills after a bundle change rewrites only the bundle skills.
The login spec installs as an admin, then refreshes as a viewer. The Codex
install tests write the skills under their own HOME with --scope user.
@frontegg-david
frontegg-david marked this pull request as ready for review November 4, 2026 22:26
@frontegg-david

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 6 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 24c3909 and a1f4b2a.

Files ignored due to path filters: 12
  • libs/cli/src/commands/build/exec/__tests__/generate-cli-entry.spec.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/__tests__/bundle-skills.spec.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/__tests__/plugin-emitter.spec.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/__tests__/skill-tree.spec.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/bundle-skills.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/generate-cli-entry.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/plugin-emitter.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/schema-extractor.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/skill-md-compose.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/skill-tree.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/cli-runtime/skilled-commands.ts is excluded by !**/build/**
  • libs/cli/src/commands/build/exec/index.ts is excluded by !**/build/**
Files selected for processing: 12
  • apps/e2e/demo-e2e-cli-exec/e2e/cli-install-bundle-skills.e2e.spec.ts
  • apps/e2e/demo-e2e-cli-exec/e2e/cli-install-claude-plugin.e2e.spec.ts
  • apps/e2e/demo-e2e-cli-exec/e2e/cli-login-authorities.e2e.spec.ts
  • apps/e2e/demo-e2e-cli-exec/e2e/helpers/exec-cli.ts
  • apps/e2e/demo-e2e-cli-exec/e2e/helpers/skilled-cli-project.ts
  • libs/cli/README.md
  • libs/sdk/src/direct/__tests__/direct-client.spec.ts
  • libs/sdk/src/direct/client.types.ts
  • libs/sdk/src/direct/direct-client.ts
  • libs/skills/catalog/frontmcp-deployment/references/build-for-cli.md
  • libs/skills/catalog/frontmcp-deployment/references/mcp-client-integration.md
  • plugins/plugin-skilled-openapi/README.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Performance Test Results

Status: ✅ All tests passed

Summary

Project Tests Passed Warnings Failed Leaks
✅ demo-e2e-agents 4 4 0 0 0
✅ demo-e2e-cache 11 11 0 0 0
✅ demo-e2e-codecall 4 4 0 0 0
✅ demo-e2e-config 4 4 0 0 0
✅ demo-e2e-direct 3 3 0 0 0
✅ demo-e2e-elicitation 1 1 0 0 0
✅ demo-e2e-errors 4 4 0 0 0
✅ demo-e2e-hooks 3 3 0 0 0
✅ demo-e2e-multiapp 4 4 0 0 0
✅ demo-e2e-notifications 3 3 0 0 0
✅ demo-e2e-openapi 2 2 0 0 0
✅ demo-e2e-providers 4 4 0 0 0
✅ demo-e2e-public 4 4 0 0 0
✅ demo-e2e-redis 15 15 0 0 0
✅ demo-e2e-remember 4 4 0 0 0
✅ demo-e2e-remote 5 5 0 0 0
✅ demo-e2e-serverless 2 2 0 0 0
✅ demo-e2e-skills 15 15 0 0 0
✅ demo-e2e-standalone 2 2 0 0 0
✅ demo-e2e-transport-recreation 3 3 0 0 0
✅ demo-e2e-ui 4 4 0 0 0

Total: 101 tests across 21 projects

📊 View full report in workflow run


Generated at: 2026-11-04T22:41:44.523Z
Commit: bd4ed5de

@frontegg-david
frontegg-david merged commit 7c79dc8 into release/1.11.x Nov 4, 2026
54 of 74 checks passed
@frontegg-david
frontegg-david deleted the feat/cli-install-bundle-skills branch November 4, 2026 23:23
frontegg-david added a commit that referenced this pull request Nov 4, 2026
…undle's skills apart from the shipped ones, and --refresh-skills rewrites them (#1104)

Cherry-picked from #1104 (merged to release/1.11.x)
Original commit: 7c79dc8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants