Skip to content

feat: make the toolset work in any coding agent, not just Claude Code - #47

Merged
travist merged 11 commits into
mainfrom
feat/multi-agent-portability
Aug 17, 2026
Merged

feat: make the toolset work in any coding agent, not just Claude Code#47
travist merged 11 commits into
mainfrom
feat/multi-agent-portability

Conversation

@travist

@travist travist commented Aug 12, 2026

Copy link
Copy Markdown
Member

What this does

Makes @formio/ai installable and usable from any coding agent — Cursor, Codex, GitHub Copilot, VS Code, Windsurf, Cline, Gemini CLI — not only Claude Code. The server behaves the same regardless of who launched it, the skill prose names no client-specific tool or command, and the shipped bundle carries only what a consumer needs.

Five OpenSpec changes, openspec validate --changes clean.

Install routes — alternatives, not steps

Route Who Installs
Plugin install agents with a marketplace — Claude Code, Cursor, Copilot CLI, VS Code, Codex skills and MCP server, plus an install-time project prompt
npx skills add formio/ai everything else (75+ agents) skills only; formio-mcp-setup connects the server on first use

Only the Claude Code plugin install is verified end to end today. The other four marketplace rows are marked Coming Soon in the README rather than carrying an unverified command — see Not done below.

The plugin directory carries three manifests over one skills/ tree and one mcp.json: plugin.json (Agent Plugins 1.0.0), .cursor-plugin/plugin.json, and .claude-plugin/plugin.json. Each client detects its own and ignores the rest.

Server

BREAKING — FORMIO_PLUGIN_CONTEXT is removed. One environment variable, set only by the Claude Code manifest, decided four things at once: whether project_set was registered, whether the per-directory project map was read, whether cwd was a required tool parameter, and whether FORMIO_BASE_URL was required or defaulted. Everything it gated was unavailable outside Claude Code. Resolution is now one precedence order for every client:

  1. FORMIO_PROJECT_URL from the environment — pins the server
  2. otherwise the ~/.formio/projects.json mapping for the caller's cwd
  3. otherwise an actionable error naming project_set

Plugin behaviour is unchanged (it supplies no project URL, so the mapping stays authoritative), and a pinned stand-alone launch stays pinned even when a stale mapping exists. The variable is now inert rather than an error, so a stale launcher degrades to correct behaviour.

  • project_set is registered for every client. cwd has one schema everywhere — optional, absolute when supplied — built once and never varying by host.
  • New project command on the formio-mcp bin, so a project can be configured before any client connects: project set --project-url … --base-url … --cwd … writes the mapping through the same module the tool uses; project get --cwd … prints what resolves and which source won. No arguments still starts the stdio server.
  • The server declares MCP instructions at initialize, stating what it needs — the Project URL, and the Base URL, which builds the portal-login URL and keys the cached token, so it must not be assumed. Previously neither the instructions nor the error mentioned the base URL, and a self-hosted user could silently log in against the wrong deployment.
  • FORMIO_DEFAULT_PROJECT_URL is offered, never applied. Surfaced as a suggestion in the instructions and the resolution error for the agent to confirm and persist with project_set; it takes no part in resolution.
  • Browser login fails fast where there is no browser. CI, containers, and SSH with no display are detected before a port is bound, with guidance to set FORMIO_API_KEY. FORMIO_FORCE_BROWSER=1 overrides. Previously those hosts waited out the full 15-minute timeout before saying anything.

Skills

  • New formio-mcp-setup, plus a preflight block in every other skill: probe for the Form.io tools before the first call, hand off to setup when they are missing, and never work around their absence with raw HTTP against a deployment. Setup captures the project configuration through the new project command, so the first tool call after a reload works instead of failing.
  • No restart boundary. formio-application Step 4 (MCP Config) is deleted — the orchestrator writes no MCP configuration and never halts for a reload. Its Deployment step resolves an existing mapping before asking, so the project is captured once and never re-requested. This also removed a real bug: Step 4 wrote FORMIO_PROJECT_URL into .mcp.json, which now takes precedence and defeated the project_set call Step 3 had just made.
  • Client-specific prose is gone and a test keeps it out. Tool availability is a capability probe rather than a mcp__plugin_formio-ai_* prefix match; the structured-question instruction names the client's mechanism instead of AskUserQuestion; the skills CLI invocation no longer hardcodes -a claude-code. frontend-design keeps its name — it is itself a portable Agent Skill — and only the assumptions about how it is registered and installed were removed.
  • The nested Angular sub-skill moved to formio-angular-resources/ so its directory matches its declared name, and its description was trimmed from 2,339 to 1,020 characters (the spec's budget is 1,024). Both were Agent Skills specification violations — invisible in Claude Code, real in any client that discovers skills by recursive scan.
  • CI now validates every SKILL.md against the specification: name charset, name/directory agreement, description length, frontmatter key allow-list.

Breaking changes beyond the env var

  • The plugin ships no hooks. plugin/hooks/verify-project-url.mjs matched a Claude-namespaced tool prefix and expanded ${user_config.*} (not permitted by the Agent Plugins spec), so it only ever fired in Claude Code with a plugin install and was inert everywhere else — including every skills-only install. It could also deny calls the server resolves fine. Its behaviour is now carried for every client by the server's instructions, the resolution error, formio-mcp-setup, and the orchestrator's Deployment step.
  • No client prompts for a project URL at install time. The Cursor prompt fed its answer into FORMIO_PROJECT_URL, locking the server to one project and silently defeating project_set — contradicting the prompt's own description. That prompt and Claude Code's are gone; both manifests now declare FORMIO_BASE_URL alone. The .mcpb desktop bundle keeps an optional project prompt (a desktop host has no working directory to interview in), and its answer now reaches the server as FORMIO_DEFAULT_PROJECT_URL — offered for confirmation rather than pinned. To pin deliberately, set FORMIO_PROJECT_URL in your MCP configuration.
  • Install path, not API: manifests reachable from a git clone launch the server with npx -y @formio/mcp rather than the bundled ${CLAUDE_PLUGIN_ROOT}/server/stdio.mjs, because a clone contains no build output. No manifest, skill, or doc hard-codes a version range: @formio/mcp is a 0.x beta line, so a floor goes stale at the next release and a ceiling would freeze installed plugins on an old server. The plugin tarball ships no server bundle (files omits server/); the build still writes dist/plugin/server/stdio.mjs for the smoke test, and the .mcpb bundle builds its own copy. .claude-plugin/marketplace.json declares "source": "./plugin", which is also what makes the skills CLI discover the library.

Shipped surface

npx skills add formio/ai copies plugin/ into the user's own project, so everything in that tree is product. Three things in it were not:

  • Eval harnesses moved out to packages/skill-tests/evals/ — ~148KB of graders, fixtures, and Claude-specific runbooks that mean nothing in the project of someone who asked for a form.
  • The OpenSpec-generated skill mirrors are no longer committed. While they were, npx skills add formio/ai offered 17 skills — the Form.io library plus this repo's own OpenSpec and TDD tooling — and the CLI's -s flag takes exact names rather than globs, so it could not be filtered at install time. It now offers 11. Contributors regenerate them with the OpenSpec CLI; see CONTRIBUTING.md.
  • The npm metadata stopped describing a Claude Code plugin, for a bundle that now targets four clients.

A test enforces the boundary by allowlist so the tree cannot drift back.

Documentation correction

There is no universal .mcp.json. The READMEs and llms-install.md each claimed one mcpServers block worked in every client. Verified against vendor docs:

Client File Key
Claude Code .mcp.json mcpServers
Cursor .cursor/mcp.json mcpServers
VS Code / Copilot .vscode/mcp.json servers
Codex .codex/config.toml TOML [mcp_servers.<name>]

The server README also gained a Privacy Policy section and the manifest a privacy_policies array, both required by the Anthropic Software Directory.

⚠️ Merge order matters

Every manifest launches npx -y @formio/mcp, and npm's latest is still 0.8.4. 0.8.4 registers project_set only under FORMIO_PLUGIN_CONTEXT=1 — which these manifests deliberately stop setting — and its resolver reads only FORMIO_PROJECT_URL, so a plugin install resolving it gets 19 tools with no way to configure a project.

So a plugin install from main is broken between merging this PR and the release that publishes @formio/mcp 0.9.0. The local build is correct (20 tools, project_set present) — purely publish ordering. Both changesets are staged, so merging here opens a Version Packages PR; merging that closes the gap.

A >=0.9.0 floor in the manifests was tried and reverted: it converts this window into a visible npm resolution error, but @formio/mcp is a 0.x beta line, so the range outlives the window it guards — stale at the next minor, and a ceiling would freeze installed plugins on an old server. Release ordering is the fix. The one silent failure a floor did cover (a pre-0.9.0 binary ignores the project arguments and exits 0 printing nothing) is now covered by rule in the skills: empty output is never an answer, and no skill may report a mapping it did not read.

Verification

  • pnpm test918 pass, 0 fail (487 in @formio/mcp across 49 files, 431 in @formio/skill-tests across 36)
  • pnpm lint, pnpm format:check — clean
  • openspec validate --changes — 6/6
  • Suite also run with CI=true, which caught two suites exercising the browser-login path; they now opt in with forceBrowser: true
  • Skills-CLI install measured end to end: 17 skills offered before, 11 after; installs once to .agents/skills/ with Claude Code symlinked
  • Angular sub-skill rename regression-checked by eval: 6 subagent runs, before and after, identical pass rates and identical failure sets (94% / 83% / 100%) — openspec/changes/neutralize-core-for-multi-agent/eval-results.md

What reviewers should look at

  1. The resolution precedence (neutralize-core-for-multi-agent/design.md D1) — environment wins over the per-cwd map. This keeps a pinned CI launch deterministic; the alternative would let a stale project_set silently redirect it.
  2. The npx switch and the merge-order note above.
  3. plugin/skills/formio-mcp-setup/SKILL.md — it writes files into a user's workspace, so the approval gate and the four config shapes deserve a careful read.
  4. Dropping the OpenSpec mirrors from version control — the one contributor-workflow change here. A fresh clone has no /opsx:* skills until someone runs the OpenSpec CLI.

Not done in this PR

  • Manual client installs — Cursor (does variables actually prompt?), Claude Code from a clean marketplace add (the ./plugin source change is a regression risk introduced here), VS Code from source. Recipes: openspec/changes/package-as-agent-plugin/client-verification.md. All three are blocked on the publish above. Until each is verified, its README row stays Coming Soon — the skills-cli-distribution spec now requires a row to name its install command only once that install has been verified end to end, rather than requiring a command unconditionally.
  • .codex-plugin/plugin.json — the packaging plan called for one; only .claude-plugin and .cursor-plugin exist.
  • Marketplace submissions — Copilot CLI needs verification that it reads .claude-plugin/marketplace.json, and the Codex directory may still gate self-serve publishing.

🤖 Generated with Claude Code

The Anthropic Software Directory requires local connectors to carry three
things, and "missing or incomplete privacy policies result in immediate
rejection". The bundle had none of them: no privacy_policies array, and zero
occurrences of "privacy" in either README.

The manifest now declares https://form.io/privacy, and the server README — the
file build-mcpb.ts packs into the bundle — gains a Privacy Policy section.

The section describes what the corporate policy cannot: requests go only to the
configured deployment; ~/.formio/mcp-tokens.json and ~/.formio/projects.json are
the only files written, both 0600; form data never touches disk; there is no
telemetry. It also names a third-party disclosure found by reading auth.ts
rather than assumed absent — the browser sign-in page pulls styling and the
renderer from cdn.form.io, cdn.jsdelivr.net and fonts.googleapis.com, so those
hosts see the browser's IP while that page is open, and FORMIO_API_KEY avoids
the flow entirely.

Tests 1.13 and 1.14 assert the manifest array (HTTPS, manifest_version >= 0.2)
and the README section inside the packed archive, so a submission cannot fail on
a field that silently went missing.

Also fixes a footnote still claiming the server refuses to start without
FORMIO_PROJECT_URL, untrue since 0.8.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@travist
travist marked this pull request as draft August 12, 2026 02:32
@travist
travist force-pushed the feat/multi-agent-portability branch 2 times, most recently from 7347a05 to 411dc1a Compare August 13, 2026 21:36
@travist
travist marked this pull request as ready for review August 14, 2026 14:59
@johnformio

Copy link
Copy Markdown
Contributor

Code review — 8 findings

Scope reviewed in depth: the MCP server source changes (config.ts, project-resolver.ts, server.ts, stdio.ts, auth.ts, browser-availability.ts, cli/project-command.ts, tools/), the three plugin manifests plus the build/smoke scripts, and the skill prose that drives file writes (formio-mcp-setup, formio-application/DEPLOYMENT.md).

Verified clean: all relative markdown links under plugin/skills/ resolve, the formio-angular-resources rename is internally consistent, and the new project-command-integration.test.ts does not touch the real ~/.formio (src/__tests__/setup.ts isolates HOME, despite the comment implying otherwise).

Ranked most severe first.


1. packages/mcp-server/src/project-resolver.ts:66 — empty FORMIO_PROJECT_URL permanently blocks project_set

Mixed truthiness semantics. Line 64 uses !envProjectUrl (falsy) to decide whether to read the cwd mapping, but line 66 uses envProjectUrl ?? mappedEnv?.FORMIO_PROJECT_URL (nullish).

With FORMIO_PROJECT_URL="" the mapping is read and then discarded, projectUrl stays '', and every tool throws "No Form.io project is configured" even though project_set wrote a valid mapping.

This is reachable on a shipped route: scripts/build-mcpb.ts declares FORMIO_PROJECT_URL: '${user_config.formio_project_url}' with required: false, so a .mcpb user who leaves Project URL blank gets an empty string — and project_set can then never fix it.

getConfig at config.ts:61 deliberately preserves '' (projectUrl === undefined ? undefined : …). Use projectUrl || undefined there, or || here.

2. packages/mcp-server/src/cli/project-command.ts:84project set silently reverts a custom base URL to the default

project set without --base-url (and with no FORMIO_BASE_URL in the invoking shell — normal, since that var is set for the MCP server process, not the terminal) writes an entry with no FORMIO_BASE_URL, and writeProjectEntry replaces the whole entry.

A directory previously mapped to https://forms.acme.com silently reverts to the https://api.form.io default — precisely the wrong-deployment login failure the rest of this PR is written to prevent. Either merge with the existing entry's base URL or refuse to drop it.

project_set the tool has the same overwrite semantics, but the CLI is new code and untested for this case: project-command.test.ts covers only the env-fallback path.

3. packages/mcp-server/src/stdio.ts:15process.exit right after stdout.write can truncate output

process.stdout.write(...) immediately followed by process.exit(result.exitCode) (line 20) can truncate or drop output: on macOS, Node pipes are asynchronous, and every documented invocation is through a pipe (npx -y @formio/mcp project get --cwd "$(pwd)" run from an agent's shell tool, per formio-mcp-setup/SKILL.md and DEPLOYMENT.md).

The agent then sees exit 0 with empty stdout and reports "no project configured", or can't confirm the mapping it just wrote. Set process.exitCode and return, or write with a flush callback before exiting.

4. packages/mcp-server/src/auth.ts:141 — the suggested devcontainer remedy does not unblock login

assertBrowserAvailable throws whenever browserlessReason fires, and browser-availability.ts:31 fires on /.dockerenv regardless of FORMIO_AUTH_HOST / FORMIO_AUTH_PORT.

A devcontainer user who follows the first sentence ("set FORMIO_AUTH_HOST=0.0.0.0 and FORMIO_AUTH_PORT to a published port") gets the identical error; only FORMIO_FORCE_BROWSER=1 gets through. This is also a behavior regression for devcontainer/Codespaces users whose ports are auto-forwarded and whose login previously worked.

Either treat an explicit authHost/authPort as consent (skip the container branch) or drop that sentence.

5. packages/mcp-server/src/config.ts:57FORMIO_DEFAULT_PROJECT_URL is never validated but is echoed as a suggestion

It's surfaced verbatim to the user/model as "A default is configured (FORMIO_DEFAULT_PROJECT_URL): … — the suggested project" (project-resolver.ts:39) and in SERVER_INSTRUCTIONS.

plugin/.cursor-plugin/plugin.json wires it to "${FORMIO_DEFAULT_PROJECT_URL}" with no default, and the PR itself notes Cursor's variables substitution is unverified — an unsubstituted placeholder becomes the "suggested project" the agent offers, and an agent that accepts it persists garbage via project_set.

Run it through normalizeHttpUrl and ignore it (or warn) when it isn't a valid http(s) URL.

6. scripts/test-plugin.ts:143 — version-agreement guard passes when all versions are missing

validateManifestVersionsAgree maps a missing version to the literal string 'missing', so three manifests that all lack version produce distinct.size === 1 and pass — the exact drift this guard exists to catch. validateAgentPluginManifests only checks $schema, so plugin.json never gets a version type check either. Fail explicitly when any version is absent.

7. README.md:75 — "A plugin install prompts for both." is no longer true

plugin/.claude-plugin/plugin.json dropped formio_default_project_url, so a Claude Code install prompts for the Base URL only. llms-install.md states this correctly ("Claude Code asks for the Base URL, Cursor asks for the Base URL and a default project"); the README contradicts it.

8. packages/mcp-server/src/project-resolver.ts:22 — optional cwd lets project_set key a mapping to the server's cwd

Making cwd optional for every client removes the schema-level guard that previously forced plugin callers to pass it. A model that omits cwd gets No Form.io project is configured. with no for cwd=…, and the recommended remedy (project_set with no cwd) keys the mapping to the server's process.cwd() (tools/project_set.ts, getServerCwd) — which for a plugin-launched server is not the user's directory.

Subsequent calls that do pass cwd then miss the mapping, producing a loop of "no project configured" plus duplicate entries. Consider having project_set refuse (or loudly warn) when cwd is omitted, rather than silently keying on the server's spawn directory.


🤖 Generated with Claude Code

@johnformio

Copy link
Copy Markdown
Contributor

Re-review — ad38a21ca1bb2a (27 files, +485/-68)

All eight findings from the previous review are fixed:

# Fix
1 Belt and braces — config.ts now projectUrl ? … : undefined, project-resolver.ts uses ||. Both tested.
2 readProjectEntry(cwd, cacheDir)?.env.FORMIO_BASE_URL as last fallback in the CLI, previousBase in project_set.
3 process.exitCode plus an else branch, so Node flushes before exiting.
4 New publishedLoginEndpoint consent option short-circuits the container / SSH / no-display branches (CI still fails). The follow-through holds up: with FORMIO_AUTH_HOST=0.0.0.0 the advertised URL rewrites to 127.0.0.1:<port>, so the remedy the error recommends now actually works from a forwarded port.
5 readSuggestedProjectUrl validates through normalizeHttpUrl and drops an unusable value with a note on stderr.
6 Missing/non-string versions now fail before the agreement check, with a smoke test that strips all three manifests.
7 README.md, llms-install.md, plugin/README.md and the changeset all rewritten; the Cursor manifest dropped the project prompt and a test pins variables.properties to ['FORMIO_BASE_URL'].
8 project_set appends a warning naming the server cwd when cwd is omitted, with both a positive and a negative test.

The design shift beyond the findings — no client prompts for a project URL, .mcpb renamed to formio_default_project_url as a suggestion rather than a pin — is coherent across the manifests, the tests, the docs, the changeset, the openspec delta, and formio-angular/SETUP.md.

Verification on ca1bb2a: packages/mcp-server 386 passed / 14 skipped, tsc --noEmit clean. (mcpb-build.test.ts could not shell out to pnpm build:mcpb in my scratch worktree — an artifact of how I linked node_modules, not a defect here.)

Two new findings.


1. packages/mcp-server/src/cli/project-command.ts:91 — an empty FORMIO_BASE_URL still drops the mapped base URL

The two new fallbacks use ??, so an empty-string environment value short-circuits them:

const declaredBaseUrl = flags['base-url'] ?? context.env.FORMIO_BASE_URL ?? mappedBaseUrl;

With FORMIO_BASE_URL="" this yields '', which is falsy on the next line, so baseUrl is undefined and the entry is rewritten without FORMIO_BASE_URL — the mapped https://forms.acme.com reverts to the api.form.io default and the login goes to the wrong deployment. That is the finding-2 consequence, reached by the finding-1 route. Confirmed with a throwaway test against this commit:

writeProjectEntry('/abs/path', {
  FORMIO_PROJECT_URL: 'https://forms.acme.com/old',
  FORMIO_BASE_URL: 'https://forms.acme.com',
}, cacheDir);

runProjectCommand(
  ['project', 'set', '--project-url', 'https://forms.acme.com/new', '--cwd', '/abs/path'],
  { cacheDir, env: { FORMIO_BASE_URL: '' } }
);

readProjectEntry('/abs/path', cacheDir)?.env.FORMIO_BASE_URL;
// expected 'https://forms.acme.com', received undefined

packages/mcp-server/src/tools/project_set.ts:64 has the identical chain (baseUrlArg ?? getEnvBaseUrl() ?? previousBase), and getEnvBaseUrl returns process.env.FORMIO_BASE_URL raw.

An empty value is reachable the same way it is for the project URL: scripts/build-mcpb.ts marks formio_base_url required: false, so a host passes an empty string for a prompt the user cleared. config.ts and project-resolver.ts were both switched to falsy tests in this very commit — these two chains want the same treatment, and the new CLI tests pass env: {} rather than env: { FORMIO_BASE_URL: '' }, which is why they don't catch it.

2. llms-install.md:38 — the docs now call the Base URL prompt optional, but the Claude manifest requires it

Install-time prompts are the Base URL and nothing else — Claude Code and Cursor both ask for it, and it is not required…

and plugin/README.md:67:

Nothing is required — the server starts unconfigured…

plugin/.claude-plugin/plugin.json declares:

"formio_base_url": { "type": "string", "title": "Form.io base URL", "…": "", "required": true }

required: true with no default, so a Claude Code plugin install blocks until the user supplies a value the docs just told them was optional. The Cursor manifest and the .mcpb bundle both declare it optional with a https://api.form.io default, and plugin-manifests.test.ts pins Cursor's variables.required to [] — nothing pins the Claude manifest, so the three disagree. Since the server defaults the base URL anyway, dropping required: true and adding the same default looks like the fix; otherwise the two sentences above should say Claude Code blocks on it.


🤖 Generated with Claude Code

@johnformio

Copy link
Copy Markdown
Contributor

Code review

Reviewed the full diff of feat/multi-agent-portability (22b1c6f) vs main — server resolution/config/CLI code, plugin manifests, build scripts, and the changed skill prose. Ran the suite in a scratch worktree to ground the review: pnpm lint clean, @formio/mcp 464/464 pass, @formio/skill-tests 430/430 pass (the two HTMLElement is not defined unhandled errors from @formio/js teardown are pre-existing and unrelated to this diff).

The core resolver rewrite holds up. project-resolver.ts, config.ts, project-map.ts, and project_set.ts are self-consistent under case analysis — env-pin vs. mapping precedence, the mappedBaseAppliesToPin guard, tolerateUnreadable, the falsy-vs-nullish choices, and the cacheDir threading through readProjectEntry/writeProjectEntry. Side note: the eval-harness move accidentally fixes a pre-existing off-by-one in formio-angular-resources/grade.py's REPO_ROOT — the five .parents were one short at the old nesting depth and are exactly right at the new one.

Findings below, most severe first.


1. plugin/.claude-plugin/plugin.json:22 — manifests launch an unpinned npx -y @formio/mcp

All three manifests launch the server unpinned (also plugin/.cursor-plugin/plugin.json:17, plugin/mcp.json:6), while .claude-plugin/marketplace.json now installs from ./plugin. Resolved against the current latest (0.8.4), the plugin is non-functional rather than loudly broken: 0.8.4 registers project_set only under FORMIO_PLUGIN_CONTEXT=1 — which these manifests deliberately stop setting — and its resolver reads only FORMIO_PROJECT_URL. Every tool call fails with "Set the FORMIO_PROJECT_URL environment variable" and there is no tool to fix it with.

The PR discloses this as a merge-ordering gap, but the same unpinned spec also permits an offline/cached older resolution after release. The skills themselves mandate @formio/mcp@>=0.9.0 on every npx invocation for exactly this reason ("an unavailable version is an npm resolution failure you can see"); the manifests should carry the same floor.

2. plugin/skills/formio-application/DEPLOYMENT.md:42project get output confirmed without reading Source:

"If it prints a Project URL and Base URL, confirm those resolved values in one line and move on" discards the Source: line, then Skip Conditions (line 131) says "the mapping already exists, so there is nothing to persist". But project get runs in the agent's shell, so it also reports a resolution sourced from the shell's FORMIO_PROJECT_URL and a Base URL sourced from the built-in default.

Concrete failure: a user (or CI) exports FORMIO_PROJECT_URL in the shell while the MCP server is launched from .mcp.json with no env — project get prints the project with Source: this shell's environment, Deployment confirms it and skips project_set, and Step 4's project_import fails with "No Form.io project is configured".

Second variant: a directory mapped by project_set without baseUrl makes project get print Base URL: https://api.form.io from Source: the default, which the agent then confirms to a self-hosted user as their configured deployment — the exact silent wrong-host outcome this PR set out to prevent.

formio-mcp-setup/SKILL.md gets this right ("Read its Source: line for what it is"); DEPLOYMENT.md needs the same rule, and must persist with project_set whenever the source is not mapping.

3. plugin/skills/formio-application/DEPLOYMENT.md:42 — every non-zero exit treated as "nothing mapped"

"If it exits non-zero or prints nothing, nothing is mapped: run the interview" conflates every non-zero exit with an unmapped directory. runProjectCommand's outer catch returns exit 1 for an unreadable ~/.formio/projects.json, a relative --cwd, a malformed stored URL, and an npm resolution failure alike. The agent then runs a full interview and calls project_set, which fails again for the same underlying reason, and the user gets an interview-then-error loop with no mention of the real cause. The map itself is safe (writeProjectEntry re-reads and rethrows, so nothing is clobbered), but the instruction should distinguish "no mapping" from "the command failed" and surface the latter.

4. plugin/package.json:30server dropped from files but still built and asserted

"server" was removed from files while scripts/build-plugin.ts still bundles dist/plugin/server/stdio.mjs and scripts/test-plugin.ts still asserts its presence, so the published @formio/ai tarball ships no server despite the PR stating "The bundle is still built and published — it is what the npm package … run". Nothing references the bundle anymore (all manifests use npx), so this is dead build output plus a false claim rather than a runtime break — but the build/smoke-test still gate on an artifact that no longer ships.

5. README.md:55 — intro contradicts the install flow

New text says a plugin install "prompts for your project URL up front", directly contradicting line 75 ("A plugin install prompts for the Base URL only. The Project URL is never an install-time question") and the manifests, which declare only a formio_base_url / FORMIO_BASE_URL variable. A user following the intro will look for a project prompt that no longer exists.

Related: the PR body's claim that the Cursor prompt "now feeds FORMIO_DEFAULT_PROJECT_URL" is also stale — plugin/.cursor-plugin/plugin.json declares no project variable at all; the changeset text is the accurate one.

6. packages/mcp-server/src/cli/project-command.ts:231 — unreadable-map rethrow is flattened into the generic failure

resolveOrNull's comment says an unreadable-map error "travels to the caller instead" of being reported as "nothing configured", but the rethrow is caught by runProjectCommand's own catch at line 258 and flattened into fail(message) — the same exitCode: 1, empty-stdout shape as the genuine "No Form.io project is configured for …" branch. The distinction the rethrow exists to preserve is therefore not observable to any caller except by substring-matching the message, which is what makes finding 3 possible. Either surface a distinct exit code, or let the error escape the outer catch.

7. scripts/build-plugin.ts:90 — a build mutates committed manifests

buildPlugin() calls syncSourceManifestVersions(), which rewrites the committed plugin/{.claude-plugin,.cursor-plugin,}plugin.json as a side effect of a build (and of prepublishOnly, i.e. during pnpm release). It is idempotent when versions already agree, so it is invisible today, but any build run after a hand-edited plugin/package.json version silently mutates tracked source outside a changeset:version run. Worth restricting the source-tree write to the sync:versions entry point and having the build only verify agreement.

@travist

travist commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Review findings addressed — 9f53258

All seven are fixed. pnpm test 902 pass (472 @formio/mcp, 430 @formio/skill-tests), pnpm lint and pnpm format:check clean, openspec validate --changes 6/6.

# Finding Fix
1 Manifests launch an unpinned npx -y @formio/mcp All three now launch npx -y "@formio/mcp@>=0.9.0" — the same floor every skill pins. New tests assert the floor is present in mcp.json, .cursor-plugin/plugin.json, and .claude-plugin/plugin.json, that no manifest carries the bare spec, and that the manifests and the skills pin the same floor (so the two cannot drift apart). The hand-written config snippets in README.md and packages/mcp-server/README.md carry it too.
2 project get output confirmed without reading Source: DEPLOYMENT.md now makes Source: the point of the output: confirm-and-skip only when the mapping supplied both URLs; a project sourced from the shell's environment is confirmed as a suggestion and persisted with project_set (the server does not share your shell); a Base URL sourced from the default is never presented as the user's deployment — derive or ask, then persist. Skip Conditions restated in the same terms.
3 Every non-zero exit treated as "nothing mapped" Split into distinct codes (finding 6) and a table in DEPLOYMENT.md: 0 read the Source: line, 1 nothing is mapped → interview, 2 the command failed → show its stderr and stop without interviewing. formio-mcp-setup/SKILL.md gained the same rule.
4 server dropped from files but still described as published The code was the intended state (4.5c asserts it deliberately) — the docs were stale. The changeset, the openspec delta, and build-plugin.ts now say what is true: files omits server/, the build writes dist/plugin/server/stdio.mjs for the smoke test's tools/list exercise, and the .mcpb bundle builds its own copy at dist/mcpb/server/index.mjs.
5 README intro contradicts the install flow Intro now says the install prompts for the Base URL — the deployment, not a project — and the table's "install prompt" reads "Base URL prompt". The PR body's stale Cursor/FORMIO_DEFAULT_PROJECT_URL claim is corrected: the Cursor manifest declares FORMIO_BASE_URL alone, and it is the .mcpb bundle whose optional project prompt feeds FORMIO_DEFAULT_PROJECT_URL.
6 Unreadable-map rethrow flattened into the generic failure formio-mcp project now has a three-code contract: EXIT_OK 0, EXIT_NOT_CONFIGURED 1 (the command ran, nothing is mapped), EXIT_FAILED 2 (usage error, malformed URL, relative --cwd, unreadable projects.json). The rethrow's distinction is now observable without substring-matching, which is what makes finding 3's guidance actionable. Documented in both READMEs and in project-map-routing's spec delta.
7 A build mutates committed manifests buildPlugin() calls assertSourceManifestVersionsAgree() instead of the writer and fails naming pnpm sync:versions. pnpm sync:versions stays the only writer and gained --check for write-free verification. Tests: --check reports drift without changing a byte, exits 0 when the manifests agree, and 1.12 asserts the committed manifests are byte-identical after a build.

One note on merge order, now that the floor is pinned: between merging this PR and the release that publishes 0.9.0, a plugin install fails at npm resolution rather than installing a 19-tool server with no way to configure a project. Loud instead of silent — the PR body's merge-order section is updated to say so.

@travist

travist commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Reverted the version pinning — 6832e35

Nothing above 0.9 is breaking, and I over-applied the floor. The answer to "what is so breaking about > 0.9": nothing. The hazard was strictly below 0.9, and it is a merge-window problem, not a lasting one:

  • pre-0.9 has no project subcommand, so npx -y @formio/mcp project get starts the stdio server, reads EOF, and exits 0 printing nothing — which reads as "nothing mapped".
  • 0.8.4 registers project_set only under FORMIO_PLUGIN_CONTEXT=1, which this PR stops setting.

Both close the moment 0.9.0 publishes, because latest then satisfies them. A >=0.9.0 range in a shipped manifest or skill outlives the window it guards: @formio/mcp is a 0.x beta line where every minor may break, so a floor is stale at the next release, and finding 8's suggested <1 ceiling would freeze installed plugins on an old server. Both are rejected.

Every launch is npx -y @formio/mcp again — the three manifests, the client configs formio-mcp-setup writes, the preflight line each skill prints, the project get / project set invocations, the README and llms-install.md snippets, the global-install fallback. Nothing hard-codes a version anywhere; a test asserts that in both directions (manifests must name the package with no suffix, no skill or repo doc may carry a range).

What covers the gap instead:

Concern Mechanism
Merge-to-release window Release ordering — publish @formio/mcp with or before the marketplace change. The merge-order section above says so, now without the pin claim.
Pre-0.9 project no-op (the one silent failure) Rule, not range: empty output is never an answer, never report a mapping you did not read, never claim a project was persisted when project set printed nothing.
A server too old to serve project_set Surfaces as missing tools, which every skill's preflight already routes to formio-mcp-setup.
npx cannot fetch the server at all Exits 1 with npm error on stderr — the exit-code table tells the agent to read stderr before treating a 1 as "unmapped".

Findings 1 and 8 are therefore resolved by removal rather than by pinning. Findings 2–7 and 9 from the second review stand as fixed in 74754ab (exit-code contract, per-entry project-map validation, FORMIO_DEFAULT_PROJECT_URL offer in project get, Source: attribution, Angular SETUP branch, CLAUDE.md inventory).

pnpm test 918 pass (487 @formio/mcp, 431 @formio/skill-tests), pnpm lint and pnpm format:check clean, openspec validate --changes 6/6.

@travist
travist force-pushed the feat/multi-agent-portability branch from 6832e35 to 1e42e36 Compare August 17, 2026 15:05
GitHub is retiring the Node 20 action runtime, so every action pinned to a
major built on it warns today and breaks when the runtime goes. Bump to the
majors running on Node 24: actions/checkout v5, actions/setup-node v6,
actions/upload-artifact v5, actions/download-artifact v6, and
pnpm/action-setup v4.

The inputs each action takes are unchanged across these majors, so no
workflow logic moves with them.
@travist
travist force-pushed the feat/multi-agent-portability branch from 1e42e36 to 1331e85 Compare August 17, 2026 15:14
travist and others added 2 commits August 17, 2026 10:20
The MCP server and the skill library were built around one client. Both now
work the same way in every agent that reads Agent Skills or speaks MCP, and
there are two ways in, alternatives rather than steps.

**A plugin install** wherever the agent has a marketplace — Claude Code,
Cursor, GitHub Copilot CLI, VS Code, Codex. One step, carrying the skills and
the MCP server. `plugin/` ships three manifests over one `skills/` tree and
one `mcp.json`: `plugin.json` (Agent Plugins 1.0.0), `.cursor-plugin/plugin.json`,
and `.claude-plugin/plugin.json`. Each client detects its own and ignores the
rest. `.claude-plugin/marketplace.json` declares `source: "./plugin"`, so an
install resolves from a clone rather than waiting on an npm publish — and
because a clone carries no build output, every manifest launches the server
with `npx -y @formio/mcp`.

**`npx skills add formio/ai`** for everything else — one write to
`.agents/skills/`, with Claude Code symlinked to the same files. That
installer handles skills only, so a new `formio-mcp-setup` skill connects the
server on first use: it writes the MCP configuration for all four client
shapes behind an approval gate, offers to capture the project, and says how to
reload. Every other skill carries a preflight that checks for its tools, hands
off to setup when they are missing, and is forbidden from working around the
gap with raw HTTP against a Form.io deployment.

One server behaviour everywhere:

- **Project resolution follows a single documented order** — `FORMIO_PROJECT_URL`
  from the environment, then the working-directory mapping in
  `~/.formio/projects.json`, then an actionable error naming `project_set`.
  Environment-first keeps a pinned CI or container launch deterministic
  regardless of a stale mapping. `project_set` is registered for every client,
  `cwd` is one schema everywhere, and `FORMIO_BASE_URL` always defaults.
- **A base URL belongs to a deployment, not a directory.** A mapped base URL
  outranks the environment global, a pinned project may borrow a mapped base
  URL only when the mapping names that same project, and a re-`set` without an
  explicit base URL keeps the one already mapped. Each of these prevents a
  self-hosted directory from silently logging in against api.form.io.
- **The project can be configured before any client connects.** The
  `formio-mcp` bin gains `project set --project-url <url> --base-url <url>
  --cwd <path>`, which writes through the same module the `project_set` tool
  uses, and `project get --cwd <path>`, which prints what resolves and which
  source supplied each half. It exits 0 when it resolved, 1 when nothing is
  mapped for that directory, and 2 when it could not answer — so a caller can
  tell "nothing here yet" from "this failed" instead of interviewing the user
  and hitting the same error again. With no arguments the bin starts the stdio
  server exactly as before.
- **An unreadable project map fails loudly** rather than reading as empty,
  per file and per entry, naming the file and the directory. Reporting a
  corrupt map as an unmapped one sent callers to a write that discarded every
  other directory's mapping.
- **The server explains itself.** It declares MCP `instructions` at initialize
  describing what it needs — the Project URL, and the Base URL, which builds
  the portal-login URL and keys the cached token and so must not be assumed.
  Used stand-alone with no skills installed, that plus the resolution error is
  the whole of what an agent needs.
- **A configured default is offered, not applied.** `FORMIO_DEFAULT_PROJECT_URL`
  is surfaced as a suggestion in the instructions, in the resolution error, and
  in `project get`, for the agent to confirm and persist. It takes no part in
  resolution. `FORMIO_PROJECT_URL` remains the opposite: it pins the server,
  and `project_set` cannot redirect it.
- **Browser login fails fast where there is no browser.** CI, containers, and
  SSH sessions with no display are detected before a port is bound, with
  guidance to set `FORMIO_API_KEY`; `FORMIO_FORCE_BROWSER=1` overrides the
  check. Such hosts previously waited out the full 15-minute timeout.

BREAKING — `FORMIO_PLUGIN_CONTEXT` is removed. It gated per-directory project
routing, `project_set` registration, and a required `FORMIO_BASE_URL`, so all
of that was unavailable outside one client. Plugin behaviour is unchanged.

BREAKING — the plugin ships no hooks. The `verify-project-url` gate matched a
Claude-namespaced tool prefix and expanded `${user_config.*}`, so it fired in
exactly one client with a plugin install and was inert everywhere else,
including every skills-only install; it could also deny calls the server
resolves fine. Its behaviour is carried identically for every client by the
server's instructions, the resolution error, `formio-mcp-setup`, and the
orchestrator's Deployment step.

BREAKING — no client prompts for a project URL at install time. The Cursor
prompt fed `FORMIO_PROJECT_URL`, which outranks every per-directory mapping,
so filling it in locked the server to one project and silently defeated
`project_set` — contradicting the prompt's own description. A deployment is
shared across a developer's projects, but a Form.io project is one-to-one with
the application built against it, so the one folder an install-time answer is
collected in is the only folder it is right for. Install asks for
`FORMIO_BASE_URL` alone. The `.mcpb` desktop bundle keeps its optional project
prompt — a desktop host has no working directory to interview in — but that
answer now arrives as `FORMIO_DEFAULT_PROJECT_URL`, offered rather than pinned.

BREAKING (install path, not API) — manifests reachable from a git clone launch
`npx -y @formio/mcp` rather than the bundled
`${CLAUDE_PLUGIN_ROOT}/server/stdio.mjs`. No manifest, skill, or doc carries a
version range: `@formio/mcp` is a 0.x line, so a floor goes stale at the next
release and a ceiling would freeze installed plugins on an old server. The
tarball ships no server bundle either; the build still writes
`dist/plugin/server/stdio.mjs` for the smoke test, and the `.mcpb` bundle
builds its own copy.

The skills read correctly in every client. Instructions name the client's
structured question mechanism rather than one client's tool, and the batching
rule — everything a step needs in one round, never a sequence of prompts — is
stated portably. Tool availability is a capability probe rather than a
tool-name prefix match. The orchestrator writes no MCP configuration and no
longer halts for a reload: a missing server routes to `formio-mcp-setup`, and
its Deployment step resolves an existing mapping before asking, reading the
`Source:` line so a value that came from the shell or from a default is
persisted rather than confirmed as though it were on record.

Release engineering: `pnpm sync:versions` is the only thing that writes a
version into the committed manifests (`--check` verifies without writing), and
the plugin build verifies instead of stamping, so a build — which runs during
`prepublishOnly` — cannot mutate tracked source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five change sets carry the work: `neutralize-core-for-multi-agent` (project
resolution, the `project` command, the server's own instructions),
`neutralize-skills-for-multi-agent` (portable skill prose, the
`formio-mcp-setup` skill, the no-restart orchestrator),
`package-as-agent-plugin` (three manifests over one skills tree, the skills-CLI
route, the release flow), `offer-default-project`
(`FORMIO_DEFAULT_PROJECT_URL` as a suggestion rather than a pin), and
`prune-shipped-surface` (what a consumer's tree receives, and what stays
local). Each keeps its motivation in the proposal, its decisions in the design,
and its behaviour as scenarios — the durable record of why the toolset resolves
a project the way it does.

`examples/apps/storyboard.md` gets the Trello-style kanban description it
should have had.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@travist
travist force-pushed the feat/multi-agent-portability branch from 1331e85 to a043efc Compare August 17, 2026 15:21
travist and others added 7 commits August 17, 2026 10:26
…ith a display

Three defects found reviewing this branch against main.

Mapped URLs bypassed validation. getConfig runs every URL it reads from
the environment through normalizeHttpUrl, but values read from
~/.formio/projects.json were only stripped of trailing slashes. A mapping
holding FORMIO_BASE_URL: "forms.mysite.com" resolved cleanly, `project
get` exited 0 and printed it as the answer, and every request then died
inside fetch as "Failed to parse URL from forms.mysite.com/current" — far
from the file that caused it. Mapped URL values are now normalized on the
read path and an unusable one raises ProjectMapUnreadableError naming the
variable, the directory and the file, so `project get` exits 2 instead.
The check deliberately does not live in validateEntry: writeProjectEntry
validates the entry it is about to overwrite, and that rewrite is the
repair. To keep the repair working, both project_set and the CLI's
`project set` now read the stored base URL tolerantly, dropping an
unusable value with a note rather than failing with the very error the
call was made to clear.

An omitted cwd resolved silently against the server's own directory.
project_set writes under an explicit cwd; resolveProject falls back to
process.cwd() when none is passed, so a write and a read could key
different directories — at best "No Form.io project is configured", at
worst a project_import into whatever project that other directory maps to.
The fallback stays, because the server cannot distinguish an omitted cwd
from a matching one and refusing it would break both the cwd-less desktop
flow and the common case where the spawn directory is correct. What
changes is that it is no longer silent: resolution through the fallback
emits a note naming the directory, the missing-project error names the
searched directory and the cwd argument instead of only naming
project_set (which loops), cwdSchema's description says to pass it on
every call, and IMPORT.md — the doc whose example produced the failure —
now shows project_import({ cwd, template }).

A container with a working display could not log in at all. The container
branch returned a reason before hasDisplay was consulted, so
assertBrowserAvailable threw before the login server bound its port. A dev
container started with the host's X socket shared in (docker run -e
DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix) sets none of the
editor-forwarding markers and opens the user's own browser. The branch now
consults the display, matching the SSH branch below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings in #49's skill security hardening. Its privacy-policy commit was
already on this branch, so only the second half merged.

One conflict, in packages/mcp-server/README.md, both halves of it the same
thing: main still describes the plugin-context split this branch removes.

The `FORMIO_PROJECT_URL` footnote — kept this branch's. #49 rewrote the old
footnote to stop claiming the server refuses to start without the variable;
this branch had already rewritten the same sentence to say that and more, so
the fix survives. What did not survive is the rest of main's text, which
names `FORMIO_PLUGIN_CONTEXT`, "plugin context", and the `verify-project-url`
SessionStart/PreToolUse hook — all gone from this branch, and the `**`
footnote it carried had no marker left in the table above it.

The `projects.json` row in the Privacy Policy table — dropped main's
"(plugin context only)" qualifier. `project_set` is registered in every
client now.

Also added the `---` separator #49 omitted before `## Privacy Policy`; every
other section in that README has one, and the conflict sat on those lines.

pnpm test (504), pnpm lint, and prettier --check on the merged README all
pass. The eight prose-wrap warnings under plugin/skills/ predate this merge
and are untouched by it.
…hiving

`local-only-artifacts.test.ts` greps the git index for `multi-agent-portability`
and exempts the files that name the roadmap in order to forbid it. One of those
exemptions was written as the literal prefix `openspec/changes/prune-shipped-surface/`,
which stopped matching the moment that change was archived: `openspec archive`
moves a change to `openspec/changes/archive/<date>-<name>/` and derives
`openspec/specs/<capability>/spec.md` from its delta on the way, so six tracked
files that had been exempt were suddenly offenders.

The exemptions are now patterns rather than literal prefixes, so the change
matches wherever it currently lives and the derived capability spec is covered
too. Nothing about the rule changes — no other tracked file may reference the
roadmap, which is what the requirement in shipped-surface-boundary says.

Rewriting those six files was the alternative and the wrong one: a requirement
that forbids depending on a file cannot be stated without naming it.

Also fixes why `pnpm test` passed locally while CI failed on the same commit.
`@formio/skill-tests` asserts on the whole repository — plugin/, openspec/,
scripts/, and the git index — while Turbo only hashes files inside the package,
so an openspec-only change left a stale cache entry to replay and the suite
reported green without running. Its `test` task is now uncacheable, because the
git state it reads is not a file Turbo can hash. The mcp-server suite reaches
outside its package too but reads only files, so it keeps its cache and the
repo-wide paths it depends on are declared as globalDependencies instead.

pnpm test (935), pnpm lint, and prettier --check all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@travist
travist merged commit a494543 into main Aug 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants