Skip to content

fix(client): Agent Skills — three contract fixes and the tests that pin them - #61

Closed
XieX wants to merge 1 commit into
xie/skills-fdv2-transportfrom
xie/skills-09-contract-fixes
Closed

XieX wants to merge 1 commit into
xie/skills-fdv2-transportfrom
xie/skills-09-contract-fixes

Conversation

@XieX

@XieX XieX commented Sep 16, 2026 •

Copy link
Copy Markdown

Three big fixes for the Agent Skills implementation, that came out of agent code review.

  1. Removes internal constants from exports (SKILL_OBJECT_KIND and MAX_SKILL_CONTENT_BYTES).
  2. Fixes a key mismatch from being a wrong_version error to an integrity_failure error.
  3. recordRevoked omits version if it's malformed rather than including version: null.

Plus two test fixes and some docs updates.

1. Unpublish SKILL_OBJECT_KIND and MAX_SKILL_CONTENT_BYTES

Both were re-exported from the package root. Neither belongs there, and the absence is itself the contract:

  • SKILL_OBJECT_KIND is an SDK-side seam string, not the wire format. It is what the accessors hand SkillStore.getObject, and an adapter is free to map it onto whatever its transport actually uses. Exporting it advertises a claim this side cannot make and could not walk back once a caller depended on it.
  • MAX_SKILL_CONTENT_BYTES is a local enforcement bound on content the platform produces, set well above the platform's own limit precisely so that limit can move without this constant following. Publishing it semver-locks a number this side does not own, and a caller pre-flighting "will my skill fit?" against it reads the backstop rather than the real bound.

The test that should have caught this was titled "the five fixed values" and carried a comment claiming callers need the cap to pre-check content — the exact reading the contract rejects. It now asserts the three public constants' values and the two names' absence, mirroring test_skills.py. Tests that need the cap keep reading it from skills-core.js.

2. Key mismatch is integrity_failure, not wrong_version

resolveFromStore returned wrong_version when skill.key !== key. wrong_version names a version mismatch specifically — there is deliberately no wrong_key to parallel it. Content was delivered and its identity did not verify, which is the one outcome a caller is expected to fail closed on; filed under wrong_version, skill substitution was invisible to a fail-closed caller. The JSDoc already described the intent correctly; only the token was wrong.

The no-signal/no-log half is deliberate and unchanged: the check runs after verification has passed, so the eight-token reason_code vocabulary does not cover it and neither telemetry surface fires. The test now asserts that asymmetry in both directions — the outcome is integrity_failure, the recording emitter saw nothing, and no ld.skills.integrity_failure record was logged — and branches on the token rather than matching detail text, which is for a human.

3. AgentControl Skill Revoked Received omits version rather than emitting null

recordRevoked built its properties unconditionally, so a manifest entry with a malformed version published version: null. It now adds the property only for a valid version, and the caller passes the raw manifest value through instead of pre-collapsing it to null. The new test uses not.toHaveProperty('version') — reading props.version and comparing to undefined would pass either way.

skill_key is now redacted through isValidSkillKey there too, as Python's record_revoked does. The key comes from the manifest — a file on disk anything with write access to the root can author — so it is exactly as attacker-controlled as a wire object.

Reachability, stated plainly: pruneEntries already refuses an entry whose key fails validation before pruneOne runs, so no Revoked signal is emitted for a hostile key today and this redaction is the second line of defense behind that guard. The end-to-end test asserts the body stays out of telemetry either way; a direct recordRevoked call pins the redaction itself, which end-to-end cannot reach. Both are commented to say so, so neither reads as stronger than it is.

Also in scope, same theme

  • getSkill asserts content is a Uint8Array. toEqual alone passes for any structurally-equal value, including a plain array of the same numbers. (The Skill-vs-SkillReference runtime discriminator in skills-fs.ts was checked and is already correct — content instanceof Uint8Array.)
  • The SkillOutcomeReason exhaustiveness check named the union through types.js. It is the root export that is fixed — a union reachable only from the implementation module is not reachable by a supported import — so it now goes through the package index. The file's two root-export suites are merged while we are in here.
  • README and agents.md documented both constants as public API. Two further doc gaps the token change opened: the agents.md resolution-reason table had no row for a key mismatch, and the README's integrity_failure row promised an ld.skills.integrity_failure record for every case reaching that token, which the key-mismatch path does not write.

🤖 Generated with Claude Code, updated by @XieX


Note

Overview
This PR tightens Agent Skills public contracts and fail-closed behavior in @launchdarkly/ai-server, with tests rewritten to match (several previously asserted the wrong outcome).

Breaking API: SKILL_OBJECT_KIND and MAX_SKILL_CONTENT_BYTES are removed from the package barrel; only SKILL_FILENAME, MANIFEST_FILENAME, and MANIFEST_VERSION stay exported. Docs explain both constants remain internal to skills-core so callers do not semver-lock the size backstop or treat the store kind string as the wire format.

getSkillResult semantics: When the store returns verified content under a different key than requested, the outcome is now integrity_failure (not wrong_version), so fail-closed branching treats skill substitution like tampering. That path still does not emit ld.skills.integrity_failure or integrity telemetry — only hash/shape failures inside verifyRawSkill do. README and agents.md document the split.

Revoked signal: recordRevoked shape-checks manifest-sourced skill_key (invalid keys redacted) and omits version when it is not a valid integer instead of sending version: null. Prune passes raw manifest values through to the recorder.

Tests add export-absence assertions, key-mismatch silence checks, revoked telemetry cases, and getSkill Uint8Array typing; package-export suites are consolidated.

Reviewed by Cursor Bugbot for commit 937131d. Bugbot is set up for automated code reviews on this repo. Configure here.

…pin them

Each of the three was pinned the wrong way by an existing test, which is why all
three survived review. The test rewrites are the substance of this change as
much as the source fixes are; each one was verified by mutation.

**1. Unpublish `SKILL_OBJECT_KIND` and `MAX_SKILL_CONTENT_BYTES`.** Both were
re-exported from the package root. Neither belongs there, and their absence is
itself the contract:

- `SKILL_OBJECT_KIND` is an SDK-side seam string, not the wire format. It is
  what the accessors hand `SkillStore.getObject`, and an adapter is free to map
  it onto whatever its transport actually uses. Exporting it advertises a claim
  this side cannot make and could not walk back once a caller depended on it.
- `MAX_SKILL_CONTENT_BYTES` is a local enforcement bound on content the platform
  produces, set well above the platform's own limit precisely so that limit can
  move without this constant following. Publishing it semver-locks a number this
  side does not own, and a caller pre-flighting "will my skill fit?" against it
  reads the backstop rather than the real bound.

The test that should have caught this asserted the opposite — titled "the five
fixed values", with a comment claiming callers need the cap to pre-check
content, which is the exact reading the contract rejects. It now asserts the
three public constants' values and the two names' *absence*, mirroring
`test_skills.py`. Tests that need the cap keep reading it from `skills-core.js`,
which is where it lives.

**2. A store answering under a different key is `integrity_failure`.** It
returned `wrong_version`, which names a version mismatch specifically — there is
deliberately no `wrong_key` to parallel it. Content was delivered and its
identity did not verify, which is the one outcome a caller is expected to fail
closed on; filed under `wrong_version`, skill substitution was invisible to a
fail-closed caller. The JSDoc already described the intent correctly; only the
token was wrong.

The no-signal/no-log half is deliberate and unchanged: the check runs *after*
verification has passed, so the eight-token `reason_code` vocabulary does not
cover it and neither telemetry surface fires. The test now asserts that
asymmetry in both directions — the outcome is `integrity_failure`, the recording
emitter saw nothing, and no `ld.skills.integrity_failure` record was logged — and
branches on the token rather than matching `detail` text, which is for a human.

**3. `AgentControl Skill Revoked Received` omits `version` rather than emitting
null.** `recordRevoked` built its properties unconditionally, so a manifest
entry with a malformed version published `version: null`. It now adds the
property only for a valid version, and the caller passes the raw manifest value
through instead of pre-collapsing it to null.

`skill_key` is now redacted through `isValidSkillKey` there too, as Python's
`record_revoked` does. The key comes from the manifest — a file on disk anything
with write access to the root can author — so it is exactly as
attacker-controlled as a wire object. Reachability note: `pruneEntries` already
refuses an entry whose key fails validation before `pruneOne` runs, so this is
the second line of defense behind that guard; the end-to-end test asserts the
body stays out of telemetry either way, and a direct call pins the redaction
itself, which end-to-end cannot reach.

Also in scope, same theme:

- `getSkill` asserts `content` is a `Uint8Array`. `toEqual` alone passes for any
  structurally-equal value, including a plain array of the same numbers.
- The `SkillOutcomeReason` exhaustiveness check named the union through
  `types.js`. It is the *root* export that is fixed — a union reachable only
  from the implementation module is not reachable by a supported import — so it
  now goes through the package index. The file's two root-export suites are
  merged while we are in here.
- README and `agents.md` documented both constants as public API, and the
  resolution-reason table had no row for a key mismatch. The README's
  `integrity_failure` row also promised an `ld.skills.integrity_failure` record
  for every case reaching that token, which the key-mismatch path does not write.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@XieX XieX changed the title fix(client)!: Agent Skills — three contract fixes and the tests that pin them fix(client): Agent Skills — three contract fixes and the tests that pin them Sep 17, 2026
@XieX
XieX requested a review from andrewklatzke September 17, 2026 19:37
@XieX

XieX commented Sep 30, 2026

Copy link
Copy Markdown
Author

Collapsed into #65, which now carries this chain's six commits as a single review against xie/skills-fdv2-transport.

Nothing was rebased or dropped — the commits are byte-identical and this PR's commit is among them. Closing here because the chain was cleanly additive (one reworked line across all five PRs), so the five separate reviews only cost context on skills-core.ts, skills-fs.ts and agents.md without ever disagreeing.

This description stays the authoritative rationale for its part of the change — #65 links back here per commit rather than restating it.

@XieX XieX closed this Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant