test(client): integrity verification and the accessor telemetry sweep - #32
Merged
XieX merged 2 commits intoSep 18, 2026
Merged
Conversation
This was referenced Aug 25, 2026
XieX
force-pushed
the
xie/skills-04-verification-tests
branch
2 times, most recently
from
August 28, 2026 20:30
9c80f28 to
422c379
Compare
XieX
force-pushed
the
xie/skills-04-verification-tests
branch
from
August 31, 2026 18:10
422c379 to
83b221a
Compare
XieX
force-pushed
the
xie/skills-04-verification-tests
branch
from
September 14, 2026 20:13
83b221a to
a718d95
Compare
XieX
marked this pull request as ready for review
September 14, 2026 20:34
|
|
||
| class ThrowingEmitter { | ||
| record(): void { | ||
| throw new Error('emitter exploded'); |
andrewklatzke
approved these changes
Sep 14, 2026
Splitting these out of the accessor commit rather than reviewing 1,500 lines at once. Both suites test code that already landed; they are the adversarial half of it. Integrity verification covers what a hostile or broken store can serve: a content hash that does not match, a hash of the wrong shape, content over the 64 KB cap, a key or version that fails revalidation, and the UTF-8 round-trip case where `TextEncoder` substitutes U+FFFD for an unpaired surrogate. That last one pins `contentHash` to the sha256 of the *substituted* bytes, so the hash comparison cannot be what rejects it — only the round-trip guard can be, which is what makes the test fail against an implementation missing it. The telemetry suite asserts the seam's shape rather than any one signal. The allowlist arrives with it: a sweep over every recorded name proves the set is exactly three and that the two names this SDK must never emit — `AgentControl Skill SDK Reference Returned` and `AgentControl Skill Content Retrieved` — appear nowhere. Also covered: no skill content and no filesystem path reaches a signal, and an emitter that throws cannot fail the operation it was observing. Client package: 395 -> 433 tests. typecheck and biome clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the regression test for the BOM fix in the previous commit, placed next to the lone-surrogate test it is the counterpart of: authentic content beginning with U+FEFF verifies and returns its exact bytes. Fails without `ignoreBOM: true`. Comment sweep, per review feedback: the vocabulary and key-order tests now describe their cross-SDK contract without naming the Python SDK, and the surrogate test explains the U+FFFD substitution as a Node behavior rather than by contrast with str.encode. No assertions changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
XieX
force-pushed
the
xie/skills-04-verification-tests
branch
from
September 16, 2026 18:26
a718d95 to
8644e92
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourth of seven. Test-only — this is the adversarial half of the previous PR, split out rather than asking anyone to review 1,500 lines in one sitting.
Integrity verification
Covers what a hostile or broken store can serve: a content hash that doesn't match, a hash of the wrong shape, content over the size cap (measured on the encoded bytes), and a key or version that fails revalidation. Verified content comes back as the exact bytes that were hashed — the multi-byte UTF-8 case asserts byte equality against the
TextEncoderencoding, not string equality.The case worth reading carefully is the UTF-8 round-trip one.
TextEncodersubstitutes U+FFFD for an unpaired surrogate rather than throwing, so the test pinscontentHashto the sha256 of the substituted bytes. That means the hash comparison cannot be what rejects it — only the round-trip guard can be, which is what makes the test fail against an implementation missing that guard.Telemetry sweep
Asserts the seam's shape rather than any one signal:
AgentControl Skill SDK Reference ReturnedandAgentControl Skill Content Retrieved— appear nowhere.getSkillResult(security review LA-2)One case per reason token, each store shaped to reach a different construction site in
resolveFromStore, so the mapping is under test and not just the union. Plus:store_unavailableis distinct fromabsent— a throwing store and an empty store must not read the same, or a caller cannot tell "retry" from "never configured".getObject: the store would answer with the newer version, the equality check would refuse it, and the outcome would reportwrong_versionfor a pin the store could have satisfied.getSkillstill resolves tonull, and still never rejects, for all four failure outcomes — the no-behaviour-change guarantee.ld.skills.integrity_failurerecord and one signal per failed retrieval, so the new accessor provably does not double-log.detailnever contains the skill content, including when a hostile store routes the body throughcontentHash.createSkillOutcomereturns a frozen object, like every sibling factory.Verification
Client package 395 → 476 tests.
typecheckandbiomeclean.🤖 Generated with Claude Code
Note
Overview
Test-only expansion of
skills.test.ts(~900 lines) that adversarially pins skill accessor security and observability contracts from the prior implementation PR.Adds
integrity verificationcases for hostile store payloads: hash mismatch, size cap on encoded bytes, key/version revalidation, UTF-8 round-trip (unpaired surrogates, BOM), and stores that throw (mapped to empty/null results).Adds
telemetry seamchecks: integrity signal shape and redaction (no skill body in signals or smuggled via key/hash), allowlist of three signal names and explicit absence of removed signals, noclient.trackfrom accessors, and fault-tolerant emitters.Adds
ld.skills.integrity_failurelog record contract tests: eightreason_codevalues, stable JSON field set and key ordering, redaction, and logging even without telemetry.Adds
getSkillResultcoverage distinguishingabsentvsintegrity_failurevsstore_unavailablevswrong_version, version pinning throughgetObject, unchangedgetSkillnull semantics, no double-logging, safedetail, pluscreateSkillOutcomeimmutability and package export checks for the fiveSkillOutcomeReasontokens.Reviewed by Cursor Bugbot for commit 8644e92. Bugbot is set up for automated code reviews on this repo. Configure here.