Skip to content

feat(eval): add hashed result provenance records - #1834

Merged
EmersonBraun merged 1 commit into
mainfrom
feat/eval-result-provenance
Oct 8, 2026
Merged

EmersonBraun merged 1 commit into
mainfrom
feat/eval-result-provenance

Conversation

@EmersonBraun

Copy link
Copy Markdown
Member

Summary

Add a generic result record at @agentskit/eval/provenance with suite, frozen case-set, subject revision/digest, runner version, and JSON metrics. Canonical serialization and SHA-256 support deterministic reruns and integrity verification; optional timestamps remain in the envelope but do not enter its content digest.

Add precision/recall helpers with explicit zero-denominator behavior, consumer documentation, a versioned record specification, and a minor changeset. Update only eval's public API snapshot entry.

Evidence

  • pnpm exec turbo run build --filter=@agentskit/eval --concurrency=1: eval plus its two dependencies passed.
  • pnpm --filter @agentskit/eval exec vitest run --coverage --maxWorkers=1: 115 tests passed; 96.95% line coverage (95% required). Includes ESM/CJS and browser/native bundler fixtures.
  • Eval, example consumer, and docs consumer lint passed; the docs dependency closure was built serially to supply declarations.
  • Scoped public API and packed-consumer gates passed: 3 packages, 31 subpaths, ESM/CJS 31/31, TypeScript 5.9 and 6.0 bundler/NodeNext. Every non-eval snapshot entry remains byte-identical.
  • Eval JSDoc ratchet: 106/106 documented symbols. Core-rules, cross-platform, source/test parity, named exports, no-any, typed-error, file-size, docs coverage, locale/index/version, and README ratchets passed.
  • Root bundle: 1,290 bytes gzip within the unchanged 1.5 KB target. New provenance bundle: 768 bytes within its derived 1 KB target; other size targets unchanged.
  • Locally regenerated documentation index, gates, and conformance passed; generated index/capabilities were restored and excluded from the commit.

Validation is scoped to eval and direct consumers. No full monorepo suite, production docs build, rendered docs visual review, release, or deployment was performed.

Reuse

Reuse canonicalJson and sha256Hex from @agentskit/core/hash, the existing shared hash implementation, and existing consumer fixtures. No new dependency, judging pipeline, or storage abstraction is introduced. The universal provenance subpath preserves the existing root bundle budget.

The digest checks integrity, not authenticity or the truth of caller-supplied artifact identity. Omitting timestamps gives byte-identical envelopes across reruns.

@EmersonBraun EmersonBraun reopened this Oct 8, 2026
@EmersonBraun
EmersonBraun force-pushed the feat/eval-result-provenance branch from 151c17e to 7e9f42f Compare October 8, 2026 06:33
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Lighthouse

URL Performance Accessibility Best practices SEO
http://127.0.0.1:3000/ 50 100 96 100
http://127.0.0.1:3000/docs 85 100 96 100
http://127.0.0.1:3000/docs/get-started/getting-started/quickstart 83 100 96 100
http://127.0.0.1:3000/stack 96 100 96 100
http://127.0.0.1:3000/showcase 91 100 96 100

Run by .github/workflows/lighthouse.yml.

@EmersonBraun
EmersonBraun merged commit cff4ba9 into main Oct 8, 2026
19 checks passed
@EmersonBraun
EmersonBraun deleted the feat/eval-result-provenance branch October 8, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant