Skip to content

chore: release - #368

Merged
joshblack merged 1 commit into
mainfrom
changeset-release/main
Oct 12, 2026
Merged

joshblack merged 1 commit into
mainfrom
changeset-release/main

Conversation

@primer

@primer primer Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@primer/agent-eval@0.13.0

Minor Changes

  • 72742af: Require runner on parsed experiment and benchmark trial outputs. Legacy output
    files without a runner default to copilot-cli when parsed. Consumers constructing
    trial output objects directly must include runner: 'copilot-cli' or
    runner: 'copilot-sdk'.

  • a5d1340: Add support for running benchmark experiments with the benchmark option.

    Calls to getExperiment and listExperiments now require benchmarksDirectory,
    including when loading only scenario-backed experiments.

Patch Changes

  • 922afff: Fix output-token totals for Copilot CLI and SDK trials by collecting usage
    statistics instead of relying on message events that no longer include token
    counts. Preserve compatibility with older session output.
  • 7fd247f: Validate manifest consistency through schemas before loading configurations or
    trial artifacts. Reject metadata IDs that differ from their manifest keys,
    duplicate benchmark capability scenario IDs, and conflicting metadata across
    benchmark result shards. Duplicate trial IDs and other merge conflicts now report
    schema errors with field paths.

@primer
primer Bot force-pushed the changeset-release/main branch 5 times, most recently from f7a6aa2 to b7d91f7 Compare October 12, 2026 16:29
@primer
primer Bot force-pushed the changeset-release/main branch from b7d91f7 to b953ec3 Compare October 12, 2026 16:32
@joshblack
joshblack merged commit 79fafdd into main Oct 12, 2026
8 checks passed
@joshblack
joshblack deleted the changeset-release/main branch October 12, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant