Skip to content

feat: publish ShellRisk-Bench v0.1 - #1

Merged
hasandemirkiran merged 1 commit into
mainfrom
publish-shellrisk-bench
Aug 18, 2026
Merged

hasandemirkiran merged 1 commit into
mainfrom
publish-shellrisk-bench

Conversation

@hasandemirkiran

@hasandemirkiran hasandemirkiran commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • publish the six-source command benchmark with pinned, download-only adapters
  • document provenance, licensing, normalization, and the in-distribution split
  • release deterministic split checksums, scoring tools, aggregate results, and hashed k-cl-1 predictions
  • exclude classifier weights, serving code, product prompts, raw upstream data, and internal research tracks

Verification

  • Python unit tests pass
  • all six adapters smoke/build checked
  • split manifest reproduced exactly: 4,194 test commands (193 risky)
  • k-cl-1 predictions reproduce P=0.9468, R=0.9223, F1=0.9344

@hasandemirkiran
hasandemirkiran merged commit 0d41632 into main Aug 18, 2026
2 checks passed
@hasandemirkiran
hasandemirkiran deleted the publish-shellrisk-bench branch August 18, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant