Skip to content

docs: link published Kestrel model #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hasandemirkiran merged 1 commit into from
Aug 19, 2026
Merged

docs: link published Kestrel model #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ The approximately 20:1 test mix is a constructed operating point for comparing p
| Shieldstral 1.0 (3B, local) | 0.406 | 0.269 | 0.324 | 186 ms |
| Llama Guard 4 (12B) | 0.023 | 0.285 | 0.042 | 1.1 s |

All systems were scored on the same 4,194 commands. Hosted-model latency was measured sequentially and includes the API round trip. Shieldstral used its default 0.5 threshold; Llama Guard counted any unsafe category as risky. Quality results, prompts, and the Kestrel per-example verdicts are under [`results/`](results/). The Kestrel implementation and weights are not part of this benchmark repository.
All systems were scored on the same 4,194 commands. Hosted-model latency was measured sequentially and includes the API round trip. Shieldstral used its default 0.5 threshold; Llama Guard counted any unsafe category as risky. Quality results, prompts, and the Kestrel per-example verdicts are under [`results/`](results/). The Kestrel implementation is not part of this benchmark repository. The portable Kestrel v0.1.0 model artifact is published separately on Hugging Face at the [pinned release revision](https://huggingface.co/kontext-security/Kestrel/tree/b3a600213c0b30ce4d4e2c59fe85978e1b416a96) and is not duplicated here. Its SHA-256 checksum is `1df8b3e5f2bfc4e1fe95230ee9b3d37f63aaa461a8551982fcc0dd7103c8221b`.

## Build

Expand Down
2 changes: 1 addition & 1 deletion results/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

`results.json` contains the aggregate comparison reported with ShellRisk-Bench v0.1. `kestrel.predictions.jsonl` contains one binary Kestrel verdict for each item in the deterministic test split, keyed by the SHA-256 identifier generated by `shellrisk_bench.prepare`.

The classifier implementation and weights are intentionally not included. The prediction artifact makes its reported confusion matrix and quality metrics independently auditable after rebuilding the upstream data.
The classifier implementation and model artifact are intentionally not included here. The portable Kestrel v0.1.0 model artifact is published separately on Hugging Face at the [pinned release revision](https://huggingface.co/kontext-security/Kestrel/tree/b3a600213c0b30ce4d4e2c59fe85978e1b416a96), with SHA-256 checksum `1df8b3e5f2bfc4e1fe95230ee9b3d37f63aaa461a8551982fcc0dd7103c8221b`. The prediction artifact makes its reported confusion matrix and quality metrics independently auditable after rebuilding the upstream data.

## Evaluation prompt

Expand Down
Loading