feat: add Hugging Face dataset export

README.md

@@ -90,10 +90,24 @@ Prediction rows contain a stable command hash and a binary verdict:
 {"id":"sha256:…","prediction":"risky"}
 ```
 
+## Hugging Face export
+
+Create the files used by the Hugging Face Dataset Viewer and
+`datasets.load_dataset()` from the verified local split:
+
+```bash
+.venv/bin/python -m shellrisk_bench.export_huggingface
+```
+
+This writes `dist/huggingface/README.md`, one Parquet file per split, the
+canonical split manifest, and an export manifest containing file hashes. The
+command does not upload anything. The dataset card is maintained under
+[`huggingface/README.md`](huggingface/README.md).
+
 ## Safety
 
 This repository processes potentially destructive commands as inert text. Nothing in the build or evaluation path executes benchmark commands. Do not pipe dataset contents into a shell.
 
 ## License
 
-The benchmark code is licensed under Apache-2.0. Upstream datasets retain their own licenses and terms; see [DATASETS.md](DATASETS.md). The generated dataset is intentionally git-ignored and is not redistributed here.
+The benchmark code is licensed under Apache-2.0. Upstream datasets retain their own licenses and terms; see [DATASETS.md](DATASETS.md). Generated data and Hub export files are git-ignored; publishing them requires a separate source-by-source redistribution review.

huggingface/README.md

@@ -0,0 +1,121 @@
+---
+pretty_name: ShellRisk-Bench
+language:
+- en
+license: other
+task_categories:
+- text-classification
+tags:
+- shell
+- bash
+- cybersecurity
+- safety
+- benchmark
+size_categories:
+- 10K<n<100K
+configs:
+- config_name: default
+  data_files:
+  - split: train
+    path: data/train-*.parquet
+  - split: test
+    path: data/test-*.parquet
+---
+
+# ShellRisk-Bench
+
+ShellRisk-Bench is a reproducible benchmark for context-free binary risk
+classification of individual shell-command submissions. It asks whether a
+command poses meaningful cyber or system risk when evaluated without task,
+user, or session context.
+
+**Release:** The v0.1 Parquet train and test splits are publicly available
+through Dataset Viewer and `load_dataset()`.
+
+The benchmark contains a deterministic train split of 16,772 rows and test
+split of 4,194 rows. The test set contains 193 `risky` and 4,001 `not_risky`
+examples.
+
+The source code, adapters, evaluation tooling, pinned revisions, and published
+results are available in the
+[ShellRisk-Bench GitHub repository](https://github.com/kontext-security/shellrisk-bench).
+
+## Usage
+
+```python
+from datasets import load_dataset
+
+dataset = load_dataset("kontext-security/ShellRisk-Bench")
+print(dataset["train"])
+print(dataset["test"])
+```
+
+Treat the `command` field as inert, untrusted text. Do not execute dataset
+contents or pipe them into a shell.
+
+## Schema
+
+| Field | Type | Description |
+|---|---|---|
+| `id` | string | Stable `sha256:` digest of the normalized command |
+| `source` | string | Source adapter key |
+| `upstream_id` | string | Row identifier assigned by the source adapter |
+| `command` | string | Normalized shell-command submission |
+| `label` | string | `risky` or `not_risky` |
+
+Single-line submissions remain whole, including pipelines and commands joined
+with `&&` or `;`. Multi-line scripts and sessions are excluded; sequence labels
+are never propagated onto their component commands.
+
+## Sources and labels
+
+The data is built from six pinned public sources:
+
+| Class | Source | Label basis |
+|---|---|---|
+| Not risky | [SWE-smith trajectories](https://huggingface.co/datasets/Kwai-Klear/SWE-smith-mini_swe_agent_plus-trajectories-66k) | Inferred from benign software-engineering tasks |
+| Not risky | [Terminal-Bench trajectories](https://huggingface.co/datasets/yoonholee/terminalbench-trajectories) | Inferred from benign terminal tasks |
+| Not risky | [nl2bash](https://github.com/TellinaTool/nl2bash) | Human-curated command corpus |
+| Risky | [Atomic Red Team](https://github.com/redcanaryco/atomic-red-team) | Executable ATT&CK tests |
+| Risky | [GTFOBins](https://github.com/GTFOBins/GTFOBins.github.io) | Curated binary-abuse techniques |
+| Risky | [InternalAllTheThings](https://github.com/swisskyrepo/InternalAllTheThings) | Curated offensive shell payloads |
+
+The benign trajectory labels are task-inferred rather than independently
+verified command-by-command. Risky labels follow the purpose of the upstream
+security collections. See the repository's
+[dataset provenance](https://github.com/kontext-security/shellrisk-bench/blob/main/DATASETS.md)
+for exact revisions and transformations.
+
+## Split construction
+
+The v0.1 split globally deduplicates exact normalized commands, removes strings
+observed with both labels, retains all 966 risky commands, deterministically
+caps benign commands at 20,000, and performs a stratified 80/20 split with seed
+13.
+
+The approximately 20:1 test mix is a constructed operating point. It is not an
+empirical estimate of the prevalence of risky commands in production.
+
+## Uses and limitations
+
+ShellRisk-Bench is intended to compare command-level classifiers and security
+guardrails on a fixed, auditable split. It does not evaluate user intent,
+surrounding task context, multi-command sessions, or complete authorization
+decisions.
+
+The headline split is same-source and in-distribution. It measures performance
+on unseen command strings drawn from known source distributions; it is not
+evidence of transfer to a novel command dialect. Keep the `source` field when
+performing source-grouped or leave-one-source-out analysis.
+
+## Licensing
+
+There is no single blanket license for the data. The benchmark code and
+documentation are Apache-2.0, while each upstream source retains its own terms.
+The pinned sources currently declare MIT, Apache-2.0, MIT, MIT, GPL-3.0, and no
+license file, respectively. Consult the source-specific links and notes in
+[DATASETS.md](https://github.com/kontext-security/shellrisk-bench/blob/main/DATASETS.md)
+before using or redistributing the data.
+
+The dataset is provided for security research and defensive evaluation. No
+Kestrel implementation, model weights, or training artifacts are included.

shellrisk_bench/export_huggingface.py

@@ -0,0 +1,115 @@
+"""Export the frozen ShellRisk-Bench split as a Hugging Face dataset repository."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import shutil
+from pathlib import Path
+
+from datasets import Dataset, load_dataset
+
+ROOT = Path(__file__).resolve().parent.parent
+DEFAULT_SPLIT_DIR = ROOT / "data" / "splits"
+DEFAULT_OUTPUT_DIR = ROOT / "dist" / "huggingface"
+CARD_PATH = ROOT / "huggingface" / "README.md"
+SPLITS = ("train", "test")
+
+
+def _sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def _jsonl_rows(path: Path) -> list[dict]:
+    with path.open(encoding="utf-8") as handle:
+        return [json.loads(line) for line in handle if line.strip()]
+
+
+def _verify_input(split_dir: Path, manifest: dict) -> None:
+    for split in SPLITS:
+        source_path = split_dir / f"{split}.jsonl"
+        if not source_path.exists():
+            raise FileNotFoundError(f"missing {source_path}")
+        expected = manifest.get(split, {}).get("sha256")
+        actual = _sha256(source_path)
+        if expected != actual:
+            raise ValueError(
+                f"{split} checksum mismatch: expected {expected!r}, got {actual!r}; "
+                "rebuild and verify the frozen split before exporting"
+            )
+
+
+def export(split_dir: Path = DEFAULT_SPLIT_DIR, output_dir: Path = DEFAULT_OUTPUT_DIR) -> dict:
+    """Write a viewer-compatible dataset repository without uploading it."""
+    manifest_path = split_dir / "manifest.json"
+    if not manifest_path.exists():
+        raise FileNotFoundError(f"missing {manifest_path}")
+    manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+    _verify_input(split_dir, manifest)
+
+    data_dir = output_dir / "data"
+    data_dir.mkdir(parents=True, exist_ok=True)
+    shutil.copyfile(CARD_PATH, output_dir / "README.md")
+    shutil.copyfile(manifest_path, output_dir / "split-manifest.json")
+
+    exported: dict[str, dict] = {}
+    for split in SPLITS:
+        rows = _jsonl_rows(split_dir / f"{split}.jsonl")
+        expected_rows = manifest[split]["n"]
+        if len(rows) != expected_rows:
+            raise ValueError(
+                f"{split} row count mismatch: expected {expected_rows}, got {len(rows)}"
+            )
+        output_path = data_dir / f"{split}-00000-of-00001.parquet"
+        temporary_path = output_path.with_suffix(".parquet.tmp")
+        try:
+            Dataset.from_list(rows).to_parquet(temporary_path)
+            temporary_path.replace(output_path)
+        finally:
+            temporary_path.unlink(missing_ok=True)
+        exported[split] = {
+            "path": str(output_path.relative_to(output_dir)),
+            "rows": len(rows),
+            "sha256": _sha256(output_path),
+        }
+
+    export_manifest = {
+        "benchmark": manifest["benchmark"],
+        "version": manifest["version"],
+        "source_manifest_sha256": _sha256(manifest_path),
+        "files": exported,
+    }
+    (output_dir / "export-manifest.json").write_text(
+        json.dumps(export_manifest, indent=2, sort_keys=True) + "\n",
+        encoding="utf-8",
+    )
+    return export_manifest
+
+
+def verify_loadable(output_dir: Path = DEFAULT_OUTPUT_DIR) -> dict[str, int]:
+    """Load the exported Parquet files using the public consumer API."""
+    files = {
+        split: str(output_dir / "data" / f"{split}-00000-of-00001.parquet")
+        for split in SPLITS
+    }
+    dataset = load_dataset("parquet", data_files=files)
+    return {split: dataset[split].num_rows for split in SPLITS}
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--split-dir", type=Path, default=DEFAULT_SPLIT_DIR)
+    parser.add_argument("--output-dir", type=Path, default=DEFAULT_OUTPUT_DIR)
+    args = parser.parse_args()
+    exported = export(args.split_dir, args.output_dir)
+    loaded = verify_loadable(args.output_dir)
+    print(json.dumps({"export": exported, "loaded_rows": loaded}, indent=2, sort_keys=True))
+
+
+if __name__ == "__main__":
+    main()

tests/test_export_huggingface.py

@@ -0,0 +1,64 @@
+import hashlib
+import json
+from pathlib import Path
+
+import pytest
+
+from shellrisk_bench.export_huggingface import export, verify_loadable
+
+
+def _write_jsonl(path: Path, rows: list[dict]) -> str:
+    path.write_text(
+        "".join(json.dumps(row, sort_keys=True) + "\n" for row in rows),
+        encoding="utf-8",
+    )
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def _row(index: int, label: str) -> dict:
+    return {
+        "id": f"sha256:{index:064x}",
+        "source": "fixture",
+        "upstream_id": f"fixture-{index:06d}",
+        "command": f"printf fixture-{index}",
+        "label": label,
+    }
+
+
+def _split_fixture(tmp_path: Path) -> Path:
+    split_dir = tmp_path / "splits"
+    split_dir.mkdir()
+    train = [_row(1, "not_risky"), _row(2, "risky")]
+    test = [_row(3, "not_risky")]
+    train_sha = _write_jsonl(split_dir / "train.jsonl", train)
+    test_sha = _write_jsonl(split_dir / "test.jsonl", test)
+    manifest = {
+        "benchmark": "ShellRisk-Bench",
+        "version": "0.1.0",
+        "train": {"n": len(train), "sha256": train_sha},
+        "test": {"n": len(test), "sha256": test_sha},
+    }
+    (split_dir / "manifest.json").write_text(json.dumps(manifest), encoding="utf-8")
+    return split_dir
+
+
+def test_export_is_loadable_by_datasets(tmp_path: Path) -> None:
+    split_dir = _split_fixture(tmp_path)
+    output_dir = tmp_path / "hub"
+
+    manifest = export(split_dir, output_dir)
+
+    assert manifest["files"]["train"]["rows"] == 2
+    assert manifest["files"]["test"]["rows"] == 1
+    assert verify_loadable(output_dir) == {"train": 2, "test": 1}
+    assert (output_dir / "README.md").exists()
+    assert (output_dir / "split-manifest.json").exists()
+
+
+def test_export_rejects_checksum_mismatch(tmp_path: Path) -> None:
+    split_dir = _split_fixture(tmp_path)
+    with (split_dir / "test.jsonl").open("a", encoding="utf-8") as handle:
+        handle.write(json.dumps(_row(4, "risky")) + "\n")
+
+    with pytest.raises(ValueError, match="test checksum mismatch"):
+        export(split_dir, tmp_path / "hub")