0.2.0 wheel: missing prompt .md data files + Windows temp-dir cleanup crash (claude backend)

[ray8875]

Thanks for open-sourcing SkillOpt! While setting up 0.2.0 on Windows with the Claude backend I hit two blocking bugs. Bug 1 affects all platforms; Bug 2 is Windows-specific. A third, minor Windows note is at the end.

Environment

• skillopt 0.2.0, installed from PyPI: pip install "skillopt[claude]"
• Python 3.14.5, Windows 10
• Backend: --backend claude (claude-agent-sdk; Claude Code CLI 2.1.173)

---

Bug 1 — PyPI wheel ships no prompt files (skillopt/prompts/*.md missing) — all platforms

The wheel installs skillopt/prompts/ containing only __init__.py; every generic prompt .md is missing, as are the per-env prompts under skillopt/envs/<env>/prompts/. Any run fails as soon as it needs a prompt:

FileNotFoundError: Prompt 'analyst_error' not found. Searched: skillopt/prompts/analyst_error.md

(The rollout stage fails the same way on rollout_system.)

Repro:

pip install skillopt
python -c "import os, skillopt; p=os.path.join(os.path.dirname(skillopt.__file__),'prompts'); print(sorted(os.listdir(p)))"
# -> ['__init__.py', '__pycache__']   # all *.md missing

Root cause: the .md files aren't declared as package data, so they're excluded from the wheel; load_prompt() (skillopt/prompts/__init__.py) then can't resolve them. The repo tree has all ~21 generic prompts + per-env prompts, so a source / editable install works — only the wheel is broken.

Suggested fix (setuptools / pyproject):

[tool.setuptools.package-data]
"*" = ["*.md"]

or an equivalent MANIFEST.in (recursive-include skillopt *.md) with include_package_data. A test that imports each registered env and asserts load_prompt(...) resolves would catch regressions.

---

Bug 2 — Windows: temp-dir cleanup race in the Claude backend → WinError 32

skillopt/model/claude_backend.py::_run_claude_print runs the CLI inside a TemporaryDirectory and passes it as the child's cwd:

with tempfile.TemporaryDirectory(prefix="skillopt_claude_") as temp_dir:   # ~L246
    ...
    proc = subprocess.run(cmd, ..., cwd=temp_dir)                          # ~L273

On Windows the spawned claude / node process still holds a handle on temp_dir when the with block exits, so TemporaryDirectory cleanup raises and the exception propagates out of the model call (masking the real result):

RuntimeError: Claude backend failed after N retries:
[WinError 32] The process cannot access the file because it is being used by another process:
'...\Temp\skillopt_claude_xxxxxxxx'

This makes the claude backend unusable on Windows. Since requires-python >= 3.10, ignore_cleanup_errors=True fixes it cleanly:

with tempfile.TemporaryDirectory(prefix="skillopt_claude_", ignore_cleanup_errors=True) as temp_dir:

(Alternatives: don't set the child cwd to the temp dir, or retry the cleanup.)

---

Also on Windows (minor) — Unicode print crashes under cp1252

The trainer prints non-ASCII (e.g. the → in the [gate] line, skillopt/engine/trainer.py), which crashes under the default Windows code page before any model call:

UnicodeEncodeError: 'charmap' codec can't encode character '→' ... cp1252

PYTHONUTF8=1 works around it; reconfiguring stdout to UTF-8 (or using ASCII in logs) would fix it for everyone.

---

With all three addressed, an end-to-end skillopt-train --backend claude run completes cleanly (EXIT=0). Happy to send a PR for Bugs 1 & 2 if useful.

[ray8875]

Possible additional Windows issue: item IDs containing : can fail path creation (WinError 267)

On Windows, a rollout item whose ID contains a colon can fail before any model call, because the ID is used as a filesystem path component. LiveMathematicianBenchDataLoader derives IDs as f"{month}:{no}" (e.g. 2025-01:5; a toy dataset with month: "smoke" gives IDs like smoke:2), and the rollout creates a per-item directory:

pred_dir = os.path.join(out_root, "predictions", item_id)  # ...\predictions\2025-01:5
os.makedirs(pred_dir, exist_ok=True)

Since : is not valid in Windows path names, os.makedirs raises [WinError 267] The directory name is invalid. process_one catches it and records it as the item's fail_reason, so the item scores 0 before the model is ever called (agent_ok=false). IDs like 2025-01:5 (real LiveMathematicianBench data) or smoke:2 (toy data) are therefore unsafe as path components on Windows.

In my case, on a small toy dataset, removing the colon-producing month field (so IDs became 1..6) avoided it and the run completed normally.

A durable fix would likely be to keep the original ID for reporting/output, but use a filesystem-safe slug when creating predictions/<id> — e.g. replacing characters that are invalid on Windows (< > : " / \ | ? *) before building the path. Happy to help if useful.

[ray8875]

Follow-up from running gated SkillOpt training end-to-end on Windows against the official LiveMathematicianBench split. Three findings.

1. A Windows-safe ID fix must use the same deterministic mapping at every write and read site

In unmodified SkillOpt, LiveMathematicianBench IDs such as 202602:22 are used directly as prediction-directory components, so Windows fails at directory creation with WinError 267.

During development of a local fix, I initially sanitized only the rollout write path. That removed the directory-creation failure, but exposed a second integration requirement: reflection and slow-update readers still looked under predictions/<raw-id> while rollout wrote under predictions/<sanitized-slug>.

The resulting path mismatch caused:

• empty trajectory text;
• reflection returning before any optimizer call;
• approximately zero seconds of reflection time;
• zero analyst/optimizer tokens;
• every optimization step being skipped.

Therefore, the upstream fix should not sanitize only the rollout writer. It should provide one shared deterministic ID-to-path-component function and use it consistently at every location that writes or reads per-item artifacts, including rollout, reflection, slow update, top-up, and longitudinal readers.

Preserve the original ID unchanged in result records and scoring.

2. _load_json in the LiveMathematicianBench dataloader doesn't force UTF-8

skillopt/envs/livemathematicianbench/dataloader.py::_load_json does
with open(path) as f: return json.load(f). On Windows the default encoding is cp1252, and the official qa_*_final.json files contain non-ASCII (math) characters, so load_items (ratio mode / materialization) fails:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position ...

Fix:

with open(path, "r", encoding="utf-8") as f:
    return json.load(f)

A targeted test that loads a small JSON with non-ASCII content would guard this. (Other open(...) JSON reads/writes without an explicit encoding= are likely affected on Windows too — e.g. conversation.json writers.)

3. Collision-guarantee caveat for the slug approach

A natural fix is: replace invalid characters, then append a short deterministic hash of the original ID only when sanitization changed it (so already-safe IDs stay byte-identical on disk). This distinguishes the common case — two distinct unsafe IDs that sanitize to the same base get different hashes — but it does not mathematically rule out a collision between an already-safe ID and the slug generated for a different, unsafe ID (e.g. a real ID that happens to equal 202602_22_<hash8>). Astronomically unlikely, but not guaranteed.

If a hard guarantee is wanted, either:

• always append the deterministic hash (IDs are then no longer human-identical on disk), or
• maintain a per-run slug→ID registry and fail closed if two distinct IDs map to the same slug.

Environment / provenance

• SkillOpt version tested: 0.2.0, editable install from source at upstream commit e4ea6a6 ("chore(release): v0.2.0"); the PyPI 0.2.0 wheel is equivalently affected. Windows 10, Python 3.14, claude backend (skillopt[claude]), target = haiku.
• Benchmark data: LiveMathematicianBench (Hugging Face) revision b72450f6ce96c26158d64d945a5d31ef7727be41.
• Local patch commits (on a local windows-fixes branch, not pushed):
  • 0bc644d — initial write-side ID sanitization.
  • f8636fd — read-side consistency correction (reflection + slow-update readers).
• The corrected implementation was exercised end-to-end on the full official 35/18/124 LiveMathematicianBench split (gated training plus a held-out 124-item test evaluation) and is covered by a targeted path regression suite of 8 tests, all passing.

Happy to send a PR covering (1) and (2).