docs(skill): build-once → run frozen for scheduler/HPC envs#37
Merged
Conversation
On scheduler/HPC clusters the prescribed `pixi run cta …` workflow solves and validates the env against a single shared prefix at runtime, causing solve OOM/SIGSEGV on memory-capped head nodes, per-call lock contention across concurrent runs, and slow first invocations. Make runtime never solve and avoid the per-call shared lock: - Prescribe `pixi install --frozen` / `pixi run --frozen cta …` as the single default on every profile (the skill ships a committed pixi.lock), with a dev escape-hatch note for intentional dependency changes. - Document the scheduler path in SKILL.md §1c: build once where there is memory, and for concurrent/batch use bypass the per-call lock via `CTA=$(pixi run --frozen which cta)` then `"$CTA" …`; don't mutate shared env state mid-run; clear stale local `.pixi/envs` with detached envs. - Add a "Build once, run frozen — the scheduler/HPC reality" subsection to reference/compute_environments.md covering the same, phrased generically (no module systems, proxies, or concrete cluster paths). - Update the prescribed-command spots in the SKILL.md commands table, tool_registry.md, and the scheduler-handoff example for consistency. Verified off a head node: `pixi install --frozen` links the committed env without solving (~74 MB peak RSS), and `pixi run --frozen cta --help` plus the direct-bin trick run cleanly. Closes #35 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
d4de028 to
98b915b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #35.
Problem
On scheduler/HPC clusters the skill's prescribed
pixi run cta …workflow makes pixi solve + validate against one shared env prefix at runtime, on every call. Observed failure modes: the dependency solve OOM/SIGSEGVs on memory-capped head/login nodes; concurrent runs serialize on the per-call exclusive prefix lock (evencta … --helpqueues behind a heavy lock-holder); slow first invocations from re-resolution.Fix (skill-side, scheduler-general)
The skill ships a committed
pixi.lock, so runtime never needs to solve. This PR makes that the default and documents the scheduler path:--frozeneverywhere.pixi install --frozen/pixi run --frozen cta …is now the single prescribed invocation on every profile — runtime uses the locked env as-is and never solves. A one-line dev escape-hatch notes when to drop--frozen(intentional dependency changes, in a separate clone).reference/compute_environments.mdcover: don't solve on a capped head node; build once where there's memory; for concurrent/batch use resolve the prefix once (CTA=$(pixi run --frozen which cta)) and call"$CTA" …directly; don't mutate shared env state mid-run; clear stale local.pixi/envswhen usingdetached-environments.tool_registry.mdbootstrap row, scheduler-handoff example).Verification (off a head node)
Ran on a login node (
eu-login-42):pixi install --frozen→ 74 MB peak RSS, exit 0 — links the committed env, does not run a memory-hungry solve.pixi run --frozen cta --help→ exit 0.CTA=$(pixi run --frozen which cta)resolves and"$CTA" --helpruns — the direct-bin concurrency path works.Scope
Skill repo only. The cluster-specific operator steps (prebuild before fan-out, concrete detached-env scratch path, per-session prompt addendum) are tracked separately for
quadbio/celltype-anno-benchdocs/running-the-skill.md, per the issue.🤖 Generated with Claude Code