Skip to content

docs(skill): build-once → run frozen for scheduler/HPC envs#37

Merged
Marius1311 merged 1 commit into
mainfrom
claude/competent-gould-6a98f4
Jun 26, 2026
Merged

docs(skill): build-once → run frozen for scheduler/HPC envs#37
Marius1311 merged 1 commit into
mainfrom
claude/competent-gould-6a98f4

Conversation

@Marius1311

Copy link
Copy Markdown
Member

Closes #35.

Problem

On scheduler/HPC clusters the skill's prescribed pixi run cta … workflow makes pixi solve + validate against one shared env prefix at runtime, on every call. Observed failure modes: the dependency solve OOM/SIGSEGVs on memory-capped head/login nodes; concurrent runs serialize on the per-call exclusive prefix lock (even cta … --help queues behind a heavy lock-holder); slow first invocations from re-resolution.

Fix (skill-side, scheduler-general)

The skill ships a committed pixi.lock, so runtime never needs to solve. This PR makes that the default and documents the scheduler path:

  • --frozen everywhere. pixi install --frozen / pixi run --frozen cta … is now the single prescribed invocation on every profile — runtime uses the locked env as-is and never solves. A one-line dev escape-hatch notes when to drop --frozen (intentional dependency changes, in a separate clone).
  • Build once, bypass the per-call lock (scheduler). SKILL.md §1c and a new "Build once, run frozen — the scheduler / HPC reality" subsection in reference/compute_environments.md cover: don't solve on a capped head node; build once where there's memory; for concurrent/batch use resolve the prefix once (CTA=$(pixi run --frozen which cta)) and call "$CTA" … directly; don't mutate shared env state mid-run; clear stale local .pixi/envs when using detached-environments.
  • Consistency. Updated the remaining prescribed-command spots (SKILL.md commands table, tool_registry.md bootstrap row, scheduler-handoff example).
  • No cluster specifics — phrased purely in generic scheduler terms (memory-capped head node + one shared, lockfile-frozen prefix); explicitly no module systems, proxies, or concrete paths.

Verification (off a head node)

Ran on a login node (eu-login-42):

  • pixi install --frozen74 MB peak RSS, exit 0 — links the committed env, does not run a memory-hungry solve.
  • pixi run --frozen cta --help → exit 0.
  • CTA=$(pixi run --frozen which cta) resolves and "$CTA" --help runs — the direct-bin concurrency path works.

Scope

Skill repo only. The cluster-specific operator steps (prebuild before fan-out, concrete detached-env scratch path, per-session prompt addendum) are tracked separately for quadbio/celltype-anno-bench docs/running-the-skill.md, per the issue.

🤖 Generated with Claude Code

On scheduler/HPC clusters the prescribed `pixi run cta …` workflow solves
and validates the env against a single shared prefix at runtime, causing
solve OOM/SIGSEGV on memory-capped head nodes, per-call lock contention
across concurrent runs, and slow first invocations.

Make runtime never solve and avoid the per-call shared lock:

- Prescribe `pixi install --frozen` / `pixi run --frozen cta …` as the
  single default on every profile (the skill ships a committed pixi.lock),
  with a dev escape-hatch note for intentional dependency changes.
- Document the scheduler path in SKILL.md §1c: build once where there is
  memory, and for concurrent/batch use bypass the per-call lock via
  `CTA=$(pixi run --frozen which cta)` then `"$CTA" …`; don't mutate shared
  env state mid-run; clear stale local `.pixi/envs` with detached envs.
- Add a "Build once, run frozen — the scheduler/HPC reality" subsection to
  reference/compute_environments.md covering the same, phrased generically
  (no module systems, proxies, or concrete cluster paths).
- Update the prescribed-command spots in the SKILL.md commands table,
  tool_registry.md, and the scheduler-handoff example for consistency.

Verified off a head node: `pixi install --frozen` links the committed env
without solving (~74 MB peak RSS), and `pixi run --frozen cta --help` plus
the direct-bin trick run cleanly.

Closes #35

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Marius1311 Marius1311 force-pushed the claude/competent-gould-6a98f4 branch from d4de028 to 98b915b Compare June 26, 2026 10:32
@Marius1311 Marius1311 merged commit ccfe2fb into main Jun 26, 2026
0 of 2 checks passed
@Marius1311 Marius1311 deleted the claude/competent-gould-6a98f4 branch June 26, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler/HPC env: build-once → run frozen (avoid runtime solves & shared-prefix lock contention)

1 participant