Skip to content

Hand off final_model to the verifier via a Harbor shared volume instead of the workspace tar - #66

Open
surelyMersad wants to merge 1 commit into
aisa-group:add_harbor_supportfrom
surelyMersad:harbor-shared-volume-verifier
Open

Hand off final_model to the verifier via a Harbor shared volume instead of the workspace tar#66
surelyMersad wants to merge 1 commit into
aisa-group:add_harbor_supportfrom
surelyMersad:harbor-shared-volume-verifier

Conversation

@surelyMersad

Copy link
Copy Markdown

Builds on #8. The trained weights currently travel inside the single workspace
artifact: tarred in the agent env, downloaded to the host, re-uploaded into the
separate verifier env. That's the transfer path that died with tar exit code -1 /
empty stderr on multi-GB workspaces (2026-05-27 and 2026-06-27 trials, see the
task.toml comments) and forced the aggressive-excludes workaround — while still
carrying the multi-GB final_model through it.

What this PR does

final_model becomes a Harbor task-declared shared volume
([[environment.volumes]], name model):

Container Mount path Access
Agent /home/agent/workspace/final_model read-write
Verifier /mnt/model read-only

Harbor creates one volume per trial (harbor-<trial>-model), both containers
attach the same volume, and Harbor deletes it at trial cleanup. On Modal the
weights move as a Modal Volume, locally as
a Docker named volume — never through the tar/host path.

Consequences:

  • The workspace artifact now carries only the contamination-judge inputs (the
    agent's source code) — final_model and stray weight-file patterns
    (*.safetensors, *.pt, *.ckpt, …) are added to the excludes, so the
    agent-side tar stays small regardless of model size.
  • The verifier's copy of the weights is read-only: the verification phase
    cannot modify the evidence it grades. The mountpoint can't be deleted, renamed,
    or unmounted by the agent — worst case it empties its own deliverable, which
    test.sh scores as reward 0.
  • Weights are no longer archived under artifacts/workspace/final_model/ on the
    host. Run with harbor's delete=False if you need to inspect weights after a
    trial.

Changes (4 files, +89/−45)

  • template/task.toml — volume declarations on [environment] and
    [verifier.environment]; final_model + weight-file patterns added to the
    workspace-artifact excludes; comments updated (the "final_model is deliberately
    kept in the tar" rationale is retired).
  • template/tests/test.sh — evaluate against read-only /mnt/model; an empty
    volume is treated as "no model" (the mountpoint always exists, so the old -d
    check alone would pass vacuously); tamper-resistance notes updated.
  • template/instruction.md — one added line: final_model is a pre-created
    mounted volume; write into it (save_pretrained("final_model") / copy files
    in), don't delete or rename the folder itself. Avoids agents hitting a
    confusing EBUSY on the common rm -rf final_model && mv checkpoint final_model pattern.
  • README.md — "Model Hand-off via Shared Volume" section + harbor install pin.

No changes to adapter.py or tests/Dockerfile — the existing separate-verifier
machinery (pristine /tests image, _copy_eval_files, judge-from-/tests) is
untouched.

Validated end-to-end on Modal

Full trial of posttrainbench-gsm8k-qwen3-1.7b (claude-code / claude-sonnet-4-6,
agent budget reduced to 30 min for the test): 31m 9s total, zero exceptions,
reward 0.207
.

Check Result
Per-trial volume harbor-fb557659f03e-model observed live on Modal mid-run; deleted after cleanup (Volume.objects.list() empty)
Agent phase (21m 21s) Downloaded the base model, ran a real 2-epoch SFT, saved to final_model on the volume
Workspace artifact 60 KB (source code only — down from multi-GB with weights in the tar)
Verifier phase (9m 36s) Own sandbox, vLLM loaded the model from read-only /mnt/model, full 1,319-sample gsm8k eval
Score parity with the condor pipeline accuracy 0.2070 ± 0.011 vs the official Qwen3-1.7B base-model gsm8k score of 20.4 (leaderboard, HTCondor pipeline) — a 0.3-point difference, well inside the eval's own ±1.1 stderr. The 30-min minimal SFT is expected to move the score barely, so landing this close to the base-model number is evidence the volume-fed eval reproduces the original pipeline's scoring, not an artifact of a different eval path
Cached re-run cost Environment setup drops from 9m50s (cold image build) to 6 s
Failure path (from an earlier run where the agent errored) Verifier correctly scored the empty volume as reward 0 via the new missing/empty check

Dependency

Requires Harbor's task-declared shared volumes feature:
harbor-framework/harbor#2603. Until it lands:

uv tool install "harbor @ git+https://github.com/surelyMersad/harbor.git@feat/task-shared-volumes"

The feature is capability-gated in Harbor (docker + modal direct-sandbox mode);
tasks fail fast with a clear error on unsupported providers. Upstream validation:
5,434 unit tests passing (40 new for this feature), plus live smoke trials on both
docker and modal.

…ad of the workspace tar

The trained weights previously traveled inside the single workspace
artifact: tarred in the agent env, downloaded to the host, re-uploaded
into the separate verifier env. On multi-GB workspaces this is the
transfer path that died with tar exit code -1 / empty stderr (2026-05-27
and 2026-06-27 trials) and forced the aggressive-excludes workaround.

final_model is now a harbor task-declared shared volume
([[environment.volumes]], name "model"): the agent writes weights to
the volume mounted read-write at /home/agent/workspace/final_model, the
verifier attaches the SAME per-trial volume read-only at /mnt/model, and
harbor creates/deletes the volume per trial. On Modal the weights move
as a Modal Volume, locally as a Docker named volume — never through the
tar/host path.

- task.toml: volume declarations on both [environment] and
  [verifier.environment]; final_model (and stray weight-file patterns)
  added to the workspace-artifact excludes, so the tar now carries only
  the contamination-judge inputs (agent source code).
- tests/test.sh: evaluate against read-only /mnt/model; treat an empty
  volume as "no model"; updated tamper-resistance notes.
- instruction.md: note that final_model is a pre-created mounted volume
  (write into it; don't delete/rename the mountpoint).
- README: model hand-off section + harbor install pin.

Requires harbor's task-declared shared volumes feature (upstream PR to
harbor-framework/harbor); until it lands, install harbor from
git+https://github.com/surelyMersad/harbor@feat/task-shared-volumes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant