Repository navigation
feat: jh job result list and jh job result download - #74
Conversation
Adds `jh job result` for working with the files a job uploads via `RESULTS_FILE`, e.g. to pull a previous batch run's output into a new job.
- `jh job result list <job-name>` lists result files with size and upload time; `--all` shows every job file (input, source, project, result) with its type, `--json` prints `{"job", "status", "files"}`.
- `jh job result download <job-name> [file-name]` downloads one or all result files; `-o` sets the destination file or directory, `--force` overwrites, `--type` selects another job file type.
Job details come from `GET /api/rest/jobs/{jobname}` (the endpoint JuliaHub.jl's `JuliaHub.job` uses). Files are fetched from `GET /jobs/{jobname}/files/{filename}?filetype=...`, which 302-redirects to a presigned object-store URL; the client drops `Authorization` on every redirect since Go only does so cross-domain. Downloads stream to a temp file, are checked against the recorded `sha2_256` hash, and are then renamed into place. JuliaHub records that hash as hex (verified against a live instance), not base64 as JuliaHub.jl's `FileHash` assumes, so hex is tried first.
Agent-Harness: Claude Code 2.1.301
Agent-Model: claude-opus-5-5
Agent-Session: 1ce5aeb7-9a97-4eb5-aa37-46b1100cfa21
Agent-Review: reviewed by @lamdor
pankgeorg
left a comment
There was a problem hiding this comment.
Approving: the security-sensitive parts hold up, and the inline comments are edge cases and suggestions, not blockers.
Checked and fine
- Redirects (against the go1.24.4
net/http/client.gosource): on every hop,Client.dore-copies the initial request's headers into a fresh map, then callsCheckRedirectlast. SoDel("Authorization")keeps the token on hop 0 only, which goes tohttps://<server>. That covers chains like API → API → store and API → store → API, and an https→http downgrade. The download request carries noCookie,Proxy-AuthorizationorX-Hasura-Role, and the hook lives on a per-command client, so other commands aren't affected. The test is meaningful: Go compares hostnames without the port, so127.0.0.1:A → 127.0.0.1:Bcounts as same-domain, and a plain client does forward the token there. I checked that with a throwaway probe. - Path safety:
..,., empty names, absolute paths,\andC:volumes all reduce to a safe base name. See the inline comment for Windows device names. - Atomic download: the temp file is removed on every error path, and nothing is left at the destination after a mismatch.
os.Renamereplaces existing files on Windows too, so--forceworks there. go build,go vetand the targeted tests pass locally.
Tests that would pin the edge cases: -o as a function of file count, pending (size: null) files in download-all, duplicate destinations, and a multi-hop redirect chain.
Nits not anchored inline
main.go: thejobCmdLongtext still says "Job functionality is currently in development"; onlylist/startare placeholders now.jobs.go:152:nameWidthcounts bytes whilepad/truncatecount runes, so non-ASCII names widen the column. Cosmetic.jobs.go:341:MkdirAllruns before verification, so-o newdir/plus a checksum mismatch leaves an emptynewdir/. Ctrl-C leaves a hidden.name.*.tmpfile. Both are acceptable.formatTokenDatenow formats job timestamps too; a neutral name would read better.
🤖 Posted for @pankgeorg by an AI agent — harness: Claude Code 2.1.301 · model: claude-opus-5-5
Session: local Claude Code session 9b44d1cd-8141-494c-9383-8742eb5d0444
- Download-all always treats `-o` as a directory, so the on-disk shape no longer depends on how many result files a job has; a named download keeps the file-or-directory rule. - Download-all skips files whose upload is still in progress (with a stderr notice); naming a pending file is an error. - `--type` is validated against input/source/project/result, and every destination is resolved before anything is downloaded, so two files flattening to the same name fail up front instead of failing partway or overwriting each other. - The download client bounds only the wait for response headers (60s) instead of the whole request, so large downloads over slow links are not aborted. - Without `--force`, the finished download is hard-linked into place, so a file created at the destination during the transfer is not replaced (falling back to re-check + rename where hard links are unsupported). - Downloads are checked against the recorded size; an empty recorded hash is treated as missing, and unverified downloads say "(not verified)". - Destination names must pass `filepath.IsLocal`, which also rejects Windows device names. - `Referer` is dropped on redirects along with `Authorization`. - `--force` loses its `-f` shorthand, which means `--file` elsewhere in the CLI. - Nits: the `jh job` help no longer calls all job functionality in development, the name column width counts runes, and `formatTokenDate` is renamed `formatTimestamp`. Agent-Harness: Claude Code 2.1.301 Agent-Model: claude-opus-5-5 Agent-Session: 29b26359-89b6-4a2c-bc20-668092480740 Agent-Review: reviewed by @lamdor
|
Thanks for the thorough review. b7a8f44 addresses every inline comment, plus these nits:
I left the 🤖 Posted by an AI agent · harness: Claude Code 2.1.301 · model: claude-opus-5-5 |
|
Is this compatible with the current juliahub? If so, let's backport to |
|
Works with JuliaHub 26.4.2. I checked every server API this PR calls against the
On the client side, |
Adds
jh job resultto list and download a job's result files.What
New
jh job resultsubcommands for the files a job uploads viaRESULTS_FILE, e.g. to pull a previous batch run's output into a new job without writing Julia against JuliaHub.jl.jh job result list <job-name>lists result files with size and upload time.--allshows every job file (input, source, project, result) with its type;--jsonprints{"job", "status", "files"}.jh job result download <job-name> [file-name]downloads one file, or every result file when no name is given.-ois the destination file or directory,--forceoverwrites existing files,--typeselects another job file type.How
GET /api/rest/jobs/{jobname}(Hasura REST,X-Hasura-Role: jhuser), the endpoint JuliaHub.jl'sJuliaHub.jobuses. An emptydetailslist means the job doesn't exist or isn't visible.GET /jobs/{jobname}/files/{filename}?filetype=..., which 302-redirects to a short-lived presigned object-store URL. The client dropsAuthorizationon every redirect. Go only does that cross-domain, and the object store rejects a second auth mechanism (and shouldn't see the token).sha2_256hash, then get renamed into place. A failed or corrupted download leaves nothing behind.FileHashbase64-decodes it, which yields garbage, though nothing there compares it. Verified against nightly: the downloaded bytes' SHA-256 equals the hex value. Hex is tried first, with base64 as a fallback.The existing
jh job list/jh job startplaceholders are unchanged.Notes
--all,--json, single and all-file download,-ofile and directory, the no-overwrite and--forcepaths,--type source, a missing file name, and an unknown job. Also ran the new e2e test.jhinstalled in the image and a~/.juliahublogin.jhdoesn't yet read the job's injected credentials, so that's a possible follow-up.🤖 Posted by an AI agent · harness: Claude Code 2.1.301 · model: claude-opus-5-5
Review: reviewed by @lamdor
Session: 1ce5aeb7-9a97-4eb5-aa37-46b1100cfa21