Skip to content

fix(ci): store the ROCm verification logs as text instead of tarballs #2266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
inureyes merged 1 commit into from
Oct 11, 2026
Merged

fix(ci): store the ROCm verification logs as text instead of tarballs #2266

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz
View file
Open in desktop
Binary file not shown.
28 changes: 28 additions & 0 deletions ...ne-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
2026-10-09T02:44:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:44:37+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:44:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:44:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r1
2026-10-09T02:44:52+0900 sample 1: clean gpu=[]
2026-10-09T02:44:54+0900 sample 2: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:44:55+0900 sample 3: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:44:57+0900 sample 4: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:44:58+0900 sample 5: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:00+0900 sample 6: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:01+0900 sample 7: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:03+0900 sample 8: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:04+0900 sample 9: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:06+0900 sample 10: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:07+0900 sample 11: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:09+0900 sample 12: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:10+0900 sample 13: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:12+0900 sample 14: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:13+0900 sample 15: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:15+0900 sample 16: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:16+0900 sample 17: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:18+0900 sample 18: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:19+0900 sample 19: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:21+0900 sample 20: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:22+0900 sample 21: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:24+0900 sample 22: clean gpu=[1130754:mlxcel-bench-en]
2026-10-09T02:45:25+0900 monitor: 22 samples
2026-10-09T02:45:25+0900 attempt 1: CLEAN, exit 0
25 changes: 25 additions & 0 deletions ...ine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
2026-10-09T02:45:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:45:30+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:45:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:45:45+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r1
2026-10-09T02:45:45+0900 sample 1: clean gpu=[]
2026-10-09T02:45:47+0900 sample 2: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:48+0900 sample 3: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:50+0900 sample 4: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:51+0900 sample 5: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:53+0900 sample 6: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:54+0900 sample 7: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:55+0900 sample 8: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:57+0900 sample 9: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:45:58+0900 sample 10: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:00+0900 sample 11: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:01+0900 sample 12: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:03+0900 sample 13: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:04+0900 sample 14: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:06+0900 sample 15: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:07+0900 sample 16: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:09+0900 sample 17: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:10+0900 sample 18: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:12+0900 sample 19: clean gpu=[1161150:mlxcel-bench-en]
2026-10-09T02:46:13+0900 monitor: 19 samples
2026-10-09T02:46:13+0900 attempt 1: CLEAN, exit 0
26 changes: 26 additions & 0 deletions ...ne-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
2026-10-09T02:47:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:47:12+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:47:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:47:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r2
2026-10-09T02:47:27+0900 sample 1: clean gpu=[]
2026-10-09T02:47:28+0900 sample 2: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:30+0900 sample 3: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:31+0900 sample 4: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:33+0900 sample 5: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:34+0900 sample 6: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:36+0900 sample 7: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:37+0900 sample 8: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:39+0900 sample 9: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:40+0900 sample 10: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:42+0900 sample 11: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:43+0900 sample 12: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:45+0900 sample 13: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:46+0900 sample 14: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:48+0900 sample 15: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:49+0900 sample 16: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:51+0900 sample 17: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:52+0900 sample 18: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:54+0900 sample 19: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:55+0900 sample 20: clean gpu=[1217341:mlxcel-bench-en]
2026-10-09T02:47:56+0900 monitor: 20 samples
2026-10-09T02:47:56+0900 attempt 1: CLEAN, exit 0
25 changes: 25 additions & 0 deletions ...ine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
2026-10-09T02:48:01+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:48:01+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:48:16+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:48:16+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r2
2026-10-09T02:48:16+0900 sample 1: clean gpu=[]
2026-10-09T02:48:18+0900 sample 2: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:19+0900 sample 3: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:21+0900 sample 4: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:22+0900 sample 5: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:24+0900 sample 6: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:25+0900 sample 7: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:27+0900 sample 8: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:28+0900 sample 9: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:30+0900 sample 10: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:31+0900 sample 11: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:33+0900 sample 12: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:34+0900 sample 13: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:36+0900 sample 14: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:37+0900 sample 15: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:39+0900 sample 16: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:40+0900 sample 17: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:42+0900 sample 18: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:43+0900 sample 19: clean gpu=[1245919:mlxcel-bench-en]
2026-10-09T02:48:44+0900 monitor: 19 samples
2026-10-09T02:48:44+0900 attempt 1: CLEAN, exit 0
26 changes: 26 additions & 0 deletions ...ne-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
2026-10-09T02:49:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:49:43+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:49:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:49:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r3
2026-10-09T02:49:59+0900 sample 1: clean gpu=[]
2026-10-09T02:50:00+0900 sample 2: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:02+0900 sample 3: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:03+0900 sample 4: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:05+0900 sample 5: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:06+0900 sample 6: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:07+0900 sample 7: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:09+0900 sample 8: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:11+0900 sample 9: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:12+0900 sample 10: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:14+0900 sample 11: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:15+0900 sample 12: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:17+0900 sample 13: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:18+0900 sample 14: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:20+0900 sample 15: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:21+0900 sample 16: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:23+0900 sample 17: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:24+0900 sample 18: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:26+0900 sample 19: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:27+0900 sample 20: clean gpu=[1302244:mlxcel-bench-en]
2026-10-09T02:50:28+0900 monitor: 20 samples
2026-10-09T02:50:28+0900 attempt 1: CLEAN, exit 0
25 changes: 25 additions & 0 deletions ...ine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
2026-10-09T02:50:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:50:33+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:50:48+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:50:48+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r3
2026-10-09T02:50:48+0900 sample 1: clean gpu=[]
2026-10-09T02:50:50+0900 sample 2: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:51+0900 sample 3: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:53+0900 sample 4: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:54+0900 sample 5: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:56+0900 sample 6: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:57+0900 sample 7: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:50:59+0900 sample 8: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:00+0900 sample 9: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:02+0900 sample 10: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:03+0900 sample 11: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:05+0900 sample 12: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:06+0900 sample 13: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:08+0900 sample 14: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:09+0900 sample 15: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:10+0900 sample 16: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:12+0900 sample 17: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:14+0900 sample 18: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:15+0900 sample 19: clean gpu=[1330719:mlxcel-bench-en]
2026-10-09T02:51:16+0900 monitor: 19 samples
2026-10-09T02:51:16+0900 attempt 1: CLEAN, exit 0
12 changes: 12 additions & 0 deletions ...ocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
2026-10-09T02:43:42+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:43:42+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:43:56+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:43:56+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r1
2026-10-09T02:43:57+0900 sample 1: clean gpu=[]
2026-10-09T02:43:58+0900 sample 2: clean gpu=[1101389:mlxcel-bench-en]
2026-10-09T02:44:00+0900 sample 3: clean gpu=[1101389:mlxcel-bench-en]
2026-10-09T02:44:01+0900 sample 4: clean gpu=[1101389:mlxcel-bench-en]
2026-10-09T02:44:03+0900 sample 5: clean gpu=[1101389:mlxcel-bench-en]
2026-10-09T02:44:04+0900 sample 6: clean gpu=[1101389:mlxcel-bench-en]
2026-10-09T02:44:05+0900 monitor: 6 samples
2026-10-09T02:44:05+0900 attempt 1: CLEAN, exit 0
11 changes: 11 additions & 0 deletions ...rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_new.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
2026-10-09T02:44:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:44:10+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:44:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:44:25+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r1
2026-10-09T02:44:25+0900 sample 1: clean gpu=[]
2026-10-09T02:44:27+0900 sample 2: clean gpu=[1116524:mlxcel-bench-en]
2026-10-09T02:44:28+0900 sample 3: clean gpu=[1116524:mlxcel-bench-en]
2026-10-09T02:44:30+0900 sample 4: clean gpu=[1116524:mlxcel-bench-en]
2026-10-09T02:44:31+0900 sample 5: clean gpu=[1116524:mlxcel-bench-en]
2026-10-09T02:44:32+0900 monitor: 5 samples
2026-10-09T02:44:32+0900 attempt 1: CLEAN, exit 0
11 changes: 11 additions & 0 deletions ...ocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
2026-10-09T02:46:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:46:18+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:46:32+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:46:32+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r2
2026-10-09T02:46:33+0900 sample 1: clean gpu=[]
2026-10-09T02:46:34+0900 sample 2: clean gpu=[1188757:mlxcel-bench-en]
2026-10-09T02:46:36+0900 sample 3: clean gpu=[1188757:mlxcel-bench-en]
2026-10-09T02:46:37+0900 sample 4: clean gpu=[1188757:mlxcel-bench-en]
2026-10-09T02:46:39+0900 sample 5: clean gpu=[1188757:mlxcel-bench-en]
2026-10-09T02:46:40+0900 monitor: 5 samples
2026-10-09T02:46:40+0900 attempt 1: CLEAN, exit 0
11 changes: 11 additions & 0 deletions ...rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_new.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
2026-10-09T02:46:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:46:45+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:46:59+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:46:59+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r2
2026-10-09T02:47:00+0900 sample 1: clean gpu=[]
2026-10-09T02:47:01+0900 sample 2: clean gpu=[1203050:mlxcel-bench-en]
2026-10-09T02:47:03+0900 sample 3: clean gpu=[1203050:mlxcel-bench-en]
2026-10-09T02:47:04+0900 sample 4: clean gpu=[1203050:mlxcel-bench-en]
2026-10-09T02:47:06+0900 sample 5: clean gpu=[1203050:mlxcel-bench-en]
2026-10-09T02:47:07+0900 monitor: 5 samples
2026-10-09T02:47:07+0900 attempt 1: CLEAN, exit 0
11 changes: 11 additions & 0 deletions ...ocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_base.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
2026-10-09T02:48:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock
2026-10-09T02:48:49+0900 waiting for 10s of idle GPU and no compiler
2026-10-09T02:49:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample
2026-10-09T02:49:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r3
2026-10-09T02:49:04+0900 sample 1: clean gpu=[]
2026-10-09T02:49:06+0900 sample 2: clean gpu=[1273600:mlxcel-bench-en]
2026-10-09T02:49:07+0900 sample 3: clean gpu=[1273600:mlxcel-bench-en]
2026-10-09T02:49:09+0900 sample 4: clean gpu=[1273600:mlxcel-bench-en]
2026-10-09T02:49:10+0900 sample 5: clean gpu=[1273600:mlxcel-bench-en]
2026-10-09T02:49:11+0900 monitor: 5 samples
2026-10-09T02:49:11+0900 attempt 1: CLEAN, exit 0
Loading
Loading