diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz new file mode 100644 index 000000000..6f1c976b3 Binary files /dev/null and b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz differ diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_base.log deleted file mode 100644 index e1f62dcda..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_base.log +++ /dev/null @@ -1,28 +0,0 @@ -2026-10-09T02:44:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:44:37+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:44:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:44:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r1 -2026-10-09T02:44:52+0900 sample 1: clean gpu=[] -2026-10-09T02:44:54+0900 sample 2: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:44:55+0900 sample 3: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:44:57+0900 sample 4: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:44:58+0900 sample 5: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:00+0900 sample 6: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:01+0900 sample 7: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:03+0900 sample 8: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:04+0900 sample 9: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:06+0900 sample 10: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:07+0900 sample 11: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:09+0900 sample 12: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:10+0900 sample 13: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:12+0900 sample 14: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:13+0900 sample 15: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:15+0900 sample 16: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:16+0900 sample 17: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:18+0900 sample 18: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:19+0900 sample 19: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:21+0900 sample 20: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:22+0900 sample 21: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:24+0900 sample 22: clean gpu=[1130754:mlxcel-bench-en] -2026-10-09T02:45:25+0900 monitor: 22 samples -2026-10-09T02:45:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log deleted file mode 100644 index 9924859b7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log +++ /dev/null @@ -1,25 +0,0 @@ -2026-10-09T02:45:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:45:30+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:45:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:45:45+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r1 -2026-10-09T02:45:45+0900 sample 1: clean gpu=[] -2026-10-09T02:45:47+0900 sample 2: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:48+0900 sample 3: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:50+0900 sample 4: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:51+0900 sample 5: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:53+0900 sample 6: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:54+0900 sample 7: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:55+0900 sample 8: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:57+0900 sample 9: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:45:58+0900 sample 10: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:00+0900 sample 11: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:01+0900 sample 12: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:03+0900 sample 13: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:04+0900 sample 14: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:06+0900 sample 15: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:07+0900 sample 16: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:09+0900 sample 17: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:10+0900 sample 18: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:12+0900 sample 19: clean gpu=[1161150:mlxcel-bench-en] -2026-10-09T02:46:13+0900 monitor: 19 samples -2026-10-09T02:46:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_base.log deleted file mode 100644 index 19bd89c8d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_base.log +++ /dev/null @@ -1,26 +0,0 @@ -2026-10-09T02:47:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:47:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:47:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:47:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r2 -2026-10-09T02:47:27+0900 sample 1: clean gpu=[] -2026-10-09T02:47:28+0900 sample 2: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:30+0900 sample 3: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:31+0900 sample 4: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:33+0900 sample 5: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:34+0900 sample 6: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:36+0900 sample 7: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:37+0900 sample 8: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:39+0900 sample 9: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:40+0900 sample 10: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:42+0900 sample 11: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:43+0900 sample 12: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:45+0900 sample 13: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:46+0900 sample 14: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:48+0900 sample 15: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:49+0900 sample 16: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:51+0900 sample 17: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:52+0900 sample 18: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:54+0900 sample 19: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:55+0900 sample 20: clean gpu=[1217341:mlxcel-bench-en] -2026-10-09T02:47:56+0900 monitor: 20 samples -2026-10-09T02:47:56+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log deleted file mode 100644 index 8f463584e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log +++ /dev/null @@ -1,25 +0,0 @@ -2026-10-09T02:48:01+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:48:01+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:48:16+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:48:16+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r2 -2026-10-09T02:48:16+0900 sample 1: clean gpu=[] -2026-10-09T02:48:18+0900 sample 2: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:19+0900 sample 3: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:21+0900 sample 4: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:22+0900 sample 5: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:24+0900 sample 6: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:25+0900 sample 7: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:27+0900 sample 8: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:28+0900 sample 9: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:30+0900 sample 10: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:31+0900 sample 11: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:33+0900 sample 12: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:34+0900 sample 13: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:36+0900 sample 14: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:37+0900 sample 15: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:39+0900 sample 16: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:40+0900 sample 17: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:42+0900 sample 18: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:43+0900 sample 19: clean gpu=[1245919:mlxcel-bench-en] -2026-10-09T02:48:44+0900 monitor: 19 samples -2026-10-09T02:48:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_base.log deleted file mode 100644 index cd339c9d7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_base.log +++ /dev/null @@ -1,26 +0,0 @@ -2026-10-09T02:49:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:49:43+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:49:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:49:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r3 -2026-10-09T02:49:59+0900 sample 1: clean gpu=[] -2026-10-09T02:50:00+0900 sample 2: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:02+0900 sample 3: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:03+0900 sample 4: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:05+0900 sample 5: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:06+0900 sample 6: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:07+0900 sample 7: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:09+0900 sample 8: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:11+0900 sample 9: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:12+0900 sample 10: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:14+0900 sample 11: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:15+0900 sample 12: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:17+0900 sample 13: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:18+0900 sample 14: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:20+0900 sample 15: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:21+0900 sample 16: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:23+0900 sample 17: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:24+0900 sample 18: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:26+0900 sample 19: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:27+0900 sample 20: clean gpu=[1302244:mlxcel-bench-en] -2026-10-09T02:50:28+0900 monitor: 20 samples -2026-10-09T02:50:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log deleted file mode 100644 index 6bc98da11..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log +++ /dev/null @@ -1,25 +0,0 @@ -2026-10-09T02:50:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:50:33+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:50:48+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:50:48+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r3 -2026-10-09T02:50:48+0900 sample 1: clean gpu=[] -2026-10-09T02:50:50+0900 sample 2: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:51+0900 sample 3: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:53+0900 sample 4: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:54+0900 sample 5: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:56+0900 sample 6: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:57+0900 sample 7: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:50:59+0900 sample 8: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:00+0900 sample 9: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:02+0900 sample 10: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:03+0900 sample 11: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:05+0900 sample 12: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:06+0900 sample 13: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:08+0900 sample 14: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:09+0900 sample 15: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:10+0900 sample 16: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:12+0900 sample 17: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:14+0900 sample 18: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:15+0900 sample 19: clean gpu=[1330719:mlxcel-bench-en] -2026-10-09T02:51:16+0900 monitor: 19 samples -2026-10-09T02:51:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_base.log deleted file mode 100644 index 9a74ed523..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_base.log +++ /dev/null @@ -1,12 +0,0 @@ -2026-10-09T02:43:42+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:43:42+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:43:56+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:43:56+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r1 -2026-10-09T02:43:57+0900 sample 1: clean gpu=[] -2026-10-09T02:43:58+0900 sample 2: clean gpu=[1101389:mlxcel-bench-en] -2026-10-09T02:44:00+0900 sample 3: clean gpu=[1101389:mlxcel-bench-en] -2026-10-09T02:44:01+0900 sample 4: clean gpu=[1101389:mlxcel-bench-en] -2026-10-09T02:44:03+0900 sample 5: clean gpu=[1101389:mlxcel-bench-en] -2026-10-09T02:44:04+0900 sample 6: clean gpu=[1101389:mlxcel-bench-en] -2026-10-09T02:44:05+0900 monitor: 6 samples -2026-10-09T02:44:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_new.log deleted file mode 100644 index 62d8bfdae..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:44:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:44:10+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:44:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:44:25+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r1 -2026-10-09T02:44:25+0900 sample 1: clean gpu=[] -2026-10-09T02:44:27+0900 sample 2: clean gpu=[1116524:mlxcel-bench-en] -2026-10-09T02:44:28+0900 sample 3: clean gpu=[1116524:mlxcel-bench-en] -2026-10-09T02:44:30+0900 sample 4: clean gpu=[1116524:mlxcel-bench-en] -2026-10-09T02:44:31+0900 sample 5: clean gpu=[1116524:mlxcel-bench-en] -2026-10-09T02:44:32+0900 monitor: 5 samples -2026-10-09T02:44:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_base.log deleted file mode 100644 index 480427df9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:46:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:46:18+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:46:32+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:46:32+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r2 -2026-10-09T02:46:33+0900 sample 1: clean gpu=[] -2026-10-09T02:46:34+0900 sample 2: clean gpu=[1188757:mlxcel-bench-en] -2026-10-09T02:46:36+0900 sample 3: clean gpu=[1188757:mlxcel-bench-en] -2026-10-09T02:46:37+0900 sample 4: clean gpu=[1188757:mlxcel-bench-en] -2026-10-09T02:46:39+0900 sample 5: clean gpu=[1188757:mlxcel-bench-en] -2026-10-09T02:46:40+0900 monitor: 5 samples -2026-10-09T02:46:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_new.log deleted file mode 100644 index 15e3b009c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:46:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:46:45+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:46:59+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:46:59+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r2 -2026-10-09T02:47:00+0900 sample 1: clean gpu=[] -2026-10-09T02:47:01+0900 sample 2: clean gpu=[1203050:mlxcel-bench-en] -2026-10-09T02:47:03+0900 sample 3: clean gpu=[1203050:mlxcel-bench-en] -2026-10-09T02:47:04+0900 sample 4: clean gpu=[1203050:mlxcel-bench-en] -2026-10-09T02:47:06+0900 sample 5: clean gpu=[1203050:mlxcel-bench-en] -2026-10-09T02:47:07+0900 monitor: 5 samples -2026-10-09T02:47:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_base.log deleted file mode 100644 index 76e93a976..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:48:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:48:49+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:49:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:49:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label base-r3 -2026-10-09T02:49:04+0900 sample 1: clean gpu=[] -2026-10-09T02:49:06+0900 sample 2: clean gpu=[1273600:mlxcel-bench-en] -2026-10-09T02:49:07+0900 sample 3: clean gpu=[1273600:mlxcel-bench-en] -2026-10-09T02:49:09+0900 sample 4: clean gpu=[1273600:mlxcel-bench-en] -2026-10-09T02:49:10+0900 sample 5: clean gpu=[1273600:mlxcel-bench-en] -2026-10-09T02:49:11+0900 monitor: 5 samples -2026-10-09T02:49:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_new.log deleted file mode 100644 index c78b3a5c8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:49:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:49:16+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:49:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:49:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-engine -m models/mlx/Qwen3-0.6B-4bit --path both --decode-storage dense --prompt-tokens 256,2048 -n 128 --warmup-tokens 20 --csv /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/engine/rounds.csv --label new-r3 -2026-10-09T02:49:31+0900 sample 1: clean gpu=[] -2026-10-09T02:49:33+0900 sample 2: clean gpu=[1287900:mlxcel-bench-en] -2026-10-09T02:49:34+0900 sample 3: clean gpu=[1287900:mlxcel-bench-en] -2026-10-09T02:49:36+0900 sample 4: clean gpu=[1287900:mlxcel-bench-en] -2026-10-09T02:49:37+0900 sample 5: clean gpu=[1287900:mlxcel-bench-en] -2026-10-09T02:49:38+0900 monitor: 5 samples -2026-10-09T02:49:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/gemma3_probe.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/gemma3_probe.log deleted file mode 100644 index 3c3c94426..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/gemma3_probe.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-09T02:51:39+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:51:39+0900 waiting for 20s of idle GPU and no compiler -2026-10-09T02:52:08+0900 idle for 20s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:52:08+0900 attempt 1/3: start: bash /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/measure/gemma3_probe.sh -2026-10-09T02:52:08+0900 sample 1: clean gpu=[] -2026-10-09T02:52:10+0900 sample 2: clean gpu=[1378646:mlxcel] -2026-10-09T02:52:11+0900 sample 3: clean gpu=[1378646:mlxcel] -2026-10-09T02:52:12+0900 monitor: 3 samples -2026-10-09T02:52:12+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/paged_pool_past_u32.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/paged_pool_past_u32.log deleted file mode 100644 index 37baaff75..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/paged_pool_past_u32.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:51:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:51:21+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:51:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:51:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/test-fast/deps/mlxcel_core-480914a0f733fc77 paged_pool_past_u32_elements_matches_gather --ignored --test-threads=1 --nocapture -2026-10-09T02:51:36+0900 sample 1: clean gpu=[] -2026-10-09T02:51:38+0900 sample 2: clean gpu=[1358129:mlxcel_core-480] -2026-10-09T02:51:39+0900 monitor: 2 samples -2026-10-09T02:51:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log deleted file mode 100644 index 5f2d4aac8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:47:04+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:47:04+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:47:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:47:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:47:19+0900 sample 1: clean gpu=[] -2026-10-09T01:47:20+0900 sample 2: clean gpu=[3394666:mlxcel-bench-de] -2026-10-09T01:47:22+0900 sample 3: clean gpu=[3394666:mlxcel-bench-de] -2026-10-09T01:47:24+0900 sample 4: clean gpu=[3394666:mlxcel-bench-de] -2026-10-09T01:47:25+0900 sample 5: clean gpu=[3394666:mlxcel-bench-de] -2026-10-09T01:47:26+0900 monitor: 5 samples -2026-10-09T01:47:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log deleted file mode 100644 index 97ee9e57d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log +++ /dev/null @@ -1,13 +0,0 @@ -2026-10-09T01:46:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:46:06+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:46:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:46:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:46:21+0900 sample 1: clean gpu=[] -2026-10-09T01:46:23+0900 sample 2: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:24+0900 sample 3: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:26+0900 sample 4: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:27+0900 sample 5: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:29+0900 sample 6: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:30+0900 sample 7: clean gpu=[3363796:mlxcel-bench-de] -2026-10-09T01:46:31+0900 monitor: 7 samples -2026-10-09T01:46:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log deleted file mode 100644 index 539d1606b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:46:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:46:36+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:46:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:46:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:46:52+0900 sample 1: clean gpu=[] -2026-10-09T01:46:53+0900 sample 2: clean gpu=[3380296:mlxcel-bench-de] -2026-10-09T01:46:55+0900 sample 3: clean gpu=[3380296:mlxcel-bench-de] -2026-10-09T01:46:56+0900 sample 4: clean gpu=[3380296:mlxcel-bench-de] -2026-10-09T01:46:58+0900 sample 5: clean gpu=[3380296:mlxcel-bench-de] -2026-10-09T01:46:59+0900 monitor: 5 samples -2026-10-09T01:46:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log deleted file mode 100644 index f82c9e230..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:06:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:06:37+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:06:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:06:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:06:53+0900 sample 1: clean gpu=[] -2026-10-09T02:06:54+0900 sample 2: clean gpu=[4075655:mlxcel-bench-de] -2026-10-09T02:06:56+0900 sample 3: clean gpu=[4075655:mlxcel-bench-de] -2026-10-09T02:06:57+0900 sample 4: clean gpu=[4075655:mlxcel-bench-de] -2026-10-09T02:06:59+0900 sample 5: clean gpu=[4075655:mlxcel-bench-de] -2026-10-09T02:07:00+0900 monitor: 5 samples -2026-10-09T02:07:00+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log deleted file mode 100644 index 67cc4e352..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:07:05+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:07:05+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:07:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:07:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:07:20+0900 sample 1: clean gpu=[] -2026-10-09T02:07:21+0900 sample 2: clean gpu=[4089997:mlxcel-bench-de] -2026-10-09T02:07:23+0900 sample 3: clean gpu=[4089997:mlxcel-bench-de] -2026-10-09T02:07:25+0900 sample 4: clean gpu=[4089997:mlxcel-bench-de] -2026-10-09T02:07:26+0900 sample 5: clean gpu=[4089997:mlxcel-bench-de] -2026-10-09T02:07:27+0900 monitor: 5 samples -2026-10-09T02:07:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log deleted file mode 100644 index 867349bea..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log +++ /dev/null @@ -1,13 +0,0 @@ -2026-10-09T02:06:07+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:06:07+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:06:22+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:06:22+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:06:22+0900 sample 1: clean gpu=[] -2026-10-09T02:06:24+0900 sample 2: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:25+0900 sample 3: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:27+0900 sample 4: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:28+0900 sample 5: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:30+0900 sample 6: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:31+0900 sample 7: clean gpu=[4059413:mlxcel-bench-de] -2026-10-09T02:06:32+0900 monitor: 7 samples -2026-10-09T02:06:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log deleted file mode 100644 index e7d25323d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,13 +0,0 @@ -2026-10-09T02:26:00+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:26:00+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:26:15+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:26:15+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:26:15+0900 sample 1: clean gpu=[] -2026-10-09T02:26:17+0900 sample 2: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:18+0900 sample 3: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:20+0900 sample 4: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:21+0900 sample 5: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:23+0900 sample 6: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:24+0900 sample 7: clean gpu=[515159:mlxcel-bench-de] -2026-10-09T02:26:25+0900 monitor: 7 samples -2026-10-09T02:26:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log deleted file mode 100644 index f4da0a005..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:26:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:26:30+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:26:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:26:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:26:46+0900 sample 1: clean gpu=[] -2026-10-09T02:26:47+0900 sample 2: clean gpu=[531413:mlxcel-bench-de] -2026-10-09T02:26:49+0900 sample 3: clean gpu=[531413:mlxcel-bench-de] -2026-10-09T02:26:50+0900 sample 4: clean gpu=[531413:mlxcel-bench-de] -2026-10-09T02:26:52+0900 sample 5: clean gpu=[531413:mlxcel-bench-de] -2026-10-09T02:26:53+0900 monitor: 5 samples -2026-10-09T02:26:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log deleted file mode 100644 index 33907e75e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:26:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:26:58+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:27:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:27:12+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:27:13+0900 sample 1: clean gpu=[] -2026-10-09T02:27:14+0900 sample 2: clean gpu=[545723:mlxcel-bench-de] -2026-10-09T02:27:16+0900 sample 3: clean gpu=[545723:mlxcel-bench-de] -2026-10-09T02:27:17+0900 sample 4: clean gpu=[545723:mlxcel-bench-de] -2026-10-09T02:27:19+0900 sample 5: clean gpu=[545723:mlxcel-bench-de] -2026-10-09T02:27:20+0900 monitor: 5 samples -2026-10-09T02:27:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log deleted file mode 100644 index c505ef67b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,44 +0,0 @@ -2026-10-09T02:02:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:02:31+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:02:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:02:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:02:46+0900 sample 1: clean gpu=[] -2026-10-09T02:02:48+0900 sample 2: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:49+0900 sample 3: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:51+0900 sample 4: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:52+0900 sample 5: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:54+0900 sample 6: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:55+0900 sample 7: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:57+0900 sample 8: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:58+0900 sample 9: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:02:59+0900 sample 10: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:01+0900 sample 11: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:02+0900 sample 12: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:04+0900 sample 13: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:05+0900 sample 14: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:07+0900 sample 15: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:08+0900 sample 16: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:10+0900 sample 17: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:11+0900 sample 18: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:13+0900 sample 19: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:14+0900 sample 20: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:16+0900 sample 21: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:17+0900 sample 22: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:19+0900 sample 23: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:21+0900 sample 24: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:22+0900 sample 25: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:24+0900 sample 26: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:26+0900 sample 27: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:28+0900 sample 28: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:29+0900 sample 29: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:31+0900 sample 30: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:32+0900 sample 31: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:34+0900 sample 32: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:35+0900 sample 33: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:37+0900 sample 34: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:38+0900 sample 35: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:40+0900 sample 36: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:41+0900 sample 37: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:43+0900 sample 38: clean gpu=[3944782:mlxcel-bench-de] -2026-10-09T02:03:44+0900 monitor: 38 samples -2026-10-09T02:03:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log deleted file mode 100644 index 23e9423d9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log +++ /dev/null @@ -1,45 +0,0 @@ -2026-10-09T01:59:53+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:59:53+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:00:07+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:00:07+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:00:08+0900 sample 1: clean gpu=[] -2026-10-09T02:00:09+0900 sample 2: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:11+0900 sample 3: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:12+0900 sample 4: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:14+0900 sample 5: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:15+0900 sample 6: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:17+0900 sample 7: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:18+0900 sample 8: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:20+0900 sample 9: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:21+0900 sample 10: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:23+0900 sample 11: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:24+0900 sample 12: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:26+0900 sample 13: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:27+0900 sample 14: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:29+0900 sample 15: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:31+0900 sample 16: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:32+0900 sample 17: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:33+0900 sample 18: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:35+0900 sample 19: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:36+0900 sample 20: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:38+0900 sample 21: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:39+0900 sample 22: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:41+0900 sample 23: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:42+0900 sample 24: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:44+0900 sample 25: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:45+0900 sample 26: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:47+0900 sample 27: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:49+0900 sample 28: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:51+0900 sample 29: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:52+0900 sample 30: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:54+0900 sample 31: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:55+0900 sample 32: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:57+0900 sample 33: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:00:58+0900 sample 34: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:00+0900 sample 35: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:01+0900 sample 36: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:03+0900 sample 37: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:04+0900 sample 38: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:06+0900 sample 39: clean gpu=[3852117:mlxcel-bench-de] -2026-10-09T02:01:07+0900 monitor: 39 samples -2026-10-09T02:01:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log deleted file mode 100644 index 02f99aa8b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log +++ /dev/null @@ -1,45 +0,0 @@ -2026-10-09T02:01:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:01:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:01:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:01:27+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:01:27+0900 sample 1: clean gpu=[] -2026-10-09T02:01:29+0900 sample 2: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:30+0900 sample 3: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:32+0900 sample 4: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:33+0900 sample 5: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:35+0900 sample 6: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:36+0900 sample 7: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:38+0900 sample 8: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:39+0900 sample 9: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:41+0900 sample 10: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:42+0900 sample 11: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:44+0900 sample 12: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:45+0900 sample 13: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:47+0900 sample 14: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:48+0900 sample 15: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:50+0900 sample 16: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:51+0900 sample 17: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:53+0900 sample 18: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:54+0900 sample 19: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:56+0900 sample 20: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:57+0900 sample 21: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:01:58+0900 sample 22: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:00+0900 sample 23: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:01+0900 sample 24: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:03+0900 sample 25: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:05+0900 sample 26: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:07+0900 sample 27: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:08+0900 sample 28: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:10+0900 sample 29: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:12+0900 sample 30: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:13+0900 sample 31: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:15+0900 sample 32: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:16+0900 sample 33: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:18+0900 sample 34: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:19+0900 sample 35: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:21+0900 sample 36: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:22+0900 sample 37: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:24+0900 sample 38: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:25+0900 sample 39: clean gpu=[3899155:mlxcel-bench-de] -2026-10-09T02:02:26+0900 monitor: 39 samples -2026-10-09T02:02:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log deleted file mode 100644 index 474ba7a9f..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,46 +0,0 @@ -2026-10-09T02:21:04+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:21:04+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:21:19+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:21:19+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:21:19+0900 sample 1: clean gpu=[] -2026-10-09T02:21:21+0900 sample 2: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:22+0900 sample 3: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:24+0900 sample 4: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:25+0900 sample 5: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:27+0900 sample 6: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:28+0900 sample 7: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:30+0900 sample 8: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:31+0900 sample 9: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:33+0900 sample 10: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:34+0900 sample 11: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:36+0900 sample 12: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:37+0900 sample 13: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:39+0900 sample 14: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:40+0900 sample 15: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:42+0900 sample 16: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:43+0900 sample 17: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:45+0900 sample 18: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:46+0900 sample 19: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:48+0900 sample 20: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:49+0900 sample 21: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:50+0900 sample 22: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:52+0900 sample 23: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:53+0900 sample 24: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:55+0900 sample 25: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:57+0900 sample 26: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:21:58+0900 sample 27: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:00+0900 sample 28: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:02+0900 sample 29: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:04+0900 sample 30: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:05+0900 sample 31: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:07+0900 sample 32: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:08+0900 sample 33: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:10+0900 sample 34: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:11+0900 sample 35: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:13+0900 sample 36: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:14+0900 sample 37: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:16+0900 sample 38: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:17+0900 sample 39: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:19+0900 sample 40: clean gpu=[355365:mlxcel-bench-de] -2026-10-09T02:22:20+0900 monitor: 40 samples -2026-10-09T02:22:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log deleted file mode 100644 index f0096413b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log +++ /dev/null @@ -1,44 +0,0 @@ -2026-10-09T02:22:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:22:25+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:22:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:22:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:22:40+0900 sample 1: clean gpu=[] -2026-10-09T02:22:41+0900 sample 2: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:43+0900 sample 3: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:44+0900 sample 4: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:46+0900 sample 5: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:47+0900 sample 6: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:49+0900 sample 7: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:50+0900 sample 8: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:52+0900 sample 9: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:53+0900 sample 10: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:55+0900 sample 11: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:56+0900 sample 12: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:22:58+0900 sample 13: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:00+0900 sample 14: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:01+0900 sample 15: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:02+0900 sample 16: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:04+0900 sample 17: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:05+0900 sample 18: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:07+0900 sample 19: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:08+0900 sample 20: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:10+0900 sample 21: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:11+0900 sample 22: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:13+0900 sample 23: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:14+0900 sample 24: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:16+0900 sample 25: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:18+0900 sample 26: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:20+0900 sample 27: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:22+0900 sample 28: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:23+0900 sample 29: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:25+0900 sample 30: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:26+0900 sample 31: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:28+0900 sample 32: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:29+0900 sample 33: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:31+0900 sample 34: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:32+0900 sample 35: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:34+0900 sample 36: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:36+0900 sample 37: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:37+0900 sample 38: clean gpu=[401749:mlxcel-bench-de] -2026-10-09T02:23:38+0900 monitor: 38 samples -2026-10-09T02:23:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log deleted file mode 100644 index b696cc2bb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log +++ /dev/null @@ -1,45 +0,0 @@ -2026-10-09T02:19:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:19:45+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:20:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:20:00+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:20:00+0900 sample 1: clean gpu=[] -2026-10-09T02:20:02+0900 sample 2: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:03+0900 sample 3: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:05+0900 sample 4: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:06+0900 sample 5: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:08+0900 sample 6: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:09+0900 sample 7: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:11+0900 sample 8: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:12+0900 sample 9: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:14+0900 sample 10: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:15+0900 sample 11: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:17+0900 sample 12: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:18+0900 sample 13: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:20+0900 sample 14: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:22+0900 sample 15: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:23+0900 sample 16: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:25+0900 sample 17: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:26+0900 sample 18: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:27+0900 sample 19: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:29+0900 sample 20: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:30+0900 sample 21: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:32+0900 sample 22: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:33+0900 sample 23: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:35+0900 sample 24: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:36+0900 sample 25: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:38+0900 sample 26: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:40+0900 sample 27: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:41+0900 sample 28: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:43+0900 sample 29: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:45+0900 sample 30: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:46+0900 sample 31: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:48+0900 sample 32: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:49+0900 sample 33: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:51+0900 sample 34: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:52+0900 sample 35: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:54+0900 sample 36: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:55+0900 sample 37: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:57+0900 sample 38: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:58+0900 sample 39: clean gpu=[309913:mlxcel-bench-de] -2026-10-09T02:20:59+0900 monitor: 39 samples -2026-10-09T02:20:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log deleted file mode 100644 index 602d4aab4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,45 +0,0 @@ -2026-10-09T02:39:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:39:31+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:39:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:39:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:39:46+0900 sample 1: clean gpu=[] -2026-10-09T02:39:47+0900 sample 2: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:49+0900 sample 3: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:50+0900 sample 4: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:52+0900 sample 5: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:53+0900 sample 6: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:55+0900 sample 7: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:56+0900 sample 8: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:58+0900 sample 9: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:39:59+0900 sample 10: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:01+0900 sample 11: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:02+0900 sample 12: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:04+0900 sample 13: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:05+0900 sample 14: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:07+0900 sample 15: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:08+0900 sample 16: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:10+0900 sample 17: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:11+0900 sample 18: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:12+0900 sample 19: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:14+0900 sample 20: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:15+0900 sample 21: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:17+0900 sample 22: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:18+0900 sample 23: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:20+0900 sample 24: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:21+0900 sample 25: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:23+0900 sample 26: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:25+0900 sample 27: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:26+0900 sample 28: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:28+0900 sample 29: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:30+0900 sample 30: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:31+0900 sample 31: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:33+0900 sample 32: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:34+0900 sample 33: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:36+0900 sample 34: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:37+0900 sample 35: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:39+0900 sample 36: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:41+0900 sample 37: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:42+0900 sample 38: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:44+0900 sample 39: clean gpu=[953720:mlxcel-bench-de] -2026-10-09T02:40:45+0900 monitor: 39 samples -2026-10-09T02:40:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log deleted file mode 100644 index 5089cd132..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log +++ /dev/null @@ -1,54 +0,0 @@ -2026-10-09T02:40:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:40:50+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:41:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:41:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:41:05+0900 sample 1: clean gpu=[] -2026-10-09T02:41:06+0900 sample 2: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:08+0900 sample 3: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:09+0900 sample 4: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:11+0900 sample 5: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:13+0900 sample 6: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:14+0900 sample 7: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:16+0900 sample 8: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:17+0900 sample 9: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:19+0900 sample 10: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:20+0900 sample 11: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:22+0900 sample 12: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:23+0900 sample 13: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:25+0900 sample 14: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:26+0900 sample 15: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:28+0900 sample 16: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:29+0900 sample 17: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:31+0900 sample 18: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:32+0900 sample 19: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:34+0900 sample 20: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:35+0900 sample 21: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:37+0900 sample 22: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:38+0900 sample 23: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:40+0900 sample 24: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:41+0900 sample 25: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:43+0900 sample 26: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:45+0900 sample 27: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:46+0900 sample 28: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:47+0900 sample 29: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:49+0900 sample 30: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:50+0900 sample 31: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:52+0900 sample 32: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:53+0900 sample 33: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:55+0900 sample 34: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:56+0900 sample 35: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:41:58+0900 sample 36: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:00+0900 sample 37: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:01+0900 sample 38: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:03+0900 sample 39: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:04+0900 sample 40: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:06+0900 sample 41: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:08+0900 sample 42: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:09+0900 sample 43: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:11+0900 sample 44: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:12+0900 sample 45: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:14+0900 sample 46: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:15+0900 sample 47: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:17+0900 sample 48: clean gpu=[1000365:mlxcel-bench-de] -2026-10-09T02:42:18+0900 monitor: 48 samples -2026-10-09T02:42:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log deleted file mode 100644 index cbe1b6f6d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log +++ /dev/null @@ -1,44 +0,0 @@ -2026-10-09T02:42:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:42:23+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:42:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:42:38+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:42:38+0900 sample 1: clean gpu=[] -2026-10-09T02:42:40+0900 sample 2: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:41+0900 sample 3: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:43+0900 sample 4: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:44+0900 sample 5: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:46+0900 sample 6: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:47+0900 sample 7: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:49+0900 sample 8: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:50+0900 sample 9: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:52+0900 sample 10: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:53+0900 sample 11: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:55+0900 sample 12: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:56+0900 sample 13: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:58+0900 sample 14: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:42:59+0900 sample 15: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:01+0900 sample 16: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:03+0900 sample 17: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:04+0900 sample 18: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:05+0900 sample 19: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:07+0900 sample 20: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:08+0900 sample 21: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:10+0900 sample 22: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:11+0900 sample 23: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:13+0900 sample 24: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:15+0900 sample 25: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:17+0900 sample 26: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:18+0900 sample 27: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:20+0900 sample 28: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:22+0900 sample 29: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:24+0900 sample 30: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:25+0900 sample 31: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:27+0900 sample 32: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:28+0900 sample 33: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:30+0900 sample 34: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:31+0900 sample 35: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:33+0900 sample 36: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:34+0900 sample 37: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:36+0900 sample 38: clean gpu=[1055761:mlxcel-bench-de] -2026-10-09T02:43:37+0900 monitor: 38 samples -2026-10-09T02:43:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log deleted file mode 100644 index 98074ebd5..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T01:56:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:56:56+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:57:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:57:11+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:57:11+0900 sample 1: clean gpu=[] -2026-10-09T01:57:13+0900 sample 2: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:15+0900 sample 3: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:16+0900 sample 4: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:18+0900 sample 5: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:19+0900 sample 6: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:21+0900 sample 7: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:22+0900 sample 8: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:24+0900 sample 9: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:25+0900 sample 10: clean gpu=[3752289:mlxcel-bench-de] -2026-10-09T01:57:26+0900 monitor: 10 samples -2026-10-09T01:57:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log deleted file mode 100644 index aadd9d79e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,17 +0,0 @@ -2026-10-09T01:55:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:55:45+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:56:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:56:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:56:00+0900 sample 1: clean gpu=[] -2026-10-09T01:56:02+0900 sample 2: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:03+0900 sample 3: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:05+0900 sample 4: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:06+0900 sample 5: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:08+0900 sample 6: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:09+0900 sample 7: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:11+0900 sample 8: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:12+0900 sample 9: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:14+0900 sample 10: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:15+0900 sample 11: clean gpu=[3713056:mlxcel-bench-de] -2026-10-09T01:56:16+0900 monitor: 11 samples -2026-10-09T01:56:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log deleted file mode 100644 index 918416306..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T01:56:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:56:21+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:56:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:56:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:56:36+0900 sample 1: clean gpu=[] -2026-10-09T01:56:38+0900 sample 2: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:39+0900 sample 3: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:41+0900 sample 4: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:43+0900 sample 5: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:44+0900 sample 6: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:46+0900 sample 7: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:47+0900 sample 8: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:49+0900 sample 9: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:50+0900 sample 10: clean gpu=[3733061:mlxcel-bench-de] -2026-10-09T01:56:51+0900 monitor: 10 samples -2026-10-09T01:56:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log deleted file mode 100644 index 1833fb3e1..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:16:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:16:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:16:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:16:27+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:16:27+0900 sample 1: clean gpu=[] -2026-10-09T02:16:29+0900 sample 2: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:30+0900 sample 3: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:32+0900 sample 4: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:33+0900 sample 5: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:35+0900 sample 6: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:36+0900 sample 7: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:38+0900 sample 8: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:39+0900 sample 9: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:41+0900 sample 10: clean gpu=[193318:mlxcel-bench-de] -2026-10-09T02:16:42+0900 monitor: 10 samples -2026-10-09T02:16:42+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log deleted file mode 100644 index 321a1d51a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:16:47+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:16:47+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:17:01+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:17:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:17:02+0900 sample 1: clean gpu=[] -2026-10-09T02:17:03+0900 sample 2: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:05+0900 sample 3: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:06+0900 sample 4: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:08+0900 sample 5: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:09+0900 sample 6: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:11+0900 sample 7: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:12+0900 sample 8: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:14+0900 sample 9: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:15+0900 sample 10: clean gpu=[211756:mlxcel-bench-de] -2026-10-09T02:17:16+0900 monitor: 10 samples -2026-10-09T02:17:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log deleted file mode 100644 index 75cf8e9f6..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,17 +0,0 @@ -2026-10-09T02:15:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:15:36+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:15:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:15:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:15:51+0900 sample 1: clean gpu=[] -2026-10-09T02:15:53+0900 sample 2: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:15:54+0900 sample 3: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:15:56+0900 sample 4: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:15:57+0900 sample 5: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:15:59+0900 sample 6: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:00+0900 sample 7: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:02+0900 sample 8: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:03+0900 sample 9: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:05+0900 sample 10: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:06+0900 sample 11: clean gpu=[174046:mlxcel-bench-de] -2026-10-09T02:16:07+0900 monitor: 11 samples -2026-10-09T02:16:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log deleted file mode 100644 index 8726437a5..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,17 +0,0 @@ -2026-10-09T02:35:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:35:21+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:35:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:35:36+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:35:36+0900 sample 1: clean gpu=[] -2026-10-09T02:35:38+0900 sample 2: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:39+0900 sample 3: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:41+0900 sample 4: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:42+0900 sample 5: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:44+0900 sample 6: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:45+0900 sample 7: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:47+0900 sample 8: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:48+0900 sample 9: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:50+0900 sample 10: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:51+0900 sample 11: clean gpu=[816050:mlxcel-bench-de] -2026-10-09T02:35:52+0900 monitor: 11 samples -2026-10-09T02:35:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log deleted file mode 100644 index 75c9e792a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:35:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:35:57+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:36:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:36:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:36:12+0900 sample 1: clean gpu=[] -2026-10-09T02:36:14+0900 sample 2: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:15+0900 sample 3: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:17+0900 sample 4: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:18+0900 sample 5: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:20+0900 sample 6: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:21+0900 sample 7: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:23+0900 sample 8: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:24+0900 sample 9: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:26+0900 sample 10: clean gpu=[835434:mlxcel-bench-de] -2026-10-09T02:36:27+0900 monitor: 10 samples -2026-10-09T02:36:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log deleted file mode 100644 index d217ae2e4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:36:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:36:32+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:36:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:36:46+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:36:47+0900 sample 1: clean gpu=[] -2026-10-09T02:36:48+0900 sample 2: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:50+0900 sample 3: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:51+0900 sample 4: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:53+0900 sample 5: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:54+0900 sample 6: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:56+0900 sample 7: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:57+0900 sample 8: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:36:59+0900 sample 9: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:37:00+0900 sample 10: clean gpu=[853905:mlxcel-bench-de] -2026-10-09T02:37:01+0900 monitor: 10 samples -2026-10-09T02:37:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base-null.log deleted file mode 100644 index 1741b9b09..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-08T23:28:27+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T00:20:29+0900 caught SIGTERM: stopping command and monitor -2026-10-09T01:19:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:19:10+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:45:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:45:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:45:57+0900 sample 1: clean gpu=[] -2026-10-09T01:45:59+0900 sample 2: clean gpu=[3351173:mlxcel-bench-de] -2026-10-09T01:46:00+0900 sample 3: clean gpu=[3351173:mlxcel-bench-de] -2026-10-09T01:46:01+0900 monitor: 3 samples -2026-10-09T01:46:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base.log deleted file mode 100644 index 21eedf38a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,12 +0,0 @@ -2026-10-08T21:33:51+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-08T22:19:00+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2709s -2026-10-08T22:19:00+0900 waiting for 60s of idle GPU and no compiler -2026-10-08T22:19:18+0900 idle streak reset after 12s: gpu=[] compilers=[1173803:cargo] -2026-10-08T22:23:33+0900 idle streak reset after 1s: gpu=[] compilers=[1334471:cargo] -2026-10-08T23:05:01+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-08T23:05:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-08T23:05:01+0900 sample 1: clean gpu=[] -2026-10-08T23:05:02+0900 sample 2: clean gpu=[2908527:mlxcel-bench-de] -2026-10-08T23:05:04+0900 sample 3: clean gpu=[2908527:mlxcel-bench-de] -2026-10-08T23:05:05+0900 monitor: 3 samples -2026-10-08T23:05:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_new.log deleted file mode 100644 index 2f164e6b9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,12 +0,0 @@ -2026-10-08T23:05:15+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-08T23:13:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 521s -2026-10-08T23:13:56+0900 waiting for 60s of idle GPU and no compiler -2026-10-08T23:14:31+0900 idle streak reset after 23s: gpu=[3262038:rocminfo] compilers=[3262000:cargo] -2026-10-08T23:23:02+0900 idle streak reset after 14s: gpu=[] compilers=[3571564:cargo,3571605:rustc,3571606:rustc,3571607:rustc,3571608:rustc,3571609:rustc] -2026-10-08T23:28:13+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-08T23:28:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-08T23:28:13+0900 sample 1: clean gpu=[] -2026-10-08T23:28:15+0900 sample 2: clean gpu=[3751605:mlxcel-bench-de] -2026-10-08T23:28:16+0900 sample 3: clean gpu=[3751605:mlxcel-bench-de] -2026-10-08T23:28:17+0900 monitor: 3 samples -2026-10-08T23:28:17+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base-null.log deleted file mode 100644 index 6060db29e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-09T02:05:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:05:19+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:05:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:05:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:05:34+0900 sample 1: clean gpu=[] -2026-10-09T02:05:36+0900 sample 2: clean gpu=[4034722:mlxcel-bench-de] -2026-10-09T02:05:37+0900 sample 3: clean gpu=[4034722:mlxcel-bench-de] -2026-10-09T02:05:38+0900 monitor: 3 samples -2026-10-09T02:05:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base.log deleted file mode 100644 index 694241de9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-09T02:05:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:05:43+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:05:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:05:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:05:58+0900 sample 1: clean gpu=[] -2026-10-09T02:06:00+0900 sample 2: clean gpu=[4047073:mlxcel-bench-de] -2026-10-09T02:06:01+0900 sample 3: clean gpu=[4047073:mlxcel-bench-de] -2026-10-09T02:06:02+0900 monitor: 3 samples -2026-10-09T02:06:02+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_new.log deleted file mode 100644 index bd3302986..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:04:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:04:57+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:05:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:05:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:05:12+0900 sample 1: clean gpu=[] -2026-10-09T02:05:13+0900 sample 2: clean gpu=[4023270:mlxcel-bench-de] -2026-10-09T02:05:14+0900 monitor: 2 samples -2026-10-09T02:05:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base-null.log deleted file mode 100644 index 5b350e952..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:24:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:24:51+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:25:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:25:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:25:06+0900 sample 1: clean gpu=[] -2026-10-09T02:25:07+0900 sample 2: clean gpu=[479954:mlxcel-bench-de] -2026-10-09T02:25:08+0900 monitor: 2 samples -2026-10-09T02:25:08+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base.log deleted file mode 100644 index 8a3bbd7a2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-09T02:25:13+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:25:13+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:25:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:25:28+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:25:28+0900 sample 1: clean gpu=[] -2026-10-09T02:25:30+0900 sample 2: clean gpu=[491350:mlxcel-bench-de] -2026-10-09T02:25:31+0900 sample 3: clean gpu=[491350:mlxcel-bench-de] -2026-10-09T02:25:32+0900 monitor: 3 samples -2026-10-09T02:25:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_new.log deleted file mode 100644 index e15670b14..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:25:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:25:37+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:25:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:25:52+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:25:52+0900 sample 1: clean gpu=[] -2026-10-09T02:25:54+0900 sample 2: clean gpu=[503716:mlxcel-bench-de] -2026-10-09T02:25:55+0900 monitor: 2 samples -2026-10-09T02:25:55+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base-null.log deleted file mode 100644 index dfbcd6f75..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-08T21:22:36+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-08T21:32:09+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 573s -2026-10-08T21:32:09+0900 waiting for 60s of idle GPU and no compiler -2026-10-08T21:33:38+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-08T21:33:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-08T21:33:39+0900 sample 1: clean gpu=[] -2026-10-08T21:33:40+0900 sample 2: clean gpu=[3633468:mlxcel-bench-de] -2026-10-08T21:33:41+0900 monitor: 2 samples -2026-10-08T21:33:41+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base.log deleted file mode 100644 index 0f61f6c29..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,13 +0,0 @@ -2026-10-08T19:17:44+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-08T19:41:27+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 1423s -2026-10-08T19:41:27+0900 waiting for 60s of idle GPU and no compiler -2026-10-08T19:42:09+0900 idle streak reset after 28s: gpu=[] compilers=[3313432:cargo] -2026-10-08T20:39:08+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-08T20:39:08+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-08T20:39:08+0900 sample 1: clean gpu=[] -2026-10-08T20:39:10+0900 sample 2: clean gpu=[] -2026-10-08T20:39:11+0900 sample 3: clean gpu=[1515758:mlxcel-bench-de] -2026-10-08T20:39:13+0900 sample 4: clean gpu=[1515758:mlxcel-bench-de] -2026-10-08T20:39:14+0900 sample 5: clean gpu=[1515758:mlxcel-bench-de] -2026-10-08T20:39:15+0900 monitor: 5 samples -2026-10-08T20:39:15+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_new.log deleted file mode 100644 index 72e8e1dd3..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,9 +0,0 @@ -2026-10-08T20:39:25+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-08T21:20:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2491s -2026-10-08T21:20:56+0900 waiting for 60s of idle GPU and no compiler -2026-10-08T21:22:23+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-08T21:22:23+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-08T21:22:23+0900 sample 1: clean gpu=[] -2026-10-08T21:22:25+0900 sample 2: clean gpu=[3213872:mlxcel-bench-de] -2026-10-08T21:22:26+0900 monitor: 2 samples -2026-10-08T21:22:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base-null.log deleted file mode 100644 index d99261dae..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:04:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:04:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:04:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:04:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:04:26+0900 sample 1: clean gpu=[] -2026-10-09T02:04:28+0900 sample 2: clean gpu=[4000296:mlxcel-bench-de] -2026-10-09T02:04:29+0900 monitor: 2 samples -2026-10-09T02:04:29+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base.log deleted file mode 100644 index 55ff939a4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:04:34+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:04:34+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:04:49+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:04:49+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:04:49+0900 sample 1: clean gpu=[] -2026-10-09T02:04:51+0900 sample 2: clean gpu=[4011756:mlxcel-bench-de] -2026-10-09T02:04:52+0900 monitor: 2 samples -2026-10-09T02:04:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_new.log deleted file mode 100644 index 2dace6b91..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:03:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:03:49+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:04:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:04:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:04:04+0900 sample 1: clean gpu=[] -2026-10-09T02:04:05+0900 sample 2: clean gpu=[3989149:mlxcel-bench-de] -2026-10-09T02:04:06+0900 monitor: 2 samples -2026-10-09T02:04:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base-null.log deleted file mode 100644 index b54f6047c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:23:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:23:43+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:23:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:23:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:23:58+0900 sample 1: clean gpu=[] -2026-10-09T02:24:00+0900 sample 2: clean gpu=[446099:mlxcel-bench-de] -2026-10-09T02:24:01+0900 monitor: 2 samples -2026-10-09T02:24:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base.log deleted file mode 100644 index 87434b7eb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:24:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:24:06+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:24:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:24:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:24:20+0900 sample 1: clean gpu=[] -2026-10-09T02:24:22+0900 sample 2: clean gpu=[457164:mlxcel-bench-de] -2026-10-09T02:24:23+0900 monitor: 2 samples -2026-10-09T02:24:23+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_new.log deleted file mode 100644 index 721abb3c9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,8 +0,0 @@ -2026-10-09T02:24:28+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:24:28+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:24:43+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:24:43+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:24:43+0900 sample 1: clean gpu=[] -2026-10-09T02:24:45+0900 sample 2: clean gpu=[468540:mlxcel-bench-de] -2026-10-09T02:24:46+0900 monitor: 2 samples -2026-10-09T02:24:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log deleted file mode 100644 index 161f3078a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T01:53:41+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:53:41+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:53:56+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:53:56+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:53:56+0900 sample 1: clean gpu=[] -2026-10-09T01:53:58+0900 sample 2: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:53:59+0900 sample 3: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:01+0900 sample 4: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:02+0900 sample 5: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:04+0900 sample 6: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:05+0900 sample 7: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:07+0900 sample 8: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:08+0900 sample 9: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:09+0900 sample 10: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:11+0900 sample 11: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:12+0900 sample 12: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:14+0900 sample 13: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:15+0900 sample 14: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:17+0900 sample 15: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:18+0900 sample 16: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:20+0900 sample 17: clean gpu=[3646745:mlxcel-bench-de] -2026-10-09T01:54:21+0900 monitor: 17 samples -2026-10-09T01:54:21+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base.log deleted file mode 100644 index 901482112..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T01:52:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:52:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:52:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:52:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:52:27+0900 sample 1: clean gpu=[] -2026-10-09T01:52:28+0900 sample 2: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:30+0900 sample 3: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:31+0900 sample 4: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:33+0900 sample 5: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:34+0900 sample 6: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:36+0900 sample 7: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:37+0900 sample 8: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:39+0900 sample 9: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:40+0900 sample 10: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:42+0900 sample 11: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:43+0900 sample 12: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:45+0900 sample 13: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:46+0900 sample 14: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:48+0900 sample 15: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:49+0900 sample 16: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:51+0900 sample 17: clean gpu=[3596927:mlxcel-bench-de] -2026-10-09T01:52:52+0900 monitor: 17 samples -2026-10-09T01:52:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_new.log deleted file mode 100644 index c68c48163..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T01:52:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:52:57+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:53:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:53:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:53:12+0900 sample 1: clean gpu=[] -2026-10-09T01:53:13+0900 sample 2: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:15+0900 sample 3: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:16+0900 sample 4: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:17+0900 sample 5: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:19+0900 sample 6: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:20+0900 sample 7: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:22+0900 sample 8: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:23+0900 sample 9: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:25+0900 sample 10: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:26+0900 sample 11: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:28+0900 sample 12: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:29+0900 sample 13: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:31+0900 sample 14: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:32+0900 sample 15: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:34+0900 sample 16: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:35+0900 sample 17: clean gpu=[3621883:mlxcel-bench-de] -2026-10-09T01:53:36+0900 monitor: 17 samples -2026-10-09T01:53:36+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log deleted file mode 100644 index a80cc0eca..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -2026-10-09T02:12:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:12:50+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:13:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:13:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:13:05+0900 sample 1: clean gpu=[] -2026-10-09T02:13:06+0900 sample 2: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:08+0900 sample 3: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:09+0900 sample 4: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:11+0900 sample 5: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:12+0900 sample 6: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:14+0900 sample 7: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:15+0900 sample 8: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:17+0900 sample 9: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:18+0900 sample 10: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:20+0900 sample 11: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:21+0900 sample 12: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:23+0900 sample 13: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:24+0900 sample 14: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:26+0900 sample 15: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:27+0900 sample 16: clean gpu=[82768:mlxcel-bench-de] -2026-10-09T02:13:28+0900 monitor: 16 samples -2026-10-09T02:13:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base.log deleted file mode 100644 index e4455fc97..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T02:13:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:13:33+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:13:48+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:13:48+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:13:48+0900 sample 1: clean gpu=[] -2026-10-09T02:13:50+0900 sample 2: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:51+0900 sample 3: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:53+0900 sample 4: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:54+0900 sample 5: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:56+0900 sample 6: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:57+0900 sample 7: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:13:59+0900 sample 8: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:00+0900 sample 9: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:02+0900 sample 10: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:03+0900 sample 11: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:05+0900 sample 12: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:06+0900 sample 13: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:08+0900 sample 14: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:09+0900 sample 15: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:11+0900 sample 16: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:12+0900 sample 17: clean gpu=[107711:mlxcel-bench-de] -2026-10-09T02:14:13+0900 monitor: 17 samples -2026-10-09T02:14:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_new.log deleted file mode 100644 index 2897db43d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,30 +0,0 @@ -2026-10-09T02:11:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:11:54+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:12:09+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:12:09+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:12:09+0900 sample 1: clean gpu=[] -2026-10-09T02:12:10+0900 sample 2: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:12+0900 sample 3: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:13+0900 sample 4: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:15+0900 sample 5: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:16+0900 sample 6: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:18+0900 sample 7: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:20+0900 sample 8: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:21+0900 sample 9: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:23+0900 sample 10: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:24+0900 sample 11: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:26+0900 sample 12: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:27+0900 sample 13: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:29+0900 sample 14: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:30+0900 sample 15: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:32+0900 sample 16: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:33+0900 sample 17: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:34+0900 sample 18: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:36+0900 sample 19: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:37+0900 sample 20: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:39+0900 sample 21: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:40+0900 sample 22: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:42+0900 sample 23: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:43+0900 sample 24: clean gpu=[50124:mlxcel-bench-de] -2026-10-09T02:12:44+0900 monitor: 24 samples -2026-10-09T02:12:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log deleted file mode 100644 index de93de17b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T02:31:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:31:48+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:32:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:32:03+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:32:03+0900 sample 1: clean gpu=[] -2026-10-09T02:32:05+0900 sample 2: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:06+0900 sample 3: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:08+0900 sample 4: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:09+0900 sample 5: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:11+0900 sample 6: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:12+0900 sample 7: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:14+0900 sample 8: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:15+0900 sample 9: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:17+0900 sample 10: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:18+0900 sample 11: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:19+0900 sample 12: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:21+0900 sample 13: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:22+0900 sample 14: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:24+0900 sample 15: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:25+0900 sample 16: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:27+0900 sample 17: clean gpu=[700676:mlxcel-bench-de] -2026-10-09T02:32:28+0900 monitor: 17 samples -2026-10-09T02:32:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base.log deleted file mode 100644 index fa79ced51..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T02:32:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:32:33+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:32:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:32:47+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:32:48+0900 sample 1: clean gpu=[] -2026-10-09T02:32:49+0900 sample 2: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:51+0900 sample 3: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:52+0900 sample 4: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:54+0900 sample 5: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:55+0900 sample 6: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:57+0900 sample 7: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:32:58+0900 sample 8: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:00+0900 sample 9: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:01+0900 sample 10: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:03+0900 sample 11: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:04+0900 sample 12: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:06+0900 sample 13: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:07+0900 sample 14: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:09+0900 sample 15: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:10+0900 sample 16: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:12+0900 sample 17: clean gpu=[725677:mlxcel-bench-de] -2026-10-09T02:33:13+0900 monitor: 17 samples -2026-10-09T02:33:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_new.log deleted file mode 100644 index 2913a0b35..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T02:33:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:33:18+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:33:32+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:33:32+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:33:33+0900 sample 1: clean gpu=[] -2026-10-09T02:33:34+0900 sample 2: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:35+0900 sample 3: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:37+0900 sample 4: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:38+0900 sample 5: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:40+0900 sample 6: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:41+0900 sample 7: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:43+0900 sample 8: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:44+0900 sample 9: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:46+0900 sample 10: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:47+0900 sample 11: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:49+0900 sample 12: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:50+0900 sample 13: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:52+0900 sample 14: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:53+0900 sample 15: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:55+0900 sample 16: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:56+0900 sample 17: clean gpu=[750642:mlxcel-bench-de] -2026-10-09T02:33:57+0900 monitor: 17 samples -2026-10-09T02:33:57+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log deleted file mode 100644 index c1c13baf8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T01:51:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:51:37+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:51:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:51:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:51:52+0900 sample 1: clean gpu=[] -2026-10-09T01:51:54+0900 sample 2: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:51:55+0900 sample 3: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:51:57+0900 sample 4: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:51:59+0900 sample 5: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:00+0900 sample 6: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:01+0900 sample 7: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:03+0900 sample 8: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:04+0900 sample 9: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:06+0900 sample 10: clean gpu=[3577931:mlxcel-bench-de] -2026-10-09T01:52:07+0900 monitor: 10 samples -2026-10-09T01:52:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base.log deleted file mode 100644 index b393faedb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,27 +0,0 @@ -2026-10-09T01:50:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:50:11+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:50:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:50:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:50:26+0900 sample 1: clean gpu=[] -2026-10-09T01:50:28+0900 sample 2: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:29+0900 sample 3: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:31+0900 sample 4: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:32+0900 sample 5: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:34+0900 sample 6: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:35+0900 sample 7: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:37+0900 sample 8: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:38+0900 sample 9: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:40+0900 sample 10: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:41+0900 sample 11: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:43+0900 sample 12: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:45+0900 sample 13: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:46+0900 sample 14: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:48+0900 sample 15: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:49+0900 sample 16: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:51+0900 sample 17: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:52+0900 sample 18: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:54+0900 sample 19: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:55+0900 sample 20: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:57+0900 sample 21: clean gpu=[3529109:mlxcel-bench-de] -2026-10-09T01:50:58+0900 monitor: 21 samples -2026-10-09T01:50:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_new.log deleted file mode 100644 index f7d499617..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T01:51:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:51:03+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:51:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:51:17+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:51:18+0900 sample 1: clean gpu=[] -2026-10-09T01:51:19+0900 sample 2: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:21+0900 sample 3: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:22+0900 sample 4: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:24+0900 sample 5: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:25+0900 sample 6: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:27+0900 sample 7: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:28+0900 sample 8: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:30+0900 sample 9: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:31+0900 sample 10: clean gpu=[3558843:mlxcel-bench-de] -2026-10-09T01:51:32+0900 monitor: 10 samples -2026-10-09T01:51:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log deleted file mode 100644 index 59dfefe37..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:10:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:10:44+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:10:59+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:10:59+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:11:00+0900 sample 1: clean gpu=[] -2026-10-09T02:11:01+0900 sample 2: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:03+0900 sample 3: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:04+0900 sample 4: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:06+0900 sample 5: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:07+0900 sample 6: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:09+0900 sample 7: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:10+0900 sample 8: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:12+0900 sample 9: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:13+0900 sample 10: clean gpu=[11664:mlxcel-bench-de] -2026-10-09T02:11:14+0900 monitor: 10 samples -2026-10-09T02:11:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base.log deleted file mode 100644 index 895d4c74e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:11:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:11:19+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:11:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:11:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:11:34+0900 sample 1: clean gpu=[] -2026-10-09T02:11:36+0900 sample 2: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:37+0900 sample 3: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:39+0900 sample 4: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:40+0900 sample 5: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:42+0900 sample 6: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:43+0900 sample 7: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:45+0900 sample 8: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:46+0900 sample 9: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:48+0900 sample 10: clean gpu=[30912:mlxcel-bench-de] -2026-10-09T02:11:49+0900 monitor: 10 samples -2026-10-09T02:11:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_new.log deleted file mode 100644 index f9b69743e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:10:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:10:10+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:10:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:10:25+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:10:25+0900 sample 1: clean gpu=[] -2026-10-09T02:10:27+0900 sample 2: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:28+0900 sample 3: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:29+0900 sample 4: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:31+0900 sample 5: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:32+0900 sample 6: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:34+0900 sample 7: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:35+0900 sample 8: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:37+0900 sample 9: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:38+0900 sample 10: clean gpu=[4186267:mlxcel-bench-de] -2026-10-09T02:10:39+0900 monitor: 10 samples -2026-10-09T02:10:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log deleted file mode 100644 index dacbb1135..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,17 +0,0 @@ -2026-10-09T02:30:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:30:03+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:30:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:30:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:30:19+0900 sample 1: clean gpu=[] -2026-10-09T02:30:20+0900 sample 2: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:21+0900 sample 3: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:23+0900 sample 4: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:24+0900 sample 5: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:26+0900 sample 6: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:27+0900 sample 7: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:29+0900 sample 8: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:30+0900 sample 9: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:32+0900 sample 10: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:33+0900 sample 11: clean gpu=[643275:mlxcel-bench-de] -2026-10-09T02:30:34+0900 monitor: 11 samples -2026-10-09T02:30:34+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base.log deleted file mode 100644 index 21b084363..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:30:39+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:30:39+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:30:54+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:30:54+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:30:54+0900 sample 1: clean gpu=[] -2026-10-09T02:30:56+0900 sample 2: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:30:57+0900 sample 3: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:30:59+0900 sample 4: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:00+0900 sample 5: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:02+0900 sample 6: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:03+0900 sample 7: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:05+0900 sample 8: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:06+0900 sample 9: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:08+0900 sample 10: clean gpu=[663423:mlxcel-bench-de] -2026-10-09T02:31:09+0900 monitor: 10 samples -2026-10-09T02:31:09+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_new.log deleted file mode 100644 index b51392ba2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,16 +0,0 @@ -2026-10-09T02:31:14+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:31:14+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:31:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:31:28+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:31:29+0900 sample 1: clean gpu=[] -2026-10-09T02:31:30+0900 sample 2: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:32+0900 sample 3: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:33+0900 sample 4: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:35+0900 sample 5: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:36+0900 sample 6: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:38+0900 sample 7: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:39+0900 sample 8: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:41+0900 sample 9: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:42+0900 sample 10: clean gpu=[682097:mlxcel-bench-de] -2026-10-09T02:31:43+0900 monitor: 10 samples -2026-10-09T02:31:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base-null.log deleted file mode 100644 index e4bde4d97..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:49:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:49:44+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:49:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:49:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:49:59+0900 sample 1: clean gpu=[] -2026-10-09T01:50:00+0900 sample 2: clean gpu=[3514781:mlxcel-bench-de] -2026-10-09T01:50:02+0900 sample 3: clean gpu=[3514781:mlxcel-bench-de] -2026-10-09T01:50:03+0900 sample 4: clean gpu=[3514781:mlxcel-bench-de] -2026-10-09T01:50:05+0900 sample 5: clean gpu=[3514781:mlxcel-bench-de] -2026-10-09T01:50:06+0900 monitor: 5 samples -2026-10-09T01:50:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base.log deleted file mode 100644 index 6a91bef7a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:48:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:48:50+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:49:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:49:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:49:05+0900 sample 1: clean gpu=[] -2026-10-09T01:49:06+0900 sample 2: clean gpu=[3486042:mlxcel-bench-de] -2026-10-09T01:49:08+0900 sample 3: clean gpu=[3486042:mlxcel-bench-de] -2026-10-09T01:49:09+0900 sample 4: clean gpu=[3486042:mlxcel-bench-de] -2026-10-09T01:49:11+0900 sample 5: clean gpu=[3486042:mlxcel-bench-de] -2026-10-09T01:49:12+0900 monitor: 5 samples -2026-10-09T01:49:12+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_new.log deleted file mode 100644 index adfb25015..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:49:17+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:49:17+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:49:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:49:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T01:49:32+0900 sample 1: clean gpu=[] -2026-10-09T01:49:33+0900 sample 2: clean gpu=[3500474:mlxcel-bench-de] -2026-10-09T01:49:35+0900 sample 3: clean gpu=[3500474:mlxcel-bench-de] -2026-10-09T01:49:36+0900 sample 4: clean gpu=[3500474:mlxcel-bench-de] -2026-10-09T01:49:38+0900 sample 5: clean gpu=[3500474:mlxcel-bench-de] -2026-10-09T01:49:39+0900 monitor: 5 samples -2026-10-09T01:49:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base-null.log deleted file mode 100644 index 37c2f6162..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:09:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:09:16+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:09:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:09:31+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:09:31+0900 sample 1: clean gpu=[] -2026-10-09T02:09:33+0900 sample 2: clean gpu=[4158223:mlxcel-bench-de] -2026-10-09T02:09:34+0900 sample 3: clean gpu=[4158223:mlxcel-bench-de] -2026-10-09T02:09:36+0900 sample 4: clean gpu=[4158223:mlxcel-bench-de] -2026-10-09T02:09:37+0900 sample 5: clean gpu=[4158223:mlxcel-bench-de] -2026-10-09T02:09:38+0900 monitor: 5 samples -2026-10-09T02:09:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base.log deleted file mode 100644 index 520ff5375..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:09:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:09:43+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:09:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:09:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:09:58+0900 sample 1: clean gpu=[] -2026-10-09T02:10:00+0900 sample 2: clean gpu=[4172265:mlxcel-bench-de] -2026-10-09T02:10:01+0900 sample 3: clean gpu=[4172265:mlxcel-bench-de] -2026-10-09T02:10:02+0900 sample 4: clean gpu=[4172265:mlxcel-bench-de] -2026-10-09T02:10:04+0900 sample 5: clean gpu=[4172265:mlxcel-bench-de] -2026-10-09T02:10:05+0900 monitor: 5 samples -2026-10-09T02:10:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_new.log deleted file mode 100644 index 8d0535c4b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:08:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:08:49+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:09:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:09:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:09:04+0900 sample 1: clean gpu=[] -2026-10-09T02:09:05+0900 sample 2: clean gpu=[4144067:mlxcel-bench-de] -2026-10-09T02:09:07+0900 sample 3: clean gpu=[4144067:mlxcel-bench-de] -2026-10-09T02:09:08+0900 sample 4: clean gpu=[4144067:mlxcel-bench-de] -2026-10-09T02:09:10+0900 sample 5: clean gpu=[4144067:mlxcel-bench-de] -2026-10-09T02:09:11+0900 monitor: 5 samples -2026-10-09T02:09:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base-null.log deleted file mode 100644 index 1aa849f12..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:28:42+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:28:42+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:28:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:28:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:28:57+0900 sample 1: clean gpu=[] -2026-10-09T02:28:59+0900 sample 2: clean gpu=[600016:mlxcel-bench-de] -2026-10-09T02:29:00+0900 sample 3: clean gpu=[600016:mlxcel-bench-de] -2026-10-09T02:29:02+0900 sample 4: clean gpu=[600016:mlxcel-bench-de] -2026-10-09T02:29:03+0900 sample 5: clean gpu=[600016:mlxcel-bench-de] -2026-10-09T02:29:04+0900 monitor: 5 samples -2026-10-09T02:29:04+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base.log deleted file mode 100644 index 6a6fbb3a2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:29:09+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:29:09+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:29:24+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:29:24+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:29:24+0900 sample 1: clean gpu=[] -2026-10-09T02:29:26+0900 sample 2: clean gpu=[614459:mlxcel-bench-de] -2026-10-09T02:29:27+0900 sample 3: clean gpu=[614459:mlxcel-bench-de] -2026-10-09T02:29:29+0900 sample 4: clean gpu=[614459:mlxcel-bench-de] -2026-10-09T02:29:30+0900 sample 5: clean gpu=[614459:mlxcel-bench-de] -2026-10-09T02:29:31+0900 monitor: 5 samples -2026-10-09T02:29:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_new.log deleted file mode 100644 index 6e64b9684..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:29:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:29:36+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:29:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:29:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -2026-10-09T02:29:51+0900 sample 1: clean gpu=[] -2026-10-09T02:29:53+0900 sample 2: clean gpu=[628844:mlxcel-bench-de] -2026-10-09T02:29:54+0900 sample 3: clean gpu=[628844:mlxcel-bench-de] -2026-10-09T02:29:56+0900 sample 4: clean gpu=[628844:mlxcel-bench-de] -2026-10-09T02:29:57+0900 sample 5: clean gpu=[628844:mlxcel-bench-de] -2026-10-09T02:29:58+0900 monitor: 5 samples -2026-10-09T02:29:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base-null.log deleted file mode 100644 index 0c4b5c694..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T01:48:24+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:48:24+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:48:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:48:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:48:39+0900 sample 1: clean gpu=[] -2026-10-09T01:48:41+0900 sample 2: clean gpu=[3468221:mlxcel-bench-de] -2026-10-09T01:48:42+0900 sample 3: clean gpu=[3468221:mlxcel-bench-de] -2026-10-09T01:48:44+0900 sample 4: clean gpu=[3468221:mlxcel-bench-de] -2026-10-09T01:48:45+0900 monitor: 4 samples -2026-10-09T01:48:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base.log deleted file mode 100644 index 4c60b7edb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T01:47:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:47:31+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:47:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:47:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:47:46+0900 sample 1: clean gpu=[] -2026-10-09T01:47:48+0900 sample 2: clean gpu=[3408879:mlxcel-bench-de] -2026-10-09T01:47:49+0900 sample 3: clean gpu=[3408879:mlxcel-bench-de] -2026-10-09T01:47:51+0900 sample 4: clean gpu=[3408879:mlxcel-bench-de] -2026-10-09T01:47:52+0900 sample 5: clean gpu=[3408879:mlxcel-bench-de] -2026-10-09T01:47:53+0900 monitor: 5 samples -2026-10-09T01:47:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_new.log deleted file mode 100644 index ee5b08fc8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_new.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T01:47:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:47:58+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:48:13+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:48:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:48:13+0900 sample 1: clean gpu=[] -2026-10-09T01:48:15+0900 sample 2: clean gpu=[3437165:mlxcel-bench-de] -2026-10-09T01:48:16+0900 sample 3: clean gpu=[3437165:mlxcel-bench-de] -2026-10-09T01:48:18+0900 sample 4: clean gpu=[3437165:mlxcel-bench-de] -2026-10-09T01:48:19+0900 monitor: 4 samples -2026-10-09T01:48:19+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base-null.log deleted file mode 100644 index 0a064da03..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:07:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:07:58+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:08:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:08:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:08:13+0900 sample 1: clean gpu=[] -2026-10-09T02:08:14+0900 sample 2: clean gpu=[4117635:mlxcel-bench-de] -2026-10-09T02:08:16+0900 sample 3: clean gpu=[4117635:mlxcel-bench-de] -2026-10-09T02:08:17+0900 sample 4: clean gpu=[4117635:mlxcel-bench-de] -2026-10-09T02:08:18+0900 monitor: 4 samples -2026-10-09T02:08:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base.log deleted file mode 100644 index da0dfe82c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:08:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:08:23+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:08:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:08:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:08:38+0900 sample 1: clean gpu=[] -2026-10-09T02:08:40+0900 sample 2: clean gpu=[4130867:mlxcel-bench-de] -2026-10-09T02:08:41+0900 sample 3: clean gpu=[4130867:mlxcel-bench-de] -2026-10-09T02:08:43+0900 sample 4: clean gpu=[4130867:mlxcel-bench-de] -2026-10-09T02:08:44+0900 monitor: 4 samples -2026-10-09T02:08:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_new.log deleted file mode 100644 index a030fa689..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_new.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:07:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:07:32+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:07:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:07:47+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:07:47+0900 sample 1: clean gpu=[] -2026-10-09T02:07:49+0900 sample 2: clean gpu=[4104326:mlxcel-bench-de] -2026-10-09T02:07:50+0900 sample 3: clean gpu=[4104326:mlxcel-bench-de] -2026-10-09T02:07:52+0900 sample 4: clean gpu=[4104326:mlxcel-bench-de] -2026-10-09T02:07:53+0900 monitor: 4 samples -2026-10-09T02:07:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base-null.log deleted file mode 100644 index 8de1c1f88..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:27:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:27:25+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:27:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:27:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:27:40+0900 sample 1: clean gpu=[] -2026-10-09T02:27:42+0900 sample 2: clean gpu=[559971:mlxcel-bench-de] -2026-10-09T02:27:43+0900 sample 3: clean gpu=[559971:mlxcel-bench-de] -2026-10-09T02:27:45+0900 sample 4: clean gpu=[559971:mlxcel-bench-de] -2026-10-09T02:27:46+0900 monitor: 4 samples -2026-10-09T02:27:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base.log deleted file mode 100644 index f227e46b1..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:27:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:27:51+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:28:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:28:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:28:06+0900 sample 1: clean gpu=[] -2026-10-09T02:28:07+0900 sample 2: clean gpu=[573242:mlxcel-bench-de] -2026-10-09T02:28:09+0900 sample 3: clean gpu=[573242:mlxcel-bench-de] -2026-10-09T02:28:10+0900 sample 4: clean gpu=[573242:mlxcel-bench-de] -2026-10-09T02:28:11+0900 monitor: 4 samples -2026-10-09T02:28:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_new.log deleted file mode 100644 index 998d1089b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_new.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:28:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:28:16+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:28:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:28:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:28:31+0900 sample 1: clean gpu=[] -2026-10-09T02:28:33+0900 sample 2: clean gpu=[586534:mlxcel-bench-de] -2026-10-09T02:28:34+0900 sample 3: clean gpu=[586534:mlxcel-bench-de] -2026-10-09T02:28:36+0900 sample 4: clean gpu=[586534:mlxcel-bench-de] -2026-10-09T02:28:37+0900 monitor: 4 samples -2026-10-09T02:28:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log deleted file mode 100644 index 0648dfb72..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log +++ /dev/null @@ -1,20 +0,0 @@ -2026-10-09T01:59:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:59:12+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:59:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:59:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:59:27+0900 sample 1: clean gpu=[] -2026-10-09T01:59:28+0900 sample 2: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:30+0900 sample 3: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:31+0900 sample 4: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:33+0900 sample 5: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:35+0900 sample 6: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:36+0900 sample 7: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:38+0900 sample 8: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:39+0900 sample 9: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:41+0900 sample 10: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:42+0900 sample 11: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:44+0900 sample 12: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:45+0900 sample 13: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:47+0900 sample 14: clean gpu=[3829107:mlxcel-bench-de] -2026-10-09T01:59:48+0900 monitor: 14 samples -2026-10-09T01:59:48+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log deleted file mode 100644 index f8fc978d1..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log +++ /dev/null @@ -1,30 +0,0 @@ -2026-10-09T01:57:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:57:31+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:57:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:57:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:57:46+0900 sample 1: clean gpu=[] -2026-10-09T01:57:48+0900 sample 2: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:49+0900 sample 3: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:51+0900 sample 4: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:52+0900 sample 5: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:54+0900 sample 6: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:56+0900 sample 7: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:57+0900 sample 8: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:57:59+0900 sample 9: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:00+0900 sample 10: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:02+0900 sample 11: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:03+0900 sample 12: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:05+0900 sample 13: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:06+0900 sample 14: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:08+0900 sample 15: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:09+0900 sample 16: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:11+0900 sample 17: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:12+0900 sample 18: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:14+0900 sample 19: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:15+0900 sample 20: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:17+0900 sample 21: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:18+0900 sample 22: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:20+0900 sample 23: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:21+0900 sample 24: clean gpu=[3771509:mlxcel-bench-de] -2026-10-09T01:58:22+0900 monitor: 24 samples -2026-10-09T01:58:22+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log deleted file mode 100644 index d2f0b65aa..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log +++ /dev/null @@ -1,22 +0,0 @@ -2026-10-09T01:58:27+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:58:27+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:58:42+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:58:42+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:58:43+0900 sample 1: clean gpu=[] -2026-10-09T01:58:44+0900 sample 2: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:46+0900 sample 3: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:47+0900 sample 4: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:49+0900 sample 5: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:50+0900 sample 6: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:52+0900 sample 7: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:53+0900 sample 8: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:55+0900 sample 9: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:56+0900 sample 10: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:58+0900 sample 11: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:58:59+0900 sample 12: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:59:01+0900 sample 13: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:59:03+0900 sample 14: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:59:04+0900 sample 15: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:59:06+0900 sample 16: clean gpu=[3804148:mlxcel-bench-de] -2026-10-09T01:59:07+0900 monitor: 16 samples -2026-10-09T01:59:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log deleted file mode 100644 index 2d5b23539..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -2026-10-09T02:18:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:18:19+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:18:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:18:33+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:18:34+0900 sample 1: clean gpu=[] -2026-10-09T02:18:35+0900 sample 2: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:37+0900 sample 3: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:38+0900 sample 4: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:40+0900 sample 5: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:41+0900 sample 6: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:43+0900 sample 7: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:44+0900 sample 8: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:46+0900 sample 9: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:47+0900 sample 10: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:49+0900 sample 11: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:50+0900 sample 12: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:52+0900 sample 13: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:54+0900 sample 14: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:55+0900 sample 15: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:57+0900 sample 16: clean gpu=[262691:mlxcel-bench-de] -2026-10-09T02:18:58+0900 monitor: 16 samples -2026-10-09T02:18:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log deleted file mode 100644 index f2e7bb2d9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log +++ /dev/null @@ -1,21 +0,0 @@ -2026-10-09T02:19:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:19:03+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:19:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:19:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:19:18+0900 sample 1: clean gpu=[] -2026-10-09T02:19:19+0900 sample 2: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:21+0900 sample 3: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:22+0900 sample 4: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:24+0900 sample 5: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:25+0900 sample 6: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:27+0900 sample 7: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:28+0900 sample 8: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:30+0900 sample 9: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:31+0900 sample 10: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:33+0900 sample 11: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:34+0900 sample 12: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:36+0900 sample 13: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:37+0900 sample 14: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:39+0900 sample 15: clean gpu=[286819:mlxcel-bench-de] -2026-10-09T02:19:40+0900 monitor: 15 samples -2026-10-09T02:19:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log deleted file mode 100644 index 31a731b87..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log +++ /dev/null @@ -1,31 +0,0 @@ -2026-10-09T02:17:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:17:21+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:17:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:17:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:17:36+0900 sample 1: clean gpu=[] -2026-10-09T02:17:38+0900 sample 2: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:39+0900 sample 3: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:41+0900 sample 4: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:42+0900 sample 5: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:44+0900 sample 6: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:45+0900 sample 7: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:47+0900 sample 8: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:48+0900 sample 9: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:50+0900 sample 10: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:51+0900 sample 11: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:53+0900 sample 12: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:54+0900 sample 13: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:56+0900 sample 14: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:57+0900 sample 15: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:17:59+0900 sample 16: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:00+0900 sample 17: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:02+0900 sample 18: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:04+0900 sample 19: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:05+0900 sample 20: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:07+0900 sample 21: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:08+0900 sample 22: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:10+0900 sample 23: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:11+0900 sample 24: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:13+0900 sample 25: clean gpu=[230262:mlxcel-bench-de] -2026-10-09T02:18:14+0900 monitor: 25 samples -2026-10-09T02:18:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log deleted file mode 100644 index 1270b6386..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log +++ /dev/null @@ -1,31 +0,0 @@ -2026-10-09T02:37:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:37:06+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:37:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:37:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:37:21+0900 sample 1: clean gpu=[] -2026-10-09T02:37:23+0900 sample 2: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:24+0900 sample 3: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:26+0900 sample 4: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:27+0900 sample 5: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:29+0900 sample 6: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:31+0900 sample 7: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:32+0900 sample 8: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:34+0900 sample 9: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:35+0900 sample 10: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:37+0900 sample 11: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:38+0900 sample 12: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:40+0900 sample 13: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:41+0900 sample 14: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:43+0900 sample 15: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:44+0900 sample 16: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:46+0900 sample 17: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:47+0900 sample 18: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:49+0900 sample 19: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:50+0900 sample 20: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:51+0900 sample 21: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:53+0900 sample 22: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:54+0900 sample 23: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:56+0900 sample 24: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:57+0900 sample 25: clean gpu=[872333:mlxcel-bench-de] -2026-10-09T02:37:58+0900 monitor: 25 samples -2026-10-09T02:37:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log deleted file mode 100644 index 28d486c30..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log +++ /dev/null @@ -1,23 +0,0 @@ -2026-10-09T02:38:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:38:03+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:38:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:38:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:38:18+0900 sample 1: clean gpu=[] -2026-10-09T02:38:20+0900 sample 2: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:21+0900 sample 3: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:23+0900 sample 4: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:24+0900 sample 5: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:26+0900 sample 6: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:27+0900 sample 7: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:29+0900 sample 8: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:30+0900 sample 9: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:32+0900 sample 10: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:33+0900 sample 11: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:35+0900 sample 12: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:36+0900 sample 13: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:38+0900 sample 14: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:39+0900 sample 15: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:41+0900 sample 16: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:42+0900 sample 17: clean gpu=[904504:mlxcel-bench-de] -2026-10-09T02:38:43+0900 monitor: 17 samples -2026-10-09T02:38:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log deleted file mode 100644 index a7718e62e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log +++ /dev/null @@ -1,21 +0,0 @@ -2026-10-09T02:38:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:38:48+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:39:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:39:03+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:39:03+0900 sample 1: clean gpu=[] -2026-10-09T02:39:05+0900 sample 2: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:06+0900 sample 3: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:08+0900 sample 4: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:09+0900 sample 5: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:11+0900 sample 6: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:13+0900 sample 7: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:14+0900 sample 8: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:16+0900 sample 9: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:17+0900 sample 10: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:19+0900 sample 11: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:20+0900 sample 12: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:22+0900 sample 13: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:23+0900 sample 14: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:24+0900 sample 15: clean gpu=[930024:mlxcel-bench-de] -2026-10-09T02:39:25+0900 monitor: 15 samples -2026-10-09T02:39:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log deleted file mode 100644 index fe68df73c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T01:55:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:55:19+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:55:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:55:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:55:34+0900 sample 1: clean gpu=[] -2026-10-09T01:55:36+0900 sample 2: clean gpu=[3699816:mlxcel-bench-de] -2026-10-09T01:55:38+0900 sample 3: clean gpu=[3699816:mlxcel-bench-de] -2026-10-09T01:55:39+0900 sample 4: clean gpu=[3699816:mlxcel-bench-de] -2026-10-09T01:55:40+0900 monitor: 4 samples -2026-10-09T01:55:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base.log deleted file mode 100644 index 13fb69f38..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base.log +++ /dev/null @@ -1,12 +0,0 @@ -2026-10-09T01:54:26+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:54:26+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:54:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:54:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:54:41+0900 sample 1: clean gpu=[] -2026-10-09T01:54:42+0900 sample 2: clean gpu=[3671563:mlxcel-bench-de] -2026-10-09T01:54:44+0900 sample 3: clean gpu=[3671563:mlxcel-bench-de] -2026-10-09T01:54:45+0900 sample 4: clean gpu=[3671563:mlxcel-bench-de] -2026-10-09T01:54:46+0900 sample 5: clean gpu=[3671563:mlxcel-bench-de] -2026-10-09T01:54:48+0900 sample 6: clean gpu=[3671563:mlxcel-bench-de] -2026-10-09T01:54:49+0900 monitor: 6 samples -2026-10-09T01:54:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_new.log deleted file mode 100644 index 14f6606ca..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_new.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T01:54:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T01:54:54+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T01:55:08+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T01:55:08+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T01:55:09+0900 sample 1: clean gpu=[] -2026-10-09T01:55:10+0900 sample 2: clean gpu=[3686559:mlxcel-bench-de] -2026-10-09T01:55:12+0900 sample 3: clean gpu=[3686559:mlxcel-bench-de] -2026-10-09T01:55:13+0900 sample 4: clean gpu=[3686559:mlxcel-bench-de] -2026-10-09T01:55:14+0900 monitor: 4 samples -2026-10-09T01:55:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log deleted file mode 100644 index b256f3565..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:14:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:14:45+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:15:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:15:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:15:00+0900 sample 1: clean gpu=[] -2026-10-09T02:15:02+0900 sample 2: clean gpu=[147991:mlxcel-bench-de] -2026-10-09T02:15:03+0900 sample 3: clean gpu=[147991:mlxcel-bench-de] -2026-10-09T02:15:05+0900 sample 4: clean gpu=[147991:mlxcel-bench-de] -2026-10-09T02:15:06+0900 monitor: 4 samples -2026-10-09T02:15:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base.log deleted file mode 100644 index bda10692f..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:15:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:15:11+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:15:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:15:25+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:15:26+0900 sample 1: clean gpu=[] -2026-10-09T02:15:27+0900 sample 2: clean gpu=[161122:mlxcel-bench-de] -2026-10-09T02:15:29+0900 sample 3: clean gpu=[161122:mlxcel-bench-de] -2026-10-09T02:15:30+0900 sample 4: clean gpu=[161122:mlxcel-bench-de] -2026-10-09T02:15:31+0900 monitor: 4 samples -2026-10-09T02:15:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_new.log deleted file mode 100644 index 4b5b60c21..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_new.log +++ /dev/null @@ -1,11 +0,0 @@ -2026-10-09T02:14:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:14:18+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:14:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:14:33+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:14:33+0900 sample 1: clean gpu=[] -2026-10-09T02:14:35+0900 sample 2: clean gpu=[133603:mlxcel-bench-de] -2026-10-09T02:14:36+0900 sample 3: clean gpu=[133603:mlxcel-bench-de] -2026-10-09T02:14:38+0900 sample 4: clean gpu=[133603:mlxcel-bench-de] -2026-10-09T02:14:39+0900 sample 5: clean gpu=[133603:mlxcel-bench-de] -2026-10-09T02:14:40+0900 monitor: 5 samples -2026-10-09T02:14:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log deleted file mode 100644 index 583bb9b7b..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,12 +0,0 @@ -2026-10-09T02:34:02+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:34:02+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:34:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:34:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:34:17+0900 sample 1: clean gpu=[] -2026-10-09T02:34:19+0900 sample 2: clean gpu=[775576:mlxcel-bench-de] -2026-10-09T02:34:20+0900 sample 3: clean gpu=[775576:mlxcel-bench-de] -2026-10-09T02:34:22+0900 sample 4: clean gpu=[775576:mlxcel-bench-de] -2026-10-09T02:34:23+0900 sample 5: clean gpu=[775576:mlxcel-bench-de] -2026-10-09T02:34:24+0900 sample 6: clean gpu=[775576:mlxcel-bench-de] -2026-10-09T02:34:25+0900 monitor: 6 samples -2026-10-09T02:34:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base.log deleted file mode 100644 index 0785bd5a8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:34:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:34:30+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:34:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:34:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:34:45+0900 sample 1: clean gpu=[] -2026-10-09T02:34:47+0900 sample 2: clean gpu=[790307:mlxcel-bench-de] -2026-10-09T02:34:48+0900 sample 3: clean gpu=[790307:mlxcel-bench-de] -2026-10-09T02:34:50+0900 sample 4: clean gpu=[790307:mlxcel-bench-de] -2026-10-09T02:34:51+0900 monitor: 4 samples -2026-10-09T02:34:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_new.log deleted file mode 100644 index 17728d15a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_new.log +++ /dev/null @@ -1,10 +0,0 @@ -2026-10-09T02:34:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -2026-10-09T02:34:56+0900 waiting for 10s of idle GPU and no compiler -2026-10-09T02:35:10+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -2026-10-09T02:35:10+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -2026-10-09T02:35:11+0900 sample 1: clean gpu=[] -2026-10-09T02:35:12+0900 sample 2: clean gpu=[803170:mlxcel-bench-de] -2026-10-09T02:35:14+0900 sample 3: clean gpu=[803170:mlxcel-bench-de] -2026-10-09T02:35:15+0900 sample 4: clean gpu=[803170:mlxcel-bench-de] -2026-10-09T02:35:16+0900 monitor: 4 samples -2026-10-09T02:35:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw-logs.tar.gz b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw-logs.tar.gz new file mode 100644 index 000000000..fc22a43bb Binary files /dev/null and b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw-logs.tar.gz differ diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log deleted file mode 100644 index 7bc5c487e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:47:04+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:47:04+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:47:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:47:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.171 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.42 GB, active+cache 5.18 GB, phase peak 6.17 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 510.48 ms (1002.98 tok/s) - Decode: 3399.29 ms (37.65 tok/s) - MLX peak memory: 6.17 GB -rocm_gpu_guard: 2026-10-09T01:47:26+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:47:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log deleted file mode 100644 index 701febea2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:46:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:46:06+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:46:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:46:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.207 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.68 GB, active+cache 5.43 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 506.72 ms (1010.42 tok/s) - Decode: 3397.42 ms (37.68 tok/s) - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T01:46:31+0900 monitor: 7 samples -rocm_gpu_guard: 2026-10-09T01:46:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log deleted file mode 100644 index 8d333433a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:46:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:46:36+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:46:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:46:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.200 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 506.60 ms (1010.65 tok/s) - Decode: 3356.95 ms (38.13 tok/s) - Decode path: engine - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T01:46:59+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:46:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log deleted file mode 100644 index 9fc0d9f1a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:06:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:06:37+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:06:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:06:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.186 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.69 GB, active+cache 5.44 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 513.71 ms (996.67 tok/s) - Decode: 3382.28 ms (37.84 tok/s) - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T02:07:00+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:07:00+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log deleted file mode 100644 index 11e0ad7bf..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:07:05+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:07:05+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:07:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:07:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.189 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 507.96 ms (1007.95 tok/s) - Decode: 3388.98 ms (37.77 tok/s) - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T02:07:27+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:07:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log deleted file mode 100644 index cb54b69d5..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:06:07+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:06:07+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:06:22+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:06:22+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.190 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.42 GB, active+cache 5.18 GB, phase peak 5.11 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 932.40 ms (549.12 tok/s) - Decode: 3416.33 ms (37.47 tok/s) - Decode path: engine - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T02:06:32+0900 monitor: 7 samples -rocm_gpu_guard: 2026-10-09T02:06:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log deleted file mode 100644 index c0c263329..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:26:00+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:26:00+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:26:15+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:26:15+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.193 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.68 GB, active+cache 5.43 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 516.54 ms (991.20 tok/s) - Decode: 3403.14 ms (37.61 tok/s) - MLX peak memory: 6.15 GB -rocm_gpu_guard: 2026-10-09T02:26:25+0900 monitor: 7 samples -rocm_gpu_guard: 2026-10-09T02:26:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log deleted file mode 100644 index aff3ad82c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:26:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:26:30+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:26:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:26:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.190 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.24 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 509.34 ms (1005.22 tok/s) - Decode: 3397.71 ms (37.67 tok/s) - MLX peak memory: 6.24 GB -rocm_gpu_guard: 2026-10-09T02:26:53+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:26:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log deleted file mode 100644 index 16455f3bc..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:26:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:26:58+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:27:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:27:12+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.191 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 5.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.25 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 513.78 ms (996.53 tok/s) - Decode: 3398.23 ms (37.67 tok/s) - Decode path: engine - MLX peak memory: 6.25 GB -rocm_gpu_guard: 2026-10-09T02:27:20+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:27:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log deleted file mode 100644 index 1efd8e5a7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:02:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:02:31+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:02:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:02:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.055 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19862.45 ms (25.78 tok/s) - Decode: 12226.42 ms (10.47 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:03:44+0900 monitor: 38 samples -rocm_gpu_guard: 2026-10-09T02:03:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log deleted file mode 100644 index 53e1d3596..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:59:53+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:59:53+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:00:07+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:00:07+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.057 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.20 GB, active+cache 28.60 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.36 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19714.36 ms (25.97 tok/s) - Decode: 13213.77 ms (9.69 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:01:07+0900 monitor: 39 samples -rocm_gpu_guard: 2026-10-09T02:01:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log deleted file mode 100644 index d6ee35222..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:01:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:01:12+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:01:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:01:27+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.057 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.22 GB, active+cache 26.61 GB, phase peak 27.51 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.22 GB, active+cache 26.61 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19839.49 ms (25.81 tok/s) - Decode: 13411.61 ms (9.54 tok/s) - Decode path: engine - MLX peak memory: 27.51 GB -rocm_gpu_guard: 2026-10-09T02:02:26+0900 monitor: 39 samples -rocm_gpu_guard: 2026-10-09T02:02:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log deleted file mode 100644 index 7c8d955a1..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:21:04+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:21:04+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:21:19+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:21:19+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.053 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 1.72 GB, active+cache 28.11 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19823.17 ms (25.83 tok/s) - Decode: 12615.91 ms (10.15 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:22:20+0900 monitor: 40 samples -rocm_gpu_guard: 2026-10-09T02:22:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log deleted file mode 100644 index 1458cbfe7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:22:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:22:25+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:22:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:22:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.070 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19874.59 ms (25.76 tok/s) - Decode: 12839.71 ms (9.97 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:23:38+0900 monitor: 38 samples -rocm_gpu_guard: 2026-10-09T02:23:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log deleted file mode 100644 index d8c70518a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:19:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:19:45+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:20:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:20:00+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.054 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.34 GB, active+cache 26.73 GB, phase peak 27.51 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.33 GB, active+cache 26.73 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19734.06 ms (25.94 tok/s) - Decode: 12681.93 ms (10.09 tok/s) - Decode path: engine - MLX peak memory: 27.51 GB -rocm_gpu_guard: 2026-10-09T02:20:59+0900 monitor: 39 samples -rocm_gpu_guard: 2026-10-09T02:20:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log deleted file mode 100644 index 113b99b6d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:39:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:39:31+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:39:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:39:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.055 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19711.80 ms (25.97 tok/s) - Decode: 13062.31 ms (9.80 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:40:45+0900 monitor: 39 samples -rocm_gpu_guard: 2026-10-09T02:40:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log deleted file mode 100644 index 0ddc91c4e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:40:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:40:50+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:41:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:41:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.054 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19663.82 ms (26.04 tok/s) - Decode: 12839.24 ms (9.97 tok/s) - MLX peak memory: 27.52 GB -rocm_gpu_guard: 2026-10-09T02:42:18+0900 monitor: 48 samples -rocm_gpu_guard: 2026-10-09T02:42:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log deleted file mode 100644 index 9ae178084..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:42:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:42:23+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:42:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:42:38+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.065 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.37 GB, active+cache 26.77 GB, phase peak 27.51 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) -[Memory] after measured pass: active 26.40 GB, cache 0.34 GB, active+cache 26.74 GB, phase peak 27.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 19878.37 ms (25.76 tok/s) - Decode: 12548.46 ms (10.20 tok/s) - Decode path: engine - MLX peak memory: 27.51 GB -rocm_gpu_guard: 2026-10-09T02:43:37+0900 monitor: 38 samples -rocm_gpu_guard: 2026-10-09T02:43:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log deleted file mode 100644 index 1b37a199e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:56:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:56:56+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:57:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:57:11+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.321 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1851.40 ms (276.55 tok/s) - Decode: 1697.24 ms (75.42 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:57:26+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T01:57:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log deleted file mode 100644 index 19ea57f93..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:55:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:55:45+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:56:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:56:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.327 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1827.53 ms (280.16 tok/s) - Decode: 1698.51 ms (75.36 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:56:16+0900 monitor: 11 samples -rocm_gpu_guard: 2026-10-09T01:56:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log deleted file mode 100644 index 66f2305b7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:56:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:56:21+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:56:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:56:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.330 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.17 GB, active+cache 18.18 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.01 GB, cache 0.26 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1719.00 ms (297.85 tok/s) - Decode: 1695.18 ms (75.51 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:56:51+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T01:56:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log deleted file mode 100644 index faad783e9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:16:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:16:12+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:16:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:16:27+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.330 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1819.72 ms (281.36 tok/s) - Decode: 1698.86 ms (75.34 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:16:42+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:16:42+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log deleted file mode 100644 index 04b8096e8..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:16:47+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:16:47+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:17:01+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:17:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.329 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.31 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.14 GB, active+cache 18.21 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1767.78 ms (289.63 tok/s) - Decode: 1703.63 ms (75.13 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:17:16+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:17:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log deleted file mode 100644 index 2d77f0b75..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:15:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:15:36+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:15:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:15:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.337 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.18 GB, active+cache 18.19 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.01 GB, cache 0.26 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 2000.43 ms (255.95 tok/s) - Decode: 1700.67 ms (75.26 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:16:07+0900 monitor: 11 samples -rocm_gpu_guard: 2026-10-09T02:16:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log deleted file mode 100644 index 301464bad..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:35:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:35:21+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:35:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:35:36+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.332 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.31 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1878.15 ms (272.61 tok/s) - Decode: 1696.57 ms (75.45 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:35:52+0900 monitor: 11 samples -rocm_gpu_guard: 2026-10-09T02:35:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log deleted file mode 100644 index 89793f6f2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,27 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:35:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:35:57+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:36:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:36:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[NemotronH] Loading config... -[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) -[NemotronH] Loading weights from safetensors... -[NemotronH] Building model... -[NemotronH] Model loaded successfully -[Load] wall: 0.333 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1794.91 ms (285.25 tok/s) - Decode: 1700.18 ms (75.29 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:36:27+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:36:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log deleted file mode 100644 index 38c5bd715..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:36:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:36:32+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:36:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:36:46+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.336 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.17 GB, active+cache 18.18 GB, phase peak 18.17 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 18.01 GB, cache 0.23 GB, active+cache 18.24 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1856.01 ms (275.86 tok/s) - Decode: 1694.83 ms (75.52 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:37:01+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:37:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log deleted file mode 100644 index 04c95815c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:19:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:19:10+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:45:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:45:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.273 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.80 GB, active+cache 2.18 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 448.08 ms (4570.58 tok/s) - Decode: 590.32 ms (216.83 tok/s) - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T01:46:01+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-09T01:46:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log deleted file mode 100644 index 31b35ff19..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,25 +0,0 @@ -rocm_gpu_guard: 2026-10-08T21:33:51+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-08T22:19:00+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2709s -rocm_gpu_guard: 2026-10-08T22:19:00+0900 waiting for 60s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-08T22:19:18+0900 idle streak reset after 12s: gpu=[] compilers=[1173803:cargo] -rocm_gpu_guard: 2026-10-08T22:23:33+0900 idle streak reset after 1s: gpu=[] compilers=[1334471:cargo] -rocm_gpu_guard: 2026-10-08T23:05:01+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-08T23:05:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.252 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.80 GB, active+cache 2.18 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 432.66 ms (4733.48 tok/s) - Decode: 594.48 ms (215.31 tok/s) - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-08T23:05:05+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-08T23:05:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log deleted file mode 100644 index f449f5bf9..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,26 +0,0 @@ -rocm_gpu_guard: 2026-10-08T23:05:15+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-08T23:13:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 521s -rocm_gpu_guard: 2026-10-08T23:13:56+0900 waiting for 60s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-08T23:14:31+0900 idle streak reset after 23s: gpu=[3262038:rocminfo] compilers=[3262000:cargo] -rocm_gpu_guard: 2026-10-08T23:23:02+0900 idle streak reset after 14s: gpu=[] compilers=[3571564:cargo,3571605:rustc,3571606:rustc,3571607:rustc,3571608:rustc,3571609:rustc] -rocm_gpu_guard: 2026-10-08T23:28:13+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-08T23:28:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.250 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 474.47 ms (4316.35 tok/s) - Decode: 580.45 ms (220.52 tok/s) - Decode path: engine - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-08T23:28:17+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-08T23:28:17+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log deleted file mode 100644 index f3165af6c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:05:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:05:19+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:05:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:05:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.195 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 460.43 ms (4447.97 tok/s) - Decode: 586.82 ms (218.13 tok/s) - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T02:05:38+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-09T02:05:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log deleted file mode 100644 index 34c9ffe01..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:05:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:05:43+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:05:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:05:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.199 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 453.61 ms (4514.86 tok/s) - Decode: 574.66 ms (222.74 tok/s) - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T02:06:02+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-09T02:06:02+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log deleted file mode 100644 index c9068650d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:04:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:04:57+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:05:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:05:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.193 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.99 GB, active+cache 1.37 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 450.84 ms (4542.64 tok/s) - Decode: 590.98 ms (216.59 tok/s) - Decode path: engine - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T02:05:14+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:05:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log deleted file mode 100644 index aff771ccf..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:24:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:24:51+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:25:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:25:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.200 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.82 GB, active+cache 2.20 GB, phase peak 1.86 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 452.00 ms (4530.98 tok/s) - Decode: 588.40 ms (217.54 tok/s) - MLX peak memory: 1.86 GB -rocm_gpu_guard: 2026-10-09T02:25:08+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:25:08+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log deleted file mode 100644 index ba360683d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:25:13+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:25:13+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:25:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:25:28+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.205 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 462.41 ms (4428.99 tok/s) - Decode: 586.97 ms (218.07 tok/s) - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T02:25:32+0900 monitor: 3 samples -rocm_gpu_guard: 2026-10-09T02:25:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log deleted file mode 100644 index 4e6ad39a4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:25:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:25:37+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:25:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:25:52+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.198 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.84 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.96 GB, active+cache 1.35 GB, phase peak 1.85 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 454.49 ms (4506.18 tok/s) - Decode: 591.13 ms (216.53 tok/s) - Decode path: engine - MLX peak memory: 1.85 GB -rocm_gpu_guard: 2026-10-09T02:25:55+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:25:55+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log deleted file mode 100644 index d4f5a5e65..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-08T21:22:36+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-08T21:32:09+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 573s -rocm_gpu_guard: 2026-10-08T21:32:09+0900 waiting for 60s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-08T21:33:38+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-08T21:33:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.192 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.77 GB, active+cache 1.15 GB, phase peak 1.04 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 70.41 ms (7271.58 tok/s) - Decode: 462.36 ms (276.84 tok/s) - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-08T21:33:41+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-08T21:33:41+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log deleted file mode 100644 index 35457a283..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-08T19:17:44+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-08T19:41:27+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 1423s -rocm_gpu_guard: 2026-10-08T19:41:27+0900 waiting for 60s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-08T19:42:09+0900 idle streak reset after 28s: gpu=[] compilers=[3313432:cargo] -rocm_gpu_guard: 2026-10-08T20:39:08+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-08T20:39:08+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.741 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.36 GB, active+cache 0.74 GB, phase peak 0.63 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 73.12 ms (7002.12 tok/s) - Decode: 465.33 ms (275.07 tok/s) - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-08T20:39:15+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-08T20:39:15+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log deleted file mode 100644 index 7112e48e7..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-08T20:39:25+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-08T21:20:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2491s -rocm_gpu_guard: 2026-10-08T21:20:56+0900 waiting for 60s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-08T21:22:23+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-08T21:22:23+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.256 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.26 GB, active+cache 0.64 GB, phase peak 0.60 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 59.61 ms (8589.40 tok/s) - Decode: 468.02 ms (273.49 tok/s) - Decode path: engine - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-08T21:22:26+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-08T21:22:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log deleted file mode 100644 index a8b18a976..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:04:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:04:12+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:04:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:04:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.197 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.67 GB, active+cache 1.06 GB, phase peak 0.95 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 61.93 ms (8267.07 tok/s) - Decode: 464.92 ms (275.31 tok/s) - MLX peak memory: 1.47 GB -rocm_gpu_guard: 2026-10-09T02:04:29+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:04:29+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log deleted file mode 100644 index 85f974b44..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:04:34+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:04:34+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:04:49+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:04:49+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.192 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.61 GB, active+cache 1.00 GB, phase peak 0.87 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.47 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 68.65 ms (7458.21 tok/s) - Decode: 463.13 ms (276.38 tok/s) - MLX peak memory: 1.47 GB -rocm_gpu_guard: 2026-10-09T02:04:52+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:04:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log deleted file mode 100644 index d92cd768f..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:03:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:03:49+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:04:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:04:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.211 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.26 GB, active+cache 0.64 GB, phase peak 0.60 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 59.82 ms (8559.12 tok/s) - Decode: 467.00 ms (274.09 tok/s) - Decode path: engine - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-09T02:04:06+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:04:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log deleted file mode 100644 index 504582ada..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:23:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:23:43+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:23:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:23:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.195 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.56 GB, active+cache 0.95 GB, phase peak 0.83 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 65.70 ms (7793.54 tok/s) - Decode: 462.61 ms (276.69 tok/s) - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-09T02:24:01+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:24:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log deleted file mode 100644 index ae61fa0ba..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:24:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:24:06+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:24:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:24:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.198 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.55 GB, active+cache 0.93 GB, phase peak 0.82 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 68.96 ms (7424.31 tok/s) - Decode: 465.01 ms (275.26 tok/s) - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-09T02:24:23+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:24:23+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log deleted file mode 100644 index 3639461b2..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:24:28+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:24:28+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:24:43+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:24:43+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.194 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.30 GB, active+cache 0.69 GB, phase peak 0.83 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 68.81 ms (7440.48 tok/s) - Decode: 465.67 ms (274.88 tok/s) - Decode path: engine - MLX peak memory: 1.46 GB -rocm_gpu_guard: 2026-10-09T02:24:46+0900 monitor: 2 samples -rocm_gpu_guard: 2026-10-09T02:24:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log deleted file mode 100644 index 7c2287d83..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:53:41+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:53:41+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:53:56+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:53:56+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.212 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.91 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7150.03 ms (286.43 tok/s) - Decode: 2200.86 ms (58.16 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T01:54:21+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T01:54:21+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log deleted file mode 100644 index c90caa743..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:52:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:52:12+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:52:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:52:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.210 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.91 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7161.06 ms (285.99 tok/s) - Decode: 2205.06 ms (58.05 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T01:52:52+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T01:52:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log deleted file mode 100644 index 5130d3887..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:52:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:52:57+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:53:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:53:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.207 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7249.95 ms (282.48 tok/s) - Decode: 2196.16 ms (58.28 tok/s) - Decode path: engine - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T01:53:36+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T01:53:36+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log deleted file mode 100644 index e00939b08..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:12:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:12:50+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:13:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:13:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.217 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7272.66 ms (281.60 tok/s) - Decode: 2200.73 ms (58.16 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:13:28+0900 monitor: 16 samples -rocm_gpu_guard: 2026-10-09T02:13:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log deleted file mode 100644 index 7fc69a5b3..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:13:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:13:33+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:13:48+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:13:48+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.251 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7144.89 ms (286.64 tok/s) - Decode: 2202.88 ms (58.11 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:14:13+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T02:14:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log deleted file mode 100644 index 7cf1a0ea4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:11:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:11:54+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:12:09+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:12:09+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.221 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7053.70 ms (290.34 tok/s) - Decode: 2201.71 ms (58.14 tok/s) - Decode path: engine - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:12:44+0900 monitor: 24 samples -rocm_gpu_guard: 2026-10-09T02:12:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log deleted file mode 100644 index d8f0ae959..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:31:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:31:48+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:32:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:32:03+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.217 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7198.39 ms (284.51 tok/s) - Decode: 2208.97 ms (57.95 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:32:28+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T02:32:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log deleted file mode 100644 index 1822eec3f..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:32:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:32:33+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:32:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:32:47+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.215 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7080.17 ms (289.26 tok/s) - Decode: 2192.55 ms (58.38 tok/s) - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:33:13+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T02:33:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log deleted file mode 100644 index e7c91f7b0..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:33:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:33:18+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:33:32+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:33:32+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.215 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.63 GB, phase peak 18.80 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 7148.04 ms (286.51 tok/s) - Decode: 2204.57 ms (58.06 tok/s) - Decode path: engine - MLX peak memory: 18.91 GB -rocm_gpu_guard: 2026-10-09T02:33:57+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T02:33:57+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log deleted file mode 100644 index a7916b02e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:51:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:51:37+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:51:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:51:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.209 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1806.26 ms (283.46 tok/s) - Decode: 2042.81 ms (62.66 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:52:07+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T01:52:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log deleted file mode 100644 index 20dccea75..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:50:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:50:11+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:50:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:50:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.216 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 2292.18 ms (223.37 tok/s) - Decode: 2046.99 ms (62.53 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:50:58+0900 monitor: 21 samples -rocm_gpu_guard: 2026-10-09T01:50:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log deleted file mode 100644 index 87bc3e5b4..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:51:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:51:03+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:51:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:51:17+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.216 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1889.81 ms (270.93 tok/s) - Decode: 2047.51 ms (62.52 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T01:51:32+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T01:51:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log deleted file mode 100644 index 0f456150f..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:10:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:10:44+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:10:59+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:10:59+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.208 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1605.19 ms (318.97 tok/s) - Decode: 2048.39 ms (62.49 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:11:14+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:11:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log deleted file mode 100644 index 634199821..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:11:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:11:19+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:11:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:11:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.213 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1736.22 ms (294.89 tok/s) - Decode: 2055.99 ms (62.26 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:11:49+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:11:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log deleted file mode 100644 index 484b481ff..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:10:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:10:10+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:10:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:10:25+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.220 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.39 GB, active+cache 17.66 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1964.26 ms (260.66 tok/s) - Decode: 2050.47 ms (62.42 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:10:39+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:10:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log deleted file mode 100644 index 47da1dbfb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:30:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:30:03+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:30:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:30:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.213 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1982.61 ms (258.25 tok/s) - Decode: 2039.94 ms (62.75 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:30:34+0900 monitor: 11 samples -rocm_gpu_guard: 2026-10-09T02:30:34+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log deleted file mode 100644 index f63e12679..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:30:39+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:30:39+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:30:54+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:30:54+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.220 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1807.65 ms (283.24 tok/s) - Decode: 2039.27 ms (62.77 tok/s) - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:31:09+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:31:09+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log deleted file mode 100644 index f268c4011..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:31:14+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:31:14+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:31:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:31:28+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.223 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) -[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1903.10 ms (269.03 tok/s) - Decode: 2041.74 ms (62.69 tok/s) - Decode path: engine - MLX peak memory: 18.66 GB -rocm_gpu_guard: 2026-10-09T02:31:43+0900 monitor: 10 samples -rocm_gpu_guard: 2026-10-09T02:31:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log deleted file mode 100644 index 90c19449e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:49:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:49:44+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:49:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:49:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.630 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.04 GB, active+cache 4.02 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 702.59 ms (2914.94 tok/s) - Decode: 2046.94 ms (62.53 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T01:50:06+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:50:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log deleted file mode 100644 index 06a2380fd..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:48:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:48:50+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:49:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:49:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.633 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.27 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 720.27 ms (2843.38 tok/s) - Decode: 2037.11 ms (62.83 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T01:49:12+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:49:12+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log deleted file mode 100644 index 813cdab26..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:49:17+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:49:17+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:49:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:49:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.639 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 1.33 GB, active+cache 4.14 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 702.60 ms (2914.89 tok/s) - Decode: 2061.91 ms (62.08 tok/s) - Decode path: engine - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T01:49:39+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:49:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log deleted file mode 100644 index 310273323..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:09:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:09:16+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:09:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:09:31+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.668 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 703.54 ms (2911.00 tok/s) - Decode: 2046.37 ms (62.55 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:09:38+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:09:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log deleted file mode 100644 index a1da50093..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:09:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:09:43+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:09:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:09:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.662 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.04 GB, active+cache 4.02 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 706.78 ms (2897.64 tok/s) - Decode: 2045.58 ms (62.57 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:10:05+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:10:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log deleted file mode 100644 index d77e36575..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:08:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:08:49+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:09:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:09:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.649 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 1.42 GB, active+cache 4.22 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 709.08 ms (2888.25 tok/s) - Decode: 2074.55 ms (61.70 tok/s) - Decode path: engine - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:09:11+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:09:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log deleted file mode 100644 index 76c7e1a9e..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:28:42+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:28:42+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:28:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:28:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.648 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.27 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 703.00 ms (2913.23 tok/s) - Decode: 2043.34 ms (62.64 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:29:04+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:29:04+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log deleted file mode 100644 index 9fd239a73..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:29:09+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:29:09+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:29:24+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:29:24+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.648 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 706.11 ms (2900.41 tok/s) - Decode: 2042.94 ms (62.65 tok/s) - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:29:31+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:29:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log deleted file mode 100644 index 5bed9a721..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:29:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:29:36+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:29:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:29:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.641 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB -[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 1.25 GB, active+cache 4.05 GB, phase peak 4.52 GB -[Profile Results] - Prompt tokens: 2048 - Generated tokens: 128 - Prefill: 701.77 ms (2918.35 tok/s) - Decode: 2060.93 ms (62.11 tok/s) - Decode path: engine - MLX peak memory: 4.52 GB -rocm_gpu_guard: 2026-10-09T02:29:58+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:29:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log deleted file mode 100644 index 69eba7246..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:48:24+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:48:24+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:48:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:48:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.623 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.25 GB, active+cache 3.16 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 228.61 ms (2239.65 tok/s) - Decode: 1856.04 ms (68.96 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T01:48:45+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T01:48:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log deleted file mode 100644 index 5002b4b18..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:47:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:47:31+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:47:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:47:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.666 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.52 GB, active+cache 3.33 GB, phase peak 3.09 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.24 GB, active+cache 3.15 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 480.68 ms (1065.16 tok/s) - Decode: 1853.53 ms (69.06 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T01:47:53+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T01:47:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log deleted file mode 100644 index b5b637887..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:47:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:47:58+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:48:13+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:48:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.660 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.38 GB, active+cache 3.18 GB, phase peak 3.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 0.37 GB, active+cache 3.18 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 231.93 ms (2207.57 tok/s) - Decode: 1852.54 ms (69.09 tok/s) - Decode path: engine - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T01:48:19+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T01:48:19+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log deleted file mode 100644 index 76a3c6c70..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:07:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:07:58+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:08:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:08:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.661 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.09 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.28 GB, active+cache 3.19 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 236.92 ms (2161.07 tok/s) - Decode: 1852.86 ms (69.08 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:08:18+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:08:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log deleted file mode 100644 index 390c92703..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:08:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:08:23+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:08:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:08:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.662 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.29 GB, active+cache 3.20 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 237.18 ms (2158.68 tok/s) - Decode: 1847.37 ms (69.29 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:08:44+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:08:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log deleted file mode 100644 index 43ebfe808..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:07:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:07:32+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:07:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:07:47+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.658 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.34 GB, active+cache 3.14 GB, phase peak 3.09 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 0.39 GB, active+cache 3.20 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 234.46 ms (2183.76 tok/s) - Decode: 1857.67 ms (68.90 tok/s) - Decode path: engine - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:07:53+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:07:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log deleted file mode 100644 index 6fb346d2a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:27:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:27:25+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:27:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:27:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.671 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.25 GB, active+cache 3.16 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 229.13 ms (2234.52 tok/s) - Decode: 1834.46 ms (69.78 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:27:46+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:27:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log deleted file mode 100644 index 29eb6b311..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:27:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:27:51+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:28:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:28:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.648 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.91 GB, cache 0.29 GB, active+cache 3.20 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 237.50 ms (2155.78 tok/s) - Decode: 1849.93 ms (69.19 tok/s) - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:28:11+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:28:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log deleted file mode 100644 index 6e00f6967..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:28:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:28:16+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:28:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:28:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. -[Load] wall: 0.638 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.38 GB, active+cache 3.18 GB, phase peak 3.09 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) -[Memory] after measured pass: active 2.80 GB, cache 0.39 GB, active+cache 3.20 GB, phase peak 3.92 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 229.49 ms (2231.04 tok/s) - Decode: 1865.00 ms (68.63 tok/s) - Decode path: engine - MLX peak memory: 3.92 GB -rocm_gpu_guard: 2026-10-09T02:28:37+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:28:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log deleted file mode 100644 index 3c51ef697..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:59:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:59:12+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:59:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:59:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.489 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 968.89 ms (528.44 tok/s) - Decode: 15015.86 ms (8.52 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T01:59:48+0900 monitor: 14 samples -rocm_gpu_guard: 2026-10-09T01:59:48+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log deleted file mode 100644 index 56373792d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:57:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:57:31+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:57:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:57:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.570 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.82 GB, active+cache 13.08 GB, phase peak 12.88 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 969.43 ms (528.15 tok/s) - Decode: 14985.87 ms (8.54 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T01:58:22+0900 monitor: 24 samples -rocm_gpu_guard: 2026-10-09T01:58:22+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log deleted file mode 100644 index 1d75c1fda..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:58:27+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:58:27+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:58:42+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:58:42+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.566 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1004.76 ms (509.57 tok/s) - Decode: 15016.75 ms (8.52 tok/s) - Decode path: engine - MLX peak memory: 13.67 GB -rocm_gpu_guard: 2026-10-09T01:59:07+0900 monitor: 16 samples -rocm_gpu_guard: 2026-10-09T01:59:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log deleted file mode 100644 index aba3ec3cd..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:18:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:18:19+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:18:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:18:33+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.555 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 973.88 ms (525.73 tok/s) - Decode: 15085.56 ms (8.48 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T02:18:58+0900 monitor: 16 samples -rocm_gpu_guard: 2026-10-09T02:18:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log deleted file mode 100644 index 111adbc8a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:19:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:19:03+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:19:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:19:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.502 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 971.68 ms (526.92 tok/s) - Decode: 15123.50 ms (8.46 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T02:19:40+0900 monitor: 15 samples -rocm_gpu_guard: 2026-10-09T02:19:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log deleted file mode 100644 index c3d0c9d91..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:17:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:17:21+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:17:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:17:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.579 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 12.88 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1007.14 ms (508.37 tok/s) - Decode: 15144.73 ms (8.45 tok/s) - Decode path: engine - MLX peak memory: 13.60 GB -rocm_gpu_guard: 2026-10-09T02:18:14+0900 monitor: 25 samples -rocm_gpu_guard: 2026-10-09T02:18:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log deleted file mode 100644 index 4e6a5e281..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:37:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:37:06+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:37:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:37:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.615 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.83 GB, active+cache 13.09 GB, phase peak 12.88 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 971.76 ms (526.88 tok/s) - Decode: 15159.36 ms (8.44 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T02:37:58+0900 monitor: 25 samples -rocm_gpu_guard: 2026-10-09T02:37:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log deleted file mode 100644 index 315b97ec5..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:38:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:38:03+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:38:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:38:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.546 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1097.23 ms (466.63 tok/s) - Decode: 15250.46 ms (8.39 tok/s) - MLX peak memory: 13.84 GB -rocm_gpu_guard: 2026-10-09T02:38:43+0900 monitor: 17 samples -rocm_gpu_guard: 2026-10-09T02:38:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log deleted file mode 100644 index ad6ea9f63..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log +++ /dev/null @@ -1,24 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:38:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:38:48+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:39:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:39:03+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) -[Load] wall: 0.499 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.67 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1002.98 ms (510.48 tok/s) - Decode: 15075.33 ms (8.49 tok/s) - Decode path: engine - MLX peak memory: 13.67 GB -rocm_gpu_guard: 2026-10-09T02:39:25+0900 monitor: 15 samples -rocm_gpu_guard: 2026-10-09T02:39:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log deleted file mode 100644 index c46dd80b3..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:55:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:55:19+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:55:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:55:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.118 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 865.72 ms (591.41 tok/s) - Decode: 1435.18 ms (89.19 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T01:55:40+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T01:55:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log deleted file mode 100644 index 726c5040c..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:54:26+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:54:26+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:54:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:54:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.137 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1141.73 ms (448.44 tok/s) - Decode: 1426.12 ms (89.75 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T01:54:49+0900 monitor: 6 samples -rocm_gpu_guard: 2026-10-09T01:54:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log deleted file mode 100644 index 059b91b1d..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T01:54:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T01:54:54+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T01:55:08+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T01:55:08+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.122 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 869.89 ms (588.58 tok/s) - Decode: 1444.81 ms (88.59 tok/s) - Decode path: engine - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T01:55:14+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T01:55:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log deleted file mode 100644 index 4b892dd4a..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:14:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:14:45+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:15:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:15:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.126 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 894.81 ms (572.19 tok/s) - Decode: 1429.58 ms (89.54 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:15:06+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:15:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log deleted file mode 100644 index 3fcd771ef..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:15:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:15:11+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:15:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:15:25+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.121 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 894.08 ms (572.66 tok/s) - Decode: 1429.87 ms (89.52 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:15:31+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:15:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log deleted file mode 100644 index 7f9a607db..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:14:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:14:18+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:14:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:14:33+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.131 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 3.96 GB, cache 1.84 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1128.64 ms (453.64 tok/s) - Decode: 1438.29 ms (88.99 tok/s) - Decode path: engine - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:14:40+0900 monitor: 5 samples -rocm_gpu_guard: 2026-10-09T02:14:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log deleted file mode 100644 index d82645bb6..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:34:02+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:34:02+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:34:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:34:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.131 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 1331.37 ms (384.57 tok/s) - Decode: 1427.41 ms (89.67 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:34:25+0900 monitor: 6 samples -rocm_gpu_guard: 2026-10-09T02:34:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log deleted file mode 100644 index c855ae2bb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log +++ /dev/null @@ -1,22 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:34:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:34:30+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:34:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:34:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.123 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.20 GB, active+cache 7.16 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 868.14 ms (589.77 tok/s) - Decode: 1431.48 ms (89.42 tok/s) - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:34:51+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:34:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log deleted file mode 100644 index ab486a6fb..000000000 --- a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log +++ /dev/null @@ -1,23 +0,0 @@ -rocm_gpu_guard: 2026-10-09T02:34:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock -rocm_gpu_guard: 2026-10-09T02:34:56+0900 waiting for 10s of idle GPU and no compiler -rocm_gpu_guard: 2026-10-09T02:35:10+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample -rocm_gpu_guard: 2026-10-09T02:35:10+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 -[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB -[Memory] cache limit: 2.15 GB -[Load] wall: 0.121 s MLX peak: 0.00 GB -[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 -[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 -[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.22 GB -[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) -[Memory] after measured pass: active 3.96 GB, cache 1.84 GB, active+cache 5.80 GB, phase peak 7.31 GB -[Profile Results] - Prompt tokens: 512 - Generated tokens: 128 - Prefill: 855.19 ms (598.70 tok/s) - Decode: 1428.92 ms (89.58 tok/s) - Decode path: engine - MLX peak memory: 7.31 GB -rocm_gpu_guard: 2026-10-09T02:35:16+0900 monitor: 4 samples -rocm_gpu_guard: 2026-10-09T02:35:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md b/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md index 6ea0d56d1..f54f96dfa 100644 --- a/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md +++ b/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md @@ -126,7 +126,7 @@ ADR 0007's ROCm paragraph: the CLI and the server engine with dense storage must ### Method -`data/.../harness/bench_rounds.sh`: for each (model, prompt length) cell, three interleaved rounds of three arms, each a fresh `mlxcel-bench-decode` process at `scripts/bench_decode.sh`'s shape (`-n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens N`, greedy): `base` is the pre-epic binary (`4c44e317`, `CxxGenerator`), `new` is `main` (`DirectEngine`, `Decode path: engine`), and `base-null` is the pre-epic binary again, the method's noise floor. The arm order rotates each round. Every run was its own `scripts/rocm_gpu_guard.sh` command inside one host-wide lock hold (`data/.../harness/full_session.sh`): 10 s of idle GPU and no compiler before each run, a 1 Hz monitor during it, and all 99 runs were CLEAN on the first attempt (`data/.../guard/logs/`). The session ran 2026-10-09 01:19 to 02:43 KST. `data/.../harness/summarize_rounds.py` computes the medians and the paired per-round deltas below (median and range over the three rounds, percent of `base`). The threshold is ADR 0007's 1.0 percent on the paired decode delta. +`data/.../harness/bench_rounds.sh`: for each (model, prompt length) cell, three interleaved rounds of three arms, each a fresh `mlxcel-bench-decode` process at `scripts/bench_decode.sh`'s shape (`-n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens N`, greedy): `base` is the pre-epic binary (`4c44e317`, `CxxGenerator`), `new` is `main` (`DirectEngine`, `Decode path: engine`), and `base-null` is the pre-epic binary again, the method's noise floor. The arm order rotates each round. Every run was its own `scripts/rocm_gpu_guard.sh` command inside one host-wide lock hold (`data/.../harness/full_session.sh`): 10 s of idle GPU and no compiler before each run, a 1 Hz monitor during it, and all 99 runs were CLEAN on the first attempt (`data/.../guard/guard-logs.tar.gz`). The session ran 2026-10-09 01:19 to 02:43 KST. `data/.../harness/summarize_rounds.py` computes the medians and the paired per-round deltas below (median and range over the three rounds, percent of `base`). The threshold is ADR 0007's 1.0 percent on the paired decode delta. ### Decode, `mlxcel generate` (DirectEngine) vs `CxxGenerator` diff --git a/scripts/ci/check_binary_assets.py b/scripts/ci/check_binary_assets.py index e1b35f432..5d067f50f 100755 --- a/scripts/ci/check_binary_assets.py +++ b/scripts/ci/check_binary_assets.py @@ -98,6 +98,17 @@ class Rule: "and the MP3 and FLAC framing cannot be synthesized in the test." ), ), + Rule( + pattern="docs/benchmark_results/data/*.tar.gz", + max_bytes=32 * 1024, + why=( + "Raw per-run logs behind a committed benchmark report, archived " + "because nobody reads them in normal review; the report's tables " + "and CSVs carry the numbers, and the archive is there to audit a " + "run. Keep each under 32 KB compressed; larger log sets belong " + "outside the repository." + ), + ), )