From e491157030013ad370cf22164917ddff0e8f8c4f Mon Sep 17 00:00:00 2001 From: Jeongkyu Shin Date: Sun, 11 Oct 2026 11:06:42 +0900 Subject: [PATCH] fix(ci): store the ROCm verification logs as text instead of tarballs PR #2250 committed docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz and rounds/raw-logs.tar.gz without declaring them in scripts/ci/check_binary_assets.py, so the binary assets job has failed on main and on every PR since. Both archives hold small plain-text run logs (212 files), so they are unpacked in place (guard/logs/ and rounds/raw/, the path the report already cites for the raw logs) instead of being declared as a binary exception: text diffs and delta-compresses, and the guard's binary budget stays for assets that need it. The report's guard-log reference now points at guard/logs/. --- .../guard/guard-logs.tar.gz | Bin 15502 -> 0 bytes ...eta-Llama-3.1-8B-Instruct-4bit_r1_base.log | 28 +++++++++ ...Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log | 25 ++++++++ ...eta-Llama-3.1-8B-Instruct-4bit_r2_base.log | 26 +++++++++ ...Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log | 25 ++++++++ ...eta-Llama-3.1-8B-Instruct-4bit_r3_base.log | 26 +++++++++ ...Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log | 25 ++++++++ .../guard/engine_Qwen3-0.6B-4bit_r1_base.log | 12 ++++ .../guard/engine_Qwen3-0.6B-4bit_r1_new.log | 11 ++++ .../guard/engine_Qwen3-0.6B-4bit_r2_base.log | 11 ++++ .../guard/engine_Qwen3-0.6B-4bit_r2_new.log | 11 ++++ .../guard/engine_Qwen3-0.6B-4bit_r3_base.log | 11 ++++ .../guard/engine_Qwen3-0.6B-4bit_r3_new.log | 11 ++++ .../guard/logs/guard/gemma3_probe.log | 9 +++ .../guard/logs/guard/paged_pool_past_u32.log | 8 +++ ....1-8B-Instruct-4bit_pp512_r1_base-null.log | 11 ++++ ...ama-3.1-8B-Instruct-4bit_pp512_r1_base.log | 13 +++++ ...lama-3.1-8B-Instruct-4bit_pp512_r1_new.log | 11 ++++ ....1-8B-Instruct-4bit_pp512_r2_base-null.log | 11 ++++ ...ama-3.1-8B-Instruct-4bit_pp512_r2_base.log | 11 ++++ ...lama-3.1-8B-Instruct-4bit_pp512_r2_new.log | 13 +++++ ....1-8B-Instruct-4bit_pp512_r3_base-null.log | 13 +++++ ...ama-3.1-8B-Instruct-4bit_pp512_r3_base.log | 11 ++++ ...lama-3.1-8B-Instruct-4bit_pp512_r3_new.log | 11 ++++ ...-Instruct-v0.1-4bit_pp512_r1_base-null.log | 44 ++++++++++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log | 45 +++++++++++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log | 45 +++++++++++++++ ...-Instruct-v0.1-4bit_pp512_r2_base-null.log | 46 +++++++++++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log | 44 ++++++++++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log | 45 +++++++++++++++ ...-Instruct-v0.1-4bit_pp512_r3_base-null.log | 45 +++++++++++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log | 54 ++++++++++++++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log | 44 ++++++++++++++ ...3-Nano-30B-A3B-4bit_pp512_r1_base-null.log | 16 ++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r1_base.log | 17 ++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r1_new.log | 16 ++++++ ...3-Nano-30B-A3B-4bit_pp512_r2_base-null.log | 16 ++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r2_base.log | 16 ++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r2_new.log | 17 ++++++ ...3-Nano-30B-A3B-4bit_pp512_r3_base-null.log | 17 ++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r3_base.log | 16 ++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r3_new.log | 16 ++++++ .../Qwen3-0.6B-4bit_pp2048_r1_base-null.log | 11 ++++ .../guard/Qwen3-0.6B-4bit_pp2048_r1_base.log | 12 ++++ .../guard/Qwen3-0.6B-4bit_pp2048_r1_new.log | 12 ++++ .../Qwen3-0.6B-4bit_pp2048_r2_base-null.log | 9 +++ .../guard/Qwen3-0.6B-4bit_pp2048_r2_base.log | 9 +++ .../guard/Qwen3-0.6B-4bit_pp2048_r2_new.log | 8 +++ .../Qwen3-0.6B-4bit_pp2048_r3_base-null.log | 8 +++ .../guard/Qwen3-0.6B-4bit_pp2048_r3_base.log | 9 +++ .../guard/Qwen3-0.6B-4bit_pp2048_r3_new.log | 8 +++ .../Qwen3-0.6B-4bit_pp512_r1_base-null.log | 9 +++ .../guard/Qwen3-0.6B-4bit_pp512_r1_base.log | 13 +++++ .../guard/Qwen3-0.6B-4bit_pp512_r1_new.log | 9 +++ .../Qwen3-0.6B-4bit_pp512_r2_base-null.log | 8 +++ .../guard/Qwen3-0.6B-4bit_pp512_r2_base.log | 8 +++ .../guard/Qwen3-0.6B-4bit_pp512_r2_new.log | 8 +++ .../Qwen3-0.6B-4bit_pp512_r3_base-null.log | 8 +++ .../guard/Qwen3-0.6B-4bit_pp512_r3_base.log | 8 +++ .../guard/Qwen3-0.6B-4bit_pp512_r3_new.log | 8 +++ ...Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp2048_r1_base.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp2048_r1_new.log | 23 ++++++++ ...Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log | 22 +++++++ .../Qwen3-30B-A3B-4bit_pp2048_r2_base.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp2048_r2_new.log | 30 ++++++++++ ...Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp2048_r3_base.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp2048_r3_new.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp512_r1_base-null.log | 16 ++++++ .../Qwen3-30B-A3B-4bit_pp512_r1_base.log | 27 +++++++++ .../guard/Qwen3-30B-A3B-4bit_pp512_r1_new.log | 16 ++++++ .../Qwen3-30B-A3B-4bit_pp512_r2_base-null.log | 16 ++++++ .../Qwen3-30B-A3B-4bit_pp512_r2_base.log | 16 ++++++ .../guard/Qwen3-30B-A3B-4bit_pp512_r2_new.log | 16 ++++++ .../Qwen3-30B-A3B-4bit_pp512_r3_base-null.log | 17 ++++++ .../Qwen3-30B-A3B-4bit_pp512_r3_base.log | 16 ++++++ .../guard/Qwen3-30B-A3B-4bit_pp512_r3_new.log | 16 ++++++ ...gemma-3-4b-it-4bit_pp2048_r1_base-null.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r1_base.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r1_new.log | 11 ++++ ...gemma-3-4b-it-4bit_pp2048_r2_base-null.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r2_base.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r2_new.log | 11 ++++ ...gemma-3-4b-it-4bit_pp2048_r3_base-null.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r3_base.log | 11 ++++ .../gemma-3-4b-it-4bit_pp2048_r3_new.log | 11 ++++ .../gemma-3-4b-it-4bit_pp512_r1_base-null.log | 10 ++++ .../gemma-3-4b-it-4bit_pp512_r1_base.log | 11 ++++ .../guard/gemma-3-4b-it-4bit_pp512_r1_new.log | 10 ++++ .../gemma-3-4b-it-4bit_pp512_r2_base-null.log | 10 ++++ .../gemma-3-4b-it-4bit_pp512_r2_base.log | 10 ++++ .../guard/gemma-3-4b-it-4bit_pp512_r2_new.log | 10 ++++ .../gemma-3-4b-it-4bit_pp512_r3_base-null.log | 10 ++++ .../gemma-3-4b-it-4bit_pp512_r3_base.log | 10 ++++ .../guard/gemma-3-4b-it-4bit_pp512_r3_new.log | 10 ++++ ...pt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log | 20 +++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log | 30 ++++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log | 22 +++++++ ...pt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log | 22 +++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log | 21 +++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log | 31 ++++++++++ ...pt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log | 31 ++++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log | 23 ++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log | 21 +++++++ ...ite-4.0-h-tiny-4bit_pp512_r1_base-null.log | 10 ++++ .../granite-4.0-h-tiny-4bit_pp512_r1_base.log | 12 ++++ .../granite-4.0-h-tiny-4bit_pp512_r1_new.log | 10 ++++ ...ite-4.0-h-tiny-4bit_pp512_r2_base-null.log | 10 ++++ .../granite-4.0-h-tiny-4bit_pp512_r2_base.log | 10 ++++ .../granite-4.0-h-tiny-4bit_pp512_r2_new.log | 11 ++++ ...ite-4.0-h-tiny-4bit_pp512_r3_base-null.log | 12 ++++ .../granite-4.0-h-tiny-4bit_pp512_r3_base.log | 10 ++++ .../granite-4.0-h-tiny-4bit_pp512_r3_new.log | 10 ++++ .../rounds/raw-logs.tar.gz | Bin 14119 -> 0 bytes ....1-8B-Instruct-4bit_pp512_r1_base-null.log | 22 +++++++ ...ama-3.1-8B-Instruct-4bit_pp512_r1_base.log | 22 +++++++ ...lama-3.1-8B-Instruct-4bit_pp512_r1_new.log | 23 ++++++++ ....1-8B-Instruct-4bit_pp512_r2_base-null.log | 22 +++++++ ...ama-3.1-8B-Instruct-4bit_pp512_r2_base.log | 22 +++++++ ...lama-3.1-8B-Instruct-4bit_pp512_r2_new.log | 23 ++++++++ ....1-8B-Instruct-4bit_pp512_r3_base-null.log | 22 +++++++ ...ama-3.1-8B-Instruct-4bit_pp512_r3_base.log | 22 +++++++ ...lama-3.1-8B-Instruct-4bit_pp512_r3_new.log | 23 ++++++++ ...-Instruct-v0.1-4bit_pp512_r1_base-null.log | 22 +++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log | 22 +++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log | 23 ++++++++ ...-Instruct-v0.1-4bit_pp512_r2_base-null.log | 22 +++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log | 22 +++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log | 23 ++++++++ ...-Instruct-v0.1-4bit_pp512_r3_base-null.log | 22 +++++++ ...-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log | 22 +++++++ ...l-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log | 23 ++++++++ ...3-Nano-30B-A3B-4bit_pp512_r1_base-null.log | 27 +++++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r1_base.log | 27 +++++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r1_new.log | 23 ++++++++ ...3-Nano-30B-A3B-4bit_pp512_r2_base-null.log | 27 +++++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r2_base.log | 27 +++++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r2_new.log | 23 ++++++++ ...3-Nano-30B-A3B-4bit_pp512_r3_base-null.log | 27 +++++++++ ...tron-3-Nano-30B-A3B-4bit_pp512_r3_base.log | 27 +++++++++ ...otron-3-Nano-30B-A3B-4bit_pp512_r3_new.log | 23 ++++++++ .../Qwen3-0.6B-4bit_pp2048_r1_base-null.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r1_base.log | 25 ++++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r1_new.log | 26 +++++++++ .../Qwen3-0.6B-4bit_pp2048_r2_base-null.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r2_base.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r2_new.log | 23 ++++++++ .../Qwen3-0.6B-4bit_pp2048_r3_base-null.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r3_base.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp2048_r3_new.log | 23 ++++++++ .../Qwen3-0.6B-4bit_pp512_r1_base-null.log | 23 ++++++++ .../raw/Qwen3-0.6B-4bit_pp512_r1_base.log | 24 ++++++++ .../raw/Qwen3-0.6B-4bit_pp512_r1_new.log | 24 ++++++++ .../Qwen3-0.6B-4bit_pp512_r2_base-null.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp512_r2_base.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp512_r2_new.log | 23 ++++++++ .../Qwen3-0.6B-4bit_pp512_r3_base-null.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp512_r3_base.log | 22 +++++++ .../raw/Qwen3-0.6B-4bit_pp512_r3_new.log | 23 ++++++++ ...Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log | 23 ++++++++ ...Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log | 23 ++++++++ ...Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp512_r1_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp512_r2_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log | 23 ++++++++ .../Qwen3-30B-A3B-4bit_pp512_r3_base-null.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log | 22 +++++++ .../raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log | 23 ++++++++ ...gemma-3-4b-it-4bit_pp2048_r1_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r1_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r1_new.log | 24 ++++++++ ...gemma-3-4b-it-4bit_pp2048_r2_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r2_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r2_new.log | 24 ++++++++ ...gemma-3-4b-it-4bit_pp2048_r3_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r3_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp2048_r3_new.log | 24 ++++++++ .../gemma-3-4b-it-4bit_pp512_r1_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r1_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r1_new.log | 24 ++++++++ .../gemma-3-4b-it-4bit_pp512_r2_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r2_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r2_new.log | 24 ++++++++ .../gemma-3-4b-it-4bit_pp512_r3_base-null.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r3_base.log | 23 ++++++++ .../raw/gemma-3-4b-it-4bit_pp512_r3_new.log | 24 ++++++++ ...pt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log | 23 ++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log | 23 ++++++++ .../raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log | 24 ++++++++ ...pt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log | 23 ++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log | 23 ++++++++ .../raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log | 24 ++++++++ ...pt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log | 23 ++++++++ .../gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log | 23 ++++++++ .../raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log | 24 ++++++++ ...ite-4.0-h-tiny-4bit_pp512_r1_base-null.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r1_base.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r1_new.log | 23 ++++++++ ...ite-4.0-h-tiny-4bit_pp512_r2_base-null.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r2_base.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r2_new.log | 23 ++++++++ ...ite-4.0-h-tiny-4bit_pp512_r3_base-null.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r3_base.log | 22 +++++++ .../granite-4.0-h-tiny-4bit_pp512_r3_new.log | 23 ++++++++ .../rocm-unified-engine-gfx1151-2026-10-08.md | 2 +- 215 files changed, 4222 insertions(+), 1 deletion(-) delete mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Meta-Llama-3.1-8B-Instruct-4bit_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/engine_Qwen3-0.6B-4bit_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/gemma3_probe.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/guard/paged_pool_past_u32.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-0.6B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/Qwen3-30B-A3B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gemma-3-4b-it-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/logs/rounds/guard/granite-4.0-h-tiny-4bit_pp512_r3_new.log delete mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw-logs.tar.gz create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log create mode 100644 docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz deleted file mode 100644 index 6f1c976b366892a654bc298d64b2226c2003d9cb..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 15502 zcma*Oc|4TS`!`OuvX&)V(rQVTP^2=HsBB3RC4>+{MwI0yTiLfF+bAK)QiLdD-`5Zt zjD5*&7-qZAbC1vG`~5wC{Qh`)yG|^s43@XeC3`5G?0Wgv_l<%=V9|fsS>G8vM+o=S|ctgLZT5=zaW%>xvg3KczVmX5oK}srl`e^28(@E;S;5IkyO37B*FM zFAWm)Ym={%-TsP{e{zK*Kj~o!miz*UY7P7T7+ZqI@NijNDjGDy$GYWryxm-tf82ff3((xaB0nL9VvuS<^84 z5vY1}Reg*W<_wRMXCH2FRl6UZ!b^49DbLCD+YR0LOqSg#?5gdrA?Kf26z@8?h|!ED zLJ)1CeG8`XP$;9s?xd>x(L30;xPPNW%3>7-=PI~;k1)|rquisxFF?C(cWg1xW*1n( zKJJpv)|vqMd%bZD_X-F&u$CxHwETHC7RcKHrp-wrC_6J=RYh9}+p>oI)}Q*#|O z^(A?gsLtWBf9pK2XREXb(5Kbkb?phO|J0K`6ud{Sg!VEC)rXv65^4_--e71BS!WXJ z4#{Iv5a7_R7O3a>Ia~MXea1eyt~I`-_z0UY)r(i3xG`)`Iev@$&3)12iCZY9a(dxg zup?Ki@uT5vF$2|$E{L+|)AZ*OX3PasOg7W>y+2b+_v)^FDqfvCVN3#Hp+EJw7;aH5 zb^Rv6-&7_r*a@)%OU=;LsyuuOm5SZwR|l>d&Jr*_NWB}F+BRFdZsTZ!@`5M*$D4Ty zI>VV>5seFTS@!SYIU-VJ#~C=fM}428pfD@Xkrsu6T!`>y&K*#jx$SO+!?TM#xQCRC(z_gf3n)yo{V$Lxd7_M8Fhd!s5!< zaQ6r=canzfb>TIDg}-6{+*|+qEA`E4So}D+!H!#p=KAwzUB5pK0Yyg-NUAkbo|BLT zP_l!4JG=w-kr@eK+|F0im*!lJ-F_cC|H9{i4cfw_n$(_{^w^qHdvHO4J8Kaw6w~Xe z#)RyHCMBQlZCie8Ey6BYx zuqY-sLNjZ-4P&a{?lv15Ms6EI78u^oG(SE8gynXOgK8Q3yJ>)JUNQ~Htk)}u3$g4! z{JTSoVd5wz_^0j*7u(<9MT_Lo-bF&Dmotf?C9kG*8<*aYvZ)6zRM zlsult?gkChOTG&vIe_Jr8mC#c)QtbWY@ri=_hk`-I&9Dmm*Gz=RK`kK2$~OV>X(Q4 zor|50wdTAr*s8#ty=W_B_^X=7K4Q+#KYjx^4Fch#+*{B|P#&$7v5LN)G6*mkitWJZ zO($?tb>CQ+0T1LLq1Npm^OO9N`?+UJ83JQXW(o|+=4>A3DI8O0_GpIXAtLnrOzCgt zV`;L?lfA8_6x6$4u%W2W7a+4mk6orNUgzWb=%!r*WxB^suK*@=9`Ip?sj)nQS00ETfE$}zFIt#Xg^C-S~U%pTZ457+vOdi&`#GY$q^ba~c$@p6V*l|v+_Ax89& zej(rp!T&6>!pDXqG99 zp-#*5tT%)8k*biNx*VCx_c23s&sLi9!2tyq2WyCg4Q%~G5eh# zGKK0vvsW7j=8U%=e$qK~Pq+U9wUp^N6~-j3hAC#lhGMJ0a~PCGYoRp?V*!uhyGCGA zNX5RkTtR%Ihvd(c+Ma+Gi_T+tpsAKG#cVb~@;?>XM4%xrjyDVqTr|}wTGhYLNjP!I zBF=;E7yZCdn2L2?1=%We-v~&u@VfNqJPq=0;{-)j(V2}jY;mV#{jl`j_(34br9{bjS|g=b)$|3#tC4ey=c3TwhGFmUe`B7@+6!eYcku7>jit5V6sC~!tg45FtEPeV6Jv|l#=b%%EpJUDfj}ta) z@Igpe1y_`6Y4ngd)3>GHe%N*8*u%k}O;dQKC^2c(fjz~?tqZ)Y_SQ7h&I_T_6~$Ob zL$$*+2c`S0dt4mQLTFKuT~+AUc{MvV&T6ix=%6CwoT+hK&%EjmqT>FqN@8;U(@0`& zhuSHX8OM_7HvQ-SzGR*Kd(rRD+MUB=p}*ZVN?ZHvxm;%^T%^R*&>lD(} z+KY5q6d+x0-N+5Bt7J?uXAc>@&;L4dQ}zDSD}BL&z_3c-Mgti08Eyxk&cQ0$Njnf1 zqkR1QHUF+1)I~dOsytyCiZ^ryeKsxPEw2oxNKEgB3j2jvmz{s>U%Qt2-c+(a6ZfV? zfmgOWvp#b<8JShF1a2kj$IL585B>B0!OQ+P8Ut~4M^PuI0(@Cb<0i`73MJ|lr3+8h zyG-1-VEuWaO+40p?U|k;_x<`>6Z@!g8^ju0ipcA}iupn2obAveF)4EIwh0}`mwG)b z1!*)fk~onyu+Cxx2ShSarA#ps-pq)?*&`8^IT?UP{yQ?oiSOGT>V6I}07cLgqgj2P z{xF8X$%T^ew)@`XFINB2)XX8K7T{W$UMP9#JGi!Krdj8N^d4(!@3Uu-E}B5p7FMCT z-R$uX>Idv&UC$x!Q99_hWmflQ`H`~tLGIiQYbv#yNXE^al-`&485CQCn5UJ?Kt(%| zWSQS`(_KU~kFW}sE`?x#qsG%t@J};|%5w(v!5ukYrkuNNmd}45cMgc#5ZHj3wxCV zatAKgEb!T1IUnfKkeL;rrOL;G3)9qQ)C{%xz=`7O))Icv9R6C78u?g){go68A8P{~ z&&H&yp8StPMm?F#!J(ea&iU`Vb&o-4pzXEjsNe-L{|C17EN#qi^I?{RNvH`;!QZ~O zgIhyo-=M9Y;yt4jhXv)^NaQH`_st-JfIU^M?)PFF<+Rp(N<=@L(0uB~I+x!Yg*zYp zOjwkBQ(nLP&e>;F9Br+}{zpMir-^W#sPY%>SOGzd1KH_W=Ig*scx}tgg>J?vj6n>!`4u0 z;W|6wV%gd|;>f$;l;)+_Cm_VDyjaB=A5vqugPwF{*54Ka$LodAuB>9(|Iu}3)(b^~ zUhAH8;oWz8Gn6T+)S{62^X8fFLQCY^da|r6q2YOrZlb)#5t^^xBYu;-{5jd&**fl* zNaxG2dPCJGLTf-Az%+3zctV+i|INb0*%$fcsFgz3GV`A}|6Kf6dySk~NYq*lF8X`^ z{j$BuKVW(wv#-ztK2M_WE%2(E=~q?2)5GHp484VuatLxyQ&rQw+yu|9owzf2Y-+C2|In zk<=yW3kFn30|iq=zcUw#=%Uf%&tTlwSQjpHH~cc!Ni z6dlpf*H7~3SOE3Lk4T`;_Z27om>)}n<}90L{LZDt#BB^8X<}2qHy>KZ=bWph>)j%C zkT7h^&$d({tTM%`Sg&Kxf+qE5@pl(0H_ntaQm$Jb1;MwFCC1PCneWvRxo+u#1J z<2clx{Z8ucFK#iG{K2DHLSG()MZX+aL_+opF*Wu(MOSn*7d??UOF1mYiuO1&Ao$?2 z;+CLZM3Np$*{;&oJ^7S#FY%I^-tEk@YSl*%$tgdp>>-K!Pz9}F(`N8-&;{1UNBv^J+T1I?2e@X7n?#7s-Cn<^OH?c;gO}FbnSRb)CGM zutsQ}vJq zJbaVVAm$11(3Zhuz%pygNTA;h7j4kIlml{Ddie(I#ti|MQCLT(mcV8!R7wrwwmA|d zB(}}Zje06_W(ka0S}d1Uo;iq<-_GBH&UTG4gQCAQ*IVAk2qd7`f6v4#uOkvhpp$Ak zbdp1Jy)++|2KawP>JGYYRvX4Mxpvzu;?h2!N;rhYeXuIQ%u5ZC&#AGORYyhdSxs#& zW8LafXYbZ12g794$JJqIIfjC7r{;Q|_-zn}0yv+x)085`8OW!{P!+z{2-Qr!PXsp| z_S_Y*aeDdP#9>cz&p5xJ+=_UajoADyIV^e)TU2D!ljU?j{@1A?!Onm%0M6b=4MBE1B+M+f5iUBa z(OMr(N+TlS<}FR|-7K`7$pLg|Y-zT!8{&IJX`@u3+b>jaN4zZXIhJ+Jc~eC!M2%eK zMLa`_xlTkG(gu6r{4!W~pXClX#cMWeEN=gR0=;PCq^#mo^pI9hBYd2Qq9!{K@%8>k ztbmJ!&%4qehq){I_4u7V_lt#8s(tbz_WlLp#OWA4p_Fbf)WQ7xdf@K`I0&?$U0}rv+rwQy7{0qZ(*1)2S z?pBf=z?3L1m$DU2;R#y}EHM{(S60StkT`IWAI)i?;C+~QN(@M^UI6lX zK0vgm(AW&>28$1x8Y8@1N(0!vh7Oi3Glt2QN+wqR zeKK%$?pA-Y?mqhn-PV&@6jT;D;wF%(qhVv0sgSi^D=0YDD&Z$E+Wq<_$|$0yL~3UW<=`A_oTPG#9bG#9Qj_+tal_dh<65tPQE@}V*VbMK{iC1iiOzxBoW_mWg7Dgp6l?73bZTTM{u#K zJ*|K*{3fCQQ~$ogg~!#MS*Mfz9$hX0CBIS3Funv#76Y9mIL7nDi@64N8lJw3W7`IR z^<-s7QYYvp^PJ5i)z|oW)w37@GS_E~*#$^bnZt zU+bEW7MLvs$S$~qT}ioiupFpm^qMV!`R-Sb0ZR(3V$i=3{*jgC-2LN3%ZqiPi#ihZ zWV}3f48I~*K2TI`D!WtSEz1UL$l}06q(UvcSO%&_7jMzcH75N11}HcG0PcC}HXwIq zB=5t4hW4MOsGEGLwF39g?4xSp4!$L_vAeS7W^wAKa`M@EAAR^vs@{pWSxcDM=zW-d zJ|Zt!s=lS@(T4!vY^%BeMbFk?#w1I?Ui+;iS=uw~bTCMmGsPrpo8aGP!igoQ4PgJWjgbHuQl(rB$nT5>ji6 z6)=-vKM4AaLep9ZH;qu2W65|j8DtnQ8=4xX8y(Mu|Y}c-P-=NM~+`#=AG~S{gB1jBPF|$J!e)~H$p(K z({cBLK0VpczQc&cw3{7r4l*@8PBuTp<*O{&A|?9%u>a)LP49K^fdHk)RYvSa!-FSb z%d4T};aHIbfb58g!t2=i`;(LFP=J363A@;|daUwF!<5$nqu1}(`91ZMpGl3w7dy3T zLw_KtBE<)>j<%`<`!mQ;oAJblhaarAx6%Cjy*VT}UqFUojDae~ekYMuXyErvJr>qh z1tDyLwZ?eoZR=R>p!EyFtjuG2C-2PDE{V}wvY{mk$>yAL`>d%3p_;3!h*8%bJ^rdq zZy{rlJQKOQQ0y|I+wI-G?iV(o|BCoXj=-mN`{U3k5O23@mT6auEOEGF(A3(;5NvHr_4m72Fl0jCm{5lZf zy*hgg0;a3d)EzA2d83`|DR6j`uNt7X73I^E9iW@WDK&oy0qQ(m`E4eOW?&%g<`Ac< zT3gQ*QHi!5&3lMIT|mSoABna#r2=oMBXxQ2q%x0LC|GM~$-GlI5>k8H^6rJ2e!94A zMQ>R#xoS=ijm3{mTy8piS7HP8XMX@c=u_)}oRzajfsDlhu)JE6YUatUCJ8tU-_?T8 zU7zH;z#jAjBn9%1#k@jN{u>9l{i2Ku4W5j;7+q`vEm8ta;FHu5l*2nIUiz^$Y8|B* zUW;Vbjn>qd$VnquCo~TcA&fImV?mrXe;a5~V4Z<%g(5akikv$ikyF04Qw-Wh`CE8% zDZj{hBZ)N>asLoAP|^+~87@z{zW=yZwuU{+{PR||Zo&VY@>lET8%dS{{xxLG*`3sw zTL)AjLT~oJo|eVgiW?%&d5pPJ!uF^pSbwn6y|*C6I^}B1AF-~Y7~Z@iDx6)nBPzh6 zC{Y_6Cj}j^cSyv>S#!4GTNJowTy2G7HdGY#qBc}S^aM-}d_z}+YriT^+?AyjFK0r+ zvl~qih#!&4mxKR*rKzn@6I1k~_w^HkhtxTDKnw)n7(7%?bB8=VARmuk12}Q7L6v*5 zV=maf)gdJYV+z*A6~^?;DuC`p8Xf^OB=74S-i6rBs6e^k6cdP%2vh$>m7lPCpq>`+ zC@mEKcg9uS?7bI@zzVl$y?NsJtkzxCo<%CCE%CpbXdbXcu0fC;W0O^n{itmuZ%`ht z^;Wm=Cg7u2b}%n;6n6mMCSNqrc`!S3o++(6Zj$qe4ufh?!mVui7cRrO4kt%jHTf})`h^#M4Ux8#(0BC^x7-VxD#yGQiyvI+**j0S|(;}~5BVrmm59{ZwqyFsVZc-%|F2MHGGjLKiY%jnlAN21c z+5i^JP>cZSDHoL1P>dItIKYK=kg21$jv6P2a+ls3%g$HvEX|EzyLEgihJ8<3#!R48 zFeOft^D?dF$U$p=+x;>7QHs|>*40n3k9Wb`Y8?PfL720&PHQChT zjOGa$brVS~{TqAJ!PjQg$vKoXehLbst=8o|1=#QaQ@v-T$p4~`#_1fEbo7z!Col`d z=1J`|lTyPUg>rV4_u=$@Ps?2MR_0-DAB)p04q3L{f7(ljwf4_ zR_3QizASt!nU$IPw^eHX!!t$p%G~tuJ)aMrG;}>%nT_)7^dV_x+RMldeypF zH}+XX$vNbHsF|_3IC^&e1^t@z^hjPV2NBt~>|q@E?vsu%%8O!L3HpX^g8YFA7(W~j zEUq4VRNX{{el$>_;qy_ABq%_RWcd-TXOITVXLV)-5zPA)M_rDtPXOdqX}T<$Iv_!_ zcvD`OS(IwXTKgMoyWe>UCCsc3;*G{%w5&_>OssKai6l@_t7NZuU|P5g;tT}nCfe3O zCtk>uWrT?dGj}N?i0|$777er^i zF{6$Yv)1fW7;2b99?CJ!EX0w9a{%6|7jWhDyqye%5-s1*R-JBaYPVAf5JC;G1D-h~ z?4|jKz@1W*+rjaq+lKHms`SmRRwGea&T|UhdpOUXp$kPsECmadfL|pte}VHB2qGEQ zk*44|1TZ|1X=dWZlhFqwrRHzo2N~tl!*d=ua#6UpvvQVH?gz8$rJp&L-t1^y%$tF{ z-ln8lWJ8;`L`{u!@hjTf7&~_7zp#A~xXAB=CW2y6l3sNWNf3C=9(}(IOVX2}A~oL9 z3R=-a|E)<9(NqSecFradkr(#jBP$r& zcv`nHP$>nIvEV>q2e@}NAe1bZmHOW|&ApYyNwlm>cWs0*C7{?q#tzgb@;ZQQFSU2l zuGYoFU-_&qJ_fA{3a&vz5NU|@ZtOBtO)hHM^djQL;QMFkNJTypM?GO0_E&C4F@OtN zcM)yh4V3+o*UmXdBF@XP>HtMH@=oh7r-JQf;+8=`ZINrMg4>N#-ID+Xa?{A`d637DMt?G^_C!`0oit`y`V zzCeN|?H-K1t8w&^w*OH4=*aQUm4$r^f;>8%4sx7j`yBRg{^3<-WPKtb_DgrKy_x!J zSFq5UDNO`!2Ey6xaJLDwL&9RX@YBC^V2z0Pq`JCO#4{skxql(HZRrzB*q~ItOxpIJ zVKlk1c3@u*oM7E!yI+j;-bt;Hb&K@Vjj&D?J3^9jY|Xq4Fm9)I)7imW?GM3V?6ttg z|G)w!>Xf3J6zmPy-jh<$uS_X$qqsO7@3V{H=5=+zwklN6BD?2C%#P~wcc zD5%Zos#wOE?L{26tq8Yzngc9^7!RvL#KZVDrI$?nZMllzzt~;ks?xyf54xrN-PBpRv z+oyPc^~-cX%m0f7SS+C)0!e+yrQ4>@GODL3fBJ6sK~|T;@9S&66&O)mlbp(Z(3eYr z(r4^%q9p!GE>Bf*z+D0K(KNXD*6w&kNOX|Unt_~PH~-dT$4`) zm>=?r#X_nR>LQq!L7&zpiI44``lcyy&%`C1#CCyq%K>m=V;Xf)?f@U`!OhUvPyYAN*OGXjv8 zVUEEK$b^V3I6nTSH@^e=t=j=9l4eowniYU_6)oc&rFFangSHs>-Oo(%Gf%nv6~Cu) z;vrV>tu<{m$#T`zD+%|{6rX>NYWDLj6LchL8Mx(;wpwAQ!9O77+zBZn#KKsV_U{I_ z$A7%g(leh`a2FCF{09;+awFK(VWcXi50Lq6eBV?zn<#K)BN@pBJA|5YK>(q;ksmdO z2v<6HhX{{VnBGsGV0++Utuba)?56zXctU&xQ(>=n_?P4AO?|dp&r_!JzJ*><|+=Uod&2mz<#C;yc&*I%w6b5a_R ziaGfnKT_|;b0GC@SkHY+cy<8k`Qs>!T#zHsIbdIWQy1Y1a7Q2vg1dRy$OVgo$beAS z$H;*9X>`WGDM^HKaMz%7V9r4aN~zJRetK-DV`os^@J*C&E|5-pMYT39-R}{A+1cv9;yX=T< z;{UdH@BA96DYE%KK+HgUci*pU_hUoc|FlMM9$Ui_8*Ivl_7Obj;BLs?PUm1=pCdS%HujkW0DFdD0ym33p=@|5(i$JHjR|@(#HBLgQMPDHu7h`i3 z`7qUw6alAdti>+66n?N^yXaYHH-f3^T{s%uN4*fp@0R^REpuBJfG$BC1;$CmI`_2CY1faTM4C5x>EC^0|PI632}o2w}|N=aoYEq4kW4k(zYThzRZ_ zuEd9thj;9UMO(JP=HhrFQ1JuM66mY!fGn>b>;7`51t2G9B>$H|@8?gh2*jrD=;4BE;c-Ky+ue6^ooCMbrBAj;OJ8&5Xb~un5B2PcA zZFDGZ!{ta^+JSIf&n)TihvX6+1jzQQ4k2vuspGW?a-kA~1D+@QBnGvoR(~j*oh7s? z!OBPG1Qq}9))KICMzKQk>y=5o6uolYzNB(`;Ks14s{O&c9c^Z*)h|kVlg;EJwU-qo*uq!4>T(VNEN~xr1NgMr zC`B{YaM*2ni7d|Uj!Zv5jtIpdXUZr72k{w~)&U)A*lTU2akyTKFcQ#6{H>N=I>h>* zNE2v=YI^u1jy|UoMsFmGc$f6%RJjq)dzTzDwi^dAGIO<_mQxw|W>eUBf_NmyI%cD6 z84$jWAhnm8L&38C|I2PXHrYr+;M_VG<^Rt92mFUK5~Mj)Tk2Ff20%)MXBe-89BX6Y zO`5V0Q|k8nRXX(zLa1;LUU3JlP;=@tw=N^gZ%##w6;IM{3fx+Vj|f_&A|$~J*ag*{ zE?0ykNJx!FR1#BCzQ{1i&wPZb$h*HX)y}8^n;39o12sAaeJ%jJ25-r$Cd^XUe}AiY z1Ctq5{vJp&*n|q=fn^9*6ZLk|)f@ZP_}k~-qTpDKPz?AmH9mz! zh?M_TMcS;dTcQuCvOZ2jh1SalqH;+1k!8bR63-Z#YN(1I18OEK5R&pyzm-tVlPu{0 zQnG3r?G8)Y@U2af@b7-h?_VBEzxBmjxv~ zHu2Uy?uN*m`7nsFifwC!)t0mMxPNfm%$hpZ-=3^(NCebH>_;%@2a32!m}i`~*MZD6 z+fD;{{|5jmyPgJ6WqIp3IO)5Mq0;bXfU8~k7A$(-T@$$Ew|y)j@kXBrhEqASZUX(# zGoOG2!lD#iP&mAWw?&?ba4=5%%|w*rh*H%^2=xzJKViR&oooBJgOqw>;yx2VlQn63f`X5g?oajo0Yfdsa=O4u=}KPT0b$OF3MNV^AKlHiV5AmrPv8-81i$gC;6h^ifK==<{)>|98JYP z_euui?bI+6+nr~rl}`EHEx^Zcf&f?>9;JZ3Apbflva*s2!DGlE;St+D?^c%Q3Ch(s z@Kbw--1j;13A3h)X@z}tV9*E+Jlpkbge@ZbM%n@qh4?<&kK+25CQkn`P_&k7X@@1~ z-Q^A_Z9!X^;!k0m2vL+6kx8t&6hnl}jz+3T8xPivvm8Q54r9ixY_X{ZjtHNofO zNEidBX6@z2;O1@YFLAz3s_7ccl>(Vczv)OURd669#EhE+!_coaPhz5?SJM65#vtBr zv~E*?ONKLOs3k~XEQyRkvgA*ziunIl;~^34gjyJri7|xdzR*~weB+6t9m|2ldp-;C zeM9C#VSil}^+FKA*?~Jkyr=&5XXh=&*TB9F_{?0OxcXiImNzWOs5@~jii^}O%r1CP zo&Umo1ZG2ch6lhzR;AJiYocow~|7_RmCB(fLpW`K&O zG(#lZ5>e}l@~fQFA8lj9g5&~~dcM2M{doR0L^DAXiR;2zVVX{X^9udunPX;emBxqZ zIle87bp=;bLGQOA;7NxL9-sREpaT~=bRez(O1{ya^@aW@o$EC}VzJL8S3g~s(*vQ! ze%C-E)4^BVR|;9!?Zr>zHp~GuTWktAn*MJIdv2L3U+_hi2ywkGHTv=pH1ZNHXG%aX zdu$y@K&tY?Ov4$?w6MMm+s6WxZ1LPd6dpy+O5Fg&$9Rv_JD`)0Q1YZ(STtk!0>$~s zwT$w#ci`xQDwZTv^@o1&cO1-2z|AeU5JR9>UAI&Q3LED+@8Xrj#X#Tzz4gt>0`?LS zxo3EFziK>nbrmeS0ut8W!Ed2+uxOb{q}z8uZGi|N6@{A5EtR*7qY;3z+`bG?F&B5PC-cXGAKodOJW zz7Oc^S6Vva{4$HSVwsAgjl%G@j?8HC%9QAr`&n-Wa6}AghMb!UJ^SQ^1$I*_aw1ON z1jDstV)N&<=4&1=I`;o5ql|SPAo47)-X}1Qo)hs|r>u^Qd-< z{*s68tjzoF^N4+Y3;2rXjJ`ivwCwJC)s(`Ky!xQUcsopOPzZFe0s1S2OWz?=%GKDG&+Jp+0{dGlY`GS7rwh6qI(C2!8_ z0erO8>Wd}&2JiqMHT8591THz;)vSD_Bc>r3BZ3$5LLGnDIKS{yQ${=eiO(~4L+{|g z^LIS|@Nsvkn;KrDvn;R(Nv9zX))^uW>Qi&uV65lIqO}jtk5=uOy}AT3okxLuJ=uxW zVW!y&0aw1fZD!oe#9ZG{Iye4nHO{sMM{rmvdGGh8Sz-pQP^{+P5sEXzKlhItk% zPxz5RySCB5553Xa*Wst#8yPAa1nftDkK1dx>4p(_BV@YLn~8(hWeUn)t4{9?io#?t t1T7aKKq(=}-u&SB@Z{|Xe(75uuf-j^8vg&(Ff)cxFvx?5sWinM^zOhQJZv=S4L z28n@eW83{3e&6?Zy}PdcyXQXVKIc<=VA0f6|2`)+a;%fH6h42!W*g+yWlcMc5+vV> zS)}%F^cT7(4fJo!jxr8&kc2B}*3@MR@mC_zW3BH{7yP*tPX_(Alxv|&#+oS5gevr( zKH!THk{`5JFt>dDB{Q^v;K#CVFl8#|nf8lC;lh{BQ*l_q`vVg?!sgG<72My~Qs%p}q!L#&Bcz-oZ-4lQF)0=_kAv}znom9=iSf%@~+urXnXb<&OFj}tuP&6uk z&Y+vLx-6x%>{@Dju)MUerLCDR@7z67y3C4{$Y0+qeomE@g}rs2W-=}TD_<42-!H)# zbdBB}uFrChaaKqYVdV4clAvMz(9jpLn2xdKZJCx!feu#T_v+6jt?~t0ajv-DEw}*< z#9J?6f1RqJU6}7f_Yh7UmycG!k;WMP; zL!Z+Te?p@#i;tRAru{)D418$3OR!cIUPFzDcv=72opg7g_y24eW;XJJ_IWzjk3AyZ zeL6B^-?H7zK8&o(a?5VMdZdF5T6%{xaaj!qtW+=6Sd#-`~SbH9G zV^09lGKZu_=oiT@t7y-p$Dt0fj&p#n zA+mSik#}pvyB@p66Kn3Uk6s5|O^#*stq}GgeQWTBe`&7A%IZf<`J;pu+}16<=@V2{ zv0##mEc^zXY)=EaT76<_s+vo9$%&u*z!oO0Mocd)S%#}9CYx8a(wgwU8sa+tYS7{J z^;0}guc`T8xgD}1(EPMGgE~A+#IrdekZwu&?cAyKlo70dzSX(pEPKxYt1n$eLiLMJ z?eIHU#O};R^v7NP7q>^N(rChp7c^xU5yaC>6Z=m05zH@drMu$p-ROLS|6ar`Ag(eM zn%i$s;dp!a#f8ZD8AkrhN-{25miTA&R50J zZ*mtk#C}`vg^CHBpNW2ESHZ*5Gg!Vju9sJ`>reofdmU93q!dgu+y!xtZ~?C}9KDVm z3FS~2_Sh2H@ULH&0k3lmkTKER1LNguJGZ7J*Lk$xK}B0*wdpp9pihrg@0S;74KY0P zSCjiS0X>JGLAIL|YRv0mboAWyWpcvTDm2!tx(Mt;S6)W0jSJ#x{O#4{+P*%$X2QgK z_KL$CRz^Z*h(kicE-FzV{w(XomSS!e&D_zqsfy9q-EWW1grxCeiQhNF7f(QP}$s_Q)6uOP$_Do^=wGevAOtH(lb1%YU+ug<_GE8H^bsJL0{l z5x;fsBRDU@quRS4t&GJqA7`+X9>}>0pFI)v@R|TTkTK~zgADK6w*y2ih2FoH8xsQc z{&vmCd&sMNlM(2xNXU@qJ}=ojL%mkYehRd$ivnNpolLkT|tQYu^9nYWBA5#cH_ z*_`1!2V>G}V3`OFULC_>IH-X_mehyba;n)HDetA z=Al-P!pi;GqDPhn=rNV36C)puo7de|Cl?d8H5AjRSCRY{ib4abOL5fV&kYBIMz~&J zyIOd*ZxZX?EBV97Mkgv3pi|XE$J-C?#BlPG<&Qwf>yt0-j&()v(fTs|g4^M@j_trG zD+*gLe!8z_^N36E?k|@0;%{D8Ey$0W>B$SK!uI5jhj>YkX%6tP^7y{ot?-+_rxn(M z4?iKBhcaskkk<>qOvCq5s_y7X4jfqgg}-3&LC_*RCT7R_-CD*DYQHqFv)?z}d6X(P z(V2TyU-4Yau!f=)mp|RixhXBJ!2X}Z3)dQ*(^I^1C66(OXkJLh?#~n>5!4d#1^TD3CzfztWO2ETLJx@ zO`v}JCkfIhOzkYj6NgSwW(_8r4&qf_5g~dcOmp?(Ct`7U#@ZTl>Wg$auX^FWc;|Uz!>urQh(p>SCNXLR9Z~Im65?AhZFg)XdS6U*j*4$Pk+(QRsGII1#@`Q+IAa@k+g1)NUJ*( zW%N0`1Ewh#c>wqdv2YquSCTycaCPp=mE@c|D7#>xs{u_SHseyCQ)JX;g9T-h|Lb&$^pdeyd7i!S%6T;mIA zQDM9CFzE0`c4z>29SjxP>U>S0LTI7fK!qTGanrr~{SHiPk^rM%8}D4fDAb`g3h^(kp%T*w9%&$M5E`aTI=m0U)WQOt zZpNMmAAJw=N?lI8Ckp3Hun0^=f(V1&%#0?i_3Y%XbM2~as{ zD5f@v%>n%jrAvY9aKE8fZFqV!7`)&B)ui{(;)p28;Lj^g%Std8V+d7) zs2h2^*F%igoLOFvWcf$jin;b%NI8OCr+9Vq?_GCd=gS*jo@Zy{G7UWSo_!k3+}g&K zl;*V7?pV|R!O-o%t}eq%4#1sf%SSw%CKvvEEpr)Vd31p1ga7vTC(uuZB`%_Ob_R%#qa!Lmb_-l-!3teucheaGhgb;#-^43KKZAoU zDNP%O@yk4^hN1UK^hukrac!D3=wJs-p)AQkbPejk8?Xro2 z0Hwpc4MCM^!u}AQetfq4XX1qVFi|ouP(;C4Wg*AO`xVamru`sv`w5xf@&Q?vcc3rQ zze$_rG`{lfCpg)Nb--Kg`~Y;y1lQ3qQ#7e562^0;iv?BbwX(6`M(wk~`6 zA!tmQYV8?o#b`p9bcDxfPV`N0CV^kx7f5Eum)MbGFBru6mc7}b~oKu$Xn`uUOs!*x(!Lu4Y_LD1;=$&9!icnt(j1dB7dQxF}^`}_SHY- z_N|I#7qd>A-4`-$(-*ovfC&1{ZT_7uY0i>sG3E&fDC% z|0xyFb^`6%KJfCxAczr6?LU$MIr-hqpzbY&a`;~pQKwA!KLc|FryRU74TeU?pR9hv zEW^p@$4p$AIq&%4GQR7n+`|!~$ko&CxC^{F6;%N`0-_mL)eG*_$O;U5Yp<*DB+_LL zH!^|qoFFJ1WK@9C;Qv(UhEK4AD)9MFa4>3~AB5cHG*gfEA`OTGu5Ku`-k}c)+J=o` zeO%y$3AJIJceQ-Tk6N~2VnMYBzu;F3)ef_Wj#6Z^9koHoiR>6z4kjpzL96-_PQ^n6;7@7=${UWF<9_Z*9XxXoQ z8>r}eagCpG{^sX_TWTWX?tAYZ2vq#Kv=VtW9cP=1DKprl$9uD^S3Fxg=N8|h^D!3a zc7k)6%MkDL1@LwBsDqis-gU_X0{`*N<6^SM&D2kn9zLI5g7?MG;1fp_*YZ>IMVwxz zkIZY*V^+^zKaVIhJTqQMEY3}BY`5Zv%XrlJd<&g)sU|*qef#+n>4jTU$9O-|fb(2O zJ*56lw&nn~S`Y4sm%e|J24s#T%ibj!iewQxYF=C$wEMVSQyi#hlS|KX-NQ684k$O1 z7)JUL8Hz3(c!Qkw+g(ug$Cx{sWFwE`ULs)8?aUIY&X!TZ!{G&D`sPM`W%k6q?6UFX z-0ZfXQSkNt8VSnaqdfEWgQd5VVIrBoDSKZ@3_m{xB#IUDL7lxxJnNEnS!m0>YW+9& zP8!7s1nB;-%+O~XUXooTls{zmpJl~v$ZkKJT>ix!d0y~*rz)pWtQ5P0vcZ#D*BdZx z6|=4A5l5pB2vyD%#Uma)-Nj9-4eg$)7AMzoq>7pTO2Q0`; zObhGud!a4#pF{IPX6=Hdk~k~iS68B$-CT6sRhVhI$~r$5ZH{`xs@qsIIhG2_?|8k; z>+6qy&ic6YX%Ex0hxubKyS~kw`0;n}X}dz-y#1vK^DXuTuKM;yS3k}QEQ6l}xoOk* zQ0-&CPvPaEMK_6|{mTQy3C%r^GvtK;;+b|`ERrV;#AvOzcPy==F4q3-g*xw&>RujE zCAT=7u~Pybmj?4HN2jlN4p97FMUJk5QV$ROTz(Dzdt!G4jY?%A>|vfP*$wwSthfQ^ zVWw4Bcrvn9Nnm)l(w!#KZFmqmh$YWNgM`s#ThWAAAjc=+62&D5XgK%)jeI%?gTEq5 z(q{mDhYpG`RKvxO?pVv!MJQsYzCsdU>?i%5v>WkWQEw+);ZGlSS8s?JDn)1EE7dlw zNx#L@jix%ZBI2y{-JTi_K`KHJV)zAM=)uQFje=-55)9aIBleB$vj?DHtg zu@gHWcO0SOD~9`g@cAQ}&SP!prx)wR;hW!z-l0i66O!d0HO8M&uaEMDZjLimT#I&Q zPLAR2W^ao(~vY84ZJ`ka@HwC}ErhtAl%)sIB?IOAbpEx0qrl#oPkqV1kL}1(8t9@h&C@96jy*34FkYV z<3SqBb!`&8%xx0=ywMN?Mf+*=LlOpIXF5W)a{GP1v^2abIjB6$@K2y_Srmu)=a(Cd z&4gTGPCXfNdyPQusBZgHl(^%6 zoMu(L-apHMS;fL&9%Pi}5X9Eo_=JJg&J|xtPv&Q~%iKXu-M@XZR9;~s?ANvH1^dqH z$EfanGL2v_kE7CZ{g}Y=BO1Ie0UCb$7>3lV<$yO~5zaK+98y(R`q(xBoO=-T8(ym; zrt(kwnuE>n+f)7vH@Q_%ybmAcr#%x>B3zaFmAGx8cY4|5C>%*nxhWunz_QOYMqw<= z>I~iS=S`|nPh)6)P&_YU{*^ z)l2;5SQ6gmQyia4$LQR7lNZ)ths62(oLM4!q^t_-)$58qPq|CXFMMcVSa(rwq82K# zNdGl7pL)^c6zj+jW4%Iw`i!ulzS@#p&w`>cw3*w!^kDEQpWI)KXDpRVmG8{;zuxp8OTIth^PTv?lPWM)aCvk_*tq=t-cz$J8P}g- z8NoZgeQG~`Ml`YHT)xUppG3(K4o|(<_zG!l0ku&AG@#X&;_H{IoBUsdV2Nb!1%`L+ zDKZiBki4)EtNM}Xi1)A|7btQJyax53)35Eqyhm;gS&)4Wf5H7&8`81Jm0pVY*w{L@ zjUgpdecpw=QzCcVUIxjpe&DS6hh>V24EmR$D^LC<6k$gR1=n}rf_Gy;Bo6^9GxE9w;!kcv}sd9mF4VUx2Fz zDUsET(U6S=7)Yqq4Vsh1?n5*bSBnzY2{}eC6%b%Uk}5llY?~GEn&ah2IMsRoz_hbYJNDdNMGurVt0uMJ%M-;7|@5108e3-20mdhb2WK>YvB2*{`w zpY_=3u3djs?UsSZ34H$kX2%YtRHEBXXFT>}iOxB7I>583bR^ezDzb0(&enTtL$OG~ zNR|&`QAw7{-D(?G#uA@AL@1_%@9#&^Ewi|7+|v~T;=tR_+Wjusyb%yW@rz#QYvJir zQ>3}=okcR)CEZ-qT(E|W!&RY#OieV5hu)`_Q|d35Afz)0kF!-T-HdrK7^20QVEWKMq=M(#tIxIBuAHZC z56z}9%0hpQJIW^D2Z#=y}qQTs&TcNZI z*r97p@=ts1bf7C?QG+u`p*?Kf$uq~6d!f3jZ9CMS6~V)a@I3fR z(0gpVTvJWnTGar8e=S~b{~a-fd!V|$2UiUALG8OphT0AAC4g+fh)tsHzY82&G$?7Q zvEnf>R$)pJ=Zh8F4jK1O{AJJF(Y}l?&NOgf%^kP%FAq!C2(GW$W=xDIfiCpn_MWN# z)P7CJwxqSMFKBKbBAl;jvcjeG*G9`O!=0M@l4wH7uX`6YzIpH{n{L&CMU>LnS7Ez( zGuN=T8KGGHHX zaI*vY#&m3B2WEC2oPVboil_H0UC`Ku#hG*Zonq;PSU*z*;pyusxsD!@-hU7XNlQk) zc=O*i#utr1w%nxjt0a_14(3T5raTQpCHki@hZ}WP*4a0bI1cC23|9+8oRAvXD&e;5 z`hdsW3@JF5==>$iuvo!t_jqF>M z?Ub^ley+DoU0spSkurC=guTtZJm-z=PN*1lkmlDJmFi%Tag?}*)Z>87*cVz(DdwYI ze<~TdBLoP4CszKZ7970J5{QyAKKZkpDP^vVdWLnoKT4lw9@Y07sksA7wkOg@mw|N3 zp%{O7W#@vjiDIG|{YCvRQ2k|=RJ_MM;+g1GR>}q7JpzcDCBvx3JEk`DVD15WXdAOF z5O;%Ipyb`kSTm!5#Z(30x4iMoA!O@P(Eg<#VlrI9Su zqCDPU(ngvR@td58f-9GnWIP%i5F9?nBfA_+%stLm*+o5d>NE{Y*4QLGJTcAo?jsJR$iff;H1z6%Be4U z;VR|dyU~J{f$MHYnemid3_3x8!WQ8P_GH#&yvLn>9T_P#5(NQLLDDL)jVEK+v8d>I z`Rd*mESZ=%9|_Bnq3W*2XS*B{)94gsgc-HNyr(@;-zFXbYvbe>;y~~(J0e2A(QU2Z z^HiF=G@726uAw&e_3U-_W$l6vl+{4xiDnpB1w17@H5KgZ2rm!i!Pq-fI8j_u*xpY4E0+*^d6KDS3ONQF**TsL5YR=s-k_ez?W?~XomI&{R^G(y( zuX-k)t^0shRR1(t+geuEp*ij9s8;{tU9Fl~AISzgM8!tzj4$Kb`FT|2A!I?+)VdBr z7s>GgOM;iQ;9#@U#h6?WxrEB8?>*WIZnA~$Yb7k^(=O8Gw`8d!x#&0CS+TSeW%|aJ{yd^z^`*)`j$hbMjCv_EUoc>Ra(EoIs{ z)Q6)^hW8OA=Nf=~WWeCfAI8;`My<<gLLncVd zNbeN+UxNOVp?d_dQ$4@|q0{5>TzWAm6Vr~5RZK#Su82w^F^!tl2{+RN{Q?n-`_Yvxd}zeDmtg3s+^4FztuMwhVy8(%-( zR{v^6;4T7C)mofkQDmw1r(wg-o;<#$TlL_I*pNhd&D7xe`!sdmiGCU~{+&eYdx$uO z(?<SD zz-j6Z9z;M<>TU3tAvAnC_}CC`x%M8>&*9&1^NL*Cfd>cJlXYyZlGaiX=xe%N|mRf>Y7xy$~iu!-yNLQ#!U`#YAGtmjuGsF=dF zTX%W8CzS*ydQR?#{|vu+{F3J7aw+}9Vmn043E2UL6W_W8WsTsyYs$A0IB5xEL4P3Z zRub=dVYTcY&oVrIYxoZ)k%;caqmE;bE}DYSq2zxE{X~p(Bn2^y_;^QhAF+;;+@aMV zG(X0_#JGzTsTmRk(n2a7Py?gZ*vzNilM0f{@L4l$x+S=3x#8|qtt~m4QlSi(7zbV= z3lQh>Kk_tkMLTwb2bK5m^o9fASFr6+!xCt{gD9YYL!SrR#fp0k8vYWLpkqhLb-XMJ zBR!w_w2Q{F&Xs{7g4IzdIP^PRqoKb5=cY}dXmjaRd7#b=k%dCQeOaI;g}puiDIY)r z6e%>8SXN6k3ySTA$}Wk?TnFZ6*U5EX0^3ZS`5pbseyVV`!96RQ0)tz-Zm%O)6b;eG zzM%We108Eq-B;Y$BbfnJOr@$?Aqdfk-0 zBJEX4@GzImg@#hHh?2tB$?2WV=`8(k|4B?ZU($~CCZ&PL@dgfL3rTQ&X~CJi-qT{k ze?JK?nE|LSUi>b~SEwms4lm*`x9=Ia(D@5)m^}m)9yaFSCBL1tcAT6#- zj9_XmYQOWGLPBb8JdSV&F3DOI6byl+gm zdDz|hl$L+xt5No0V{j&~^7G&F4z~}gpOrh-0ueF9?@lW0I`>exq zf?jUKZDJ+D#C?ZHw+NuC%0WT<3W{M+1_(+I?Q8ln5C6A&-El=xl-6F30|9@|dAt^u zw$@eZeo~Ql``j}<amt*;1kg(XWZND|Oj z?Ts1E2~yjc*kyPMgfA~M+R=q^$`x3M$prhmk~zuuZ`8Ral-zwz8T*xrpBg(DV*-?N zji{*$B#(8eFQJ&18bWN+h?2UAAWVZ1;Y|*0B@k%|kXeVv`oOZ43d;62CjlZDp%qbd zD0O41>?BMV=SU|}gQhd;4QABF9Cr8g-E|am#K9ZQ;~g&tV7-TFx{Y2pAG}K9T1}sP zT&>ihfd)bupi9afM^7zBMB7&Y9Xqx3OB?>*X~lHAAPI`_rI*hiPR9PzXPGcrxIR1Z z5pRwJ`=$uJ9k`jUPZ|z+(&6t?cYH^n)Q!l@OMLM}ktnt~_!IXc#~XZmTOF4eryOuw zj=_%*%AB81qA=ubI7FuZOuF2puo`5iN?w@$3)#3Xp`3*J2@pj`+4U??9{dqkNgTdP zf=QTNUV=|hu=v}-M0!^Eauc|W>|4C&-b(zt!}@tjxkXtyLsl0w=I0MUh)*uE~l^SC$|@k>R#CUrV9*jZ`_DesOscQaYkQ$s3#ZVkBUeff1^CoFMi?N z2eAh|ave1{@$A3Y>Jy8vlyaZfqkEP&s`RJDr>26;Q^hmw>ekFXy@$mt>hV~SLxn&o z%zktaFy^iwL4As;o{;Q+_QGJ3vN#?RX0l6=EK0A-g0jTb(-itg^DZD58$UAM6)EUlk7oY|n$$1D6qSSq2nI4~a_b zL96(oz}Q~MX$h*eEYASP|D0R@+J>44-}ivK(Z0>}(S>`!XGt*(&ZKo@FD6a-^91Bn z@o@V27;QaX<-0*imgJU4>r}+<7SSfR4dbMM_Xd4_9N5bE*MKvZ+<-d7k6>-F_aq}! zi&X#YcZH+ht4$mms(cvVy zkMC}={j?9Mu&fz|2D#ImOrw}eY0a@bUB((}RbMl%%(*YVN%VG|%@x!5ppX&R_n_qs zz1lkw0lqPWiScb>rL4Sgk_t>qe!f(BcnZn7u0fSm+f)1gxlt1hLLcf#&|39V2 z0BVku-sE@V9r(I)7jbK9sg7>$n~BC)MXu+Ui~hq>Bk}Jfl01K&Suo7c4D%FVj%`JD zu`cp+y0cK;(P^IG<9ri#{uPYmPVd+eKjW`*;j{ge`3@?PN)+WIju40c&p67G4 ze;~QWs$dt^)q|__r5G=oMf7?UD6j6oudOOjAR!WDjxQ|iDTe7S4@z-e?&#E1e8;Cj zG}w5{3PL?kzMkAcA(8HT?i+QqbDQtW5wnQcfjq&}KvnYLhlP7*R!lP{mMML!s z2Kdu}2pgGS=BTwvY?*nWBk+is1LU)_!F?ci6%AO>@lR9DzxT&4JXu%peH1{f|AvE6 zjWii9YF@%dZATkDORdLf$8XSQ%8*&hkXYBgLwAxu@CflWD^pV+(U%St%bT;m&EYs$uy#gJcv-&v zobfh4>O$w~nlA)0AMGbDeC(bo=j?ILSH880nhGa2pIBS)7R=yTm_e358SO-*XOkri zQT^v2mNJ0GJIZ%>0P>*~9@;^v@9>LjND5&e#Q}+U*iEW-bOS1?Am5B_G7{_%M&7`7 zY#0@o86(MCu3EJmj#`|b&G3>x_b>?rxDq%|k~W~W+G3b34Ts0PD%dlCN>--*VQTEOF}{*>|30*ZPSL&;%m@$nfrB!kQZMTSWB*eIR> zV@C_x{g`5(vc9eTixTcrp4sma(bzTC%f2pydf4SrQXCnh6%#ertJ42i!s9)DlX#)| z#gq5e8$mZ)Y-(#b9BZQ;cy~}e>%f&5f2RN(XB^1}dUm-kbn>aLqDQdX4JkaN1QSKnVspd1yVh8Z(vR_IM}S$SWX*+65xLI_!~EG8 zvry+ZB)bQ+r}rQ!-T|VuZa^+b?0hXLY`Sqjkc48Y+`$<@q6US)Z{GPUaRPO@te(8( zA#XrWQ2xm!?36t?T_1$20_RB!Z-ZA+``gD?(2L|dl0V2**PjnV_>Lv~$N+Y!W1(X! zSKhjRcinnJKAwJx-YD6P!g)~nSLUJ?X*)U09{Jg; z!L`b_3?;7I?ElWzp{Jwa*kgVw;X|78N$0a-KNn*r?+s2C&(_9;C5uL1l9uj$J*J>+ zQ_N;u`ynAMjMuGY*-GT=%xhL|x?2BU1gj(vTmcu2ACMo7b^@(GK<|-N(u=))NciY- z^+rL^yt6)Ox!=QOsqU+>hy#LVS^h@nfO1RglKQU2@_mtKTr>)C_TC7l$=xh=ca64u zS9--=4%NEM>$IEw<*k=Z*`j=frv_O|X`_JNG?obS5l+o5RPFI2H{({3;LJF!Jr3 z=0#qzS#|J(vxw&|gjYzCb zrOPPubx=VEy6v_-bW>-Zb-=Fmbj`jTQKa>&{>6N=@f7`S_!F+PUMWuu3#2ENlI%_r zVkI4n9SD6N(>_Xt6{ywdGP4R9S|lN5unwoYD`&m7l0$Z6g=ceIm2CaLp&?NMBnQ7w z=BLDOC;urMpS#!oXQq3h8_5)VvbOn~qNR#{%YV$+ZRz)zBQ80+zKaW)IF2t55{Q#3 zJk%0elt1_GMZ>(j_yw4&#&@Dd%qzPdk5lc-bh30BN-GWs&9_|rxp);-aAF=xo&Y(W zAmfz`pfLx6M~qETQYt$Xa{C=8xCT`I>#pM*+T@yowG!N$ksXY9)`%lEx&WtrLTw`( z(=NXe!LT?m^BB$S8{OsKn$EYXBRNYvB=lC}y*RFfO(|C(5Nls591fXZi|?8w;Zi{mt*T&Bm*x;!p2 zOgt%KbEo3Y`PaNX7BrQd>JFScR__wgQ4O(_yu$Y=uei(O*ExGz?$2XB-}#V%$@-|A zs?*JFCD*m1k0+M&r1hHSe;U%?j|Vp1}DYyi>P9ZzpG0Ns%8 zI|}?K5}&-2cy|;M@qtJ#gE8Rpa3BM;ups#TQyYf+Jt~)V>1tk&o1zVS2eFJ~P68+9{xM$V;f;E7*DY#hir7V>BwO1aYk3DIWY zC0_(cUQ?z4bE3HT!UecYH$(v}5kyKRY@#o|{f>n=_R&d1O6M;B`oQVcPEd%~;sQpw zIBPb6ag!V81zyupss6a8Hvjeq9!4+>TxNtCY7gZ@Kt06*-laJC%mwI*W zYyD;lueL7yOAjANgJ*xvU1ot+k0UL0eKr*G;?}c4-izWVJy9a)UM5Y1F_ctfjc541 z&dI3JD|cZikC0FAbaFS=RNGefQk(J#uNa*eoXuc5D?vf*H$RWEw&;HDA%@u0ui24F z@3_R_(m`Y`IGpH%X!{<3OsSXW!P&SPvJPX={u@J}zCwrP+E$G;`DMm9_oL7Z0sqET zxqj6h3T}|~>)n=w&px{T;zf|!#L%i&gimBO!EoF(MO7J|X^|VCtPJyuy&GINX1y&5 zy}!+%J2v=yai#-}MzTwWFTm}Zo5qITp^?h&1IRGirccx$Qw zBC`Fp`?0Em+7FMjLqKHYp51d%z!Vuh7i8XduZ%ThSI0s^{R*S7KI54LV_Wv)#7-)B z5x2a%WD2`VSk=Til2Ay~{spE)LS!i?%$jdCS(i{MNaBPc6ZS-<`O9>M(^L4CBI9cs zL!+oPnXg2}%li#hLCc^F!NgNO<{QL6~9OijU`aUdS9iB8P1zZSE zpV_I;LnOpa-}-t5vgGP^OXZ?43prga28~rM;b%<*AQWsYO)Sm0RO3X#1E# zwKYagLv?h(_?-!o`2BZu>L-K0UnHHdvuh;SMqVf_oDfX>B`Rq9lIOt-xy1J7blQCH zPncN?q4l(+1Ri5|x5M#!0SRC}1;XJgR{=WP)@~j}>rK*R=+ zd)tJxebkYK;HbZ^8))$-LoHI44y using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.42 GB, active+cache 5.18 GB, phase peak 6.17 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 510.48 ms (1002.98 tok/s) + Decode: 3399.29 ms (37.65 tok/s) + MLX peak memory: 6.17 GB +rocm_gpu_guard: 2026-10-09T01:47:26+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:47:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log new file mode 100644 index 000000000..701febea2 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:46:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:46:06+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:46:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:46:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.207 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.68 GB, active+cache 5.43 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 506.72 ms (1010.42 tok/s) + Decode: 3397.42 ms (37.68 tok/s) + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T01:46:31+0900 monitor: 7 samples +rocm_gpu_guard: 2026-10-09T01:46:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log new file mode 100644 index 000000000..8d333433a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:46:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:46:36+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:46:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:46:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.200 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 506.60 ms (1010.65 tok/s) + Decode: 3356.95 ms (38.13 tok/s) + Decode path: engine + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T01:46:59+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:46:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..9fc0d9f1a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:06:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:06:37+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:06:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:06:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.186 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.69 GB, active+cache 5.44 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 513.71 ms (996.67 tok/s) + Decode: 3382.28 ms (37.84 tok/s) + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T02:07:00+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:07:00+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log new file mode 100644 index 000000000..11e0ad7bf --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:07:05+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:07:05+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:07:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:07:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.189 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 507.96 ms (1007.95 tok/s) + Decode: 3388.98 ms (37.77 tok/s) + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T02:07:27+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:07:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log new file mode 100644 index 000000000..cb54b69d5 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:06:07+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:06:07+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:06:22+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:06:22+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.190 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.42 GB, active+cache 5.18 GB, phase peak 5.11 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 932.40 ms (549.12 tok/s) + Decode: 3416.33 ms (37.47 tok/s) + Decode path: engine + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T02:06:32+0900 monitor: 7 samples +rocm_gpu_guard: 2026-10-09T02:06:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..c0c263329 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:26:00+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:26:00+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:26:15+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:26:15+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.193 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.68 GB, active+cache 5.43 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.15 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 516.54 ms (991.20 tok/s) + Decode: 3403.14 ms (37.61 tok/s) + MLX peak memory: 6.15 GB +rocm_gpu_guard: 2026-10-09T02:26:25+0900 monitor: 7 samples +rocm_gpu_guard: 2026-10-09T02:26:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log new file mode 100644 index 000000000..aff3ad82c --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:26:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:26:30+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:26:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:26:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.190 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.81 GB, active+cache 5.56 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.43 GB, active+cache 5.18 GB, phase peak 6.24 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 509.34 ms (1005.22 tok/s) + Decode: 3397.71 ms (37.67 tok/s) + MLX peak memory: 6.24 GB +rocm_gpu_guard: 2026-10-09T02:26:53+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:26:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log new file mode 100644 index 000000000..16455f3bc --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Meta-Llama-3.1-8B-Instruct-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:26:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:26:58+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:27:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:27:12+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Meta-Llama-3.1-8B-Instruct-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.191 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 5.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.75 GB, cache 0.38 GB, active+cache 5.13 GB, phase peak 6.25 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 513.78 ms (996.53 tok/s) + Decode: 3398.23 ms (37.67 tok/s) + Decode path: engine + MLX peak memory: 6.25 GB +rocm_gpu_guard: 2026-10-09T02:27:20+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:27:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..1efd8e5a7 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:02:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:02:31+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:02:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:02:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.055 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19862.45 ms (25.78 tok/s) + Decode: 12226.42 ms (10.47 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:03:44+0900 monitor: 38 samples +rocm_gpu_guard: 2026-10-09T02:03:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log new file mode 100644 index 000000000..53e1d3596 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:59:53+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:59:53+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:00:07+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:00:07+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.057 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.20 GB, active+cache 28.60 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.36 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19714.36 ms (25.97 tok/s) + Decode: 13213.77 ms (9.69 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:01:07+0900 monitor: 39 samples +rocm_gpu_guard: 2026-10-09T02:01:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log new file mode 100644 index 000000000..d6ee35222 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:01:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:01:12+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:01:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:01:27+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.057 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.22 GB, active+cache 26.61 GB, phase peak 27.51 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.22 GB, active+cache 26.61 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19839.49 ms (25.81 tok/s) + Decode: 13411.61 ms (9.54 tok/s) + Decode path: engine + MLX peak memory: 27.51 GB +rocm_gpu_guard: 2026-10-09T02:02:26+0900 monitor: 39 samples +rocm_gpu_guard: 2026-10-09T02:02:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..7c8d955a1 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:21:04+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:21:04+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:21:19+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:21:19+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.053 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 1.72 GB, active+cache 28.11 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19823.17 ms (25.83 tok/s) + Decode: 12615.91 ms (10.15 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:22:20+0900 monitor: 40 samples +rocm_gpu_guard: 2026-10-09T02:22:20+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log new file mode 100644 index 000000000..1458cbfe7 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:22:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:22:25+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:22:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:22:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.070 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19874.59 ms (25.76 tok/s) + Decode: 12839.71 ms (9.97 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:23:38+0900 monitor: 38 samples +rocm_gpu_guard: 2026-10-09T02:23:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log new file mode 100644 index 000000000..d8c70518a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:19:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:19:45+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:20:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:20:00+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.054 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.34 GB, active+cache 26.73 GB, phase peak 27.51 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.33 GB, active+cache 26.73 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19734.06 ms (25.94 tok/s) + Decode: 12681.93 ms (10.09 tok/s) + Decode path: engine + MLX peak memory: 27.51 GB +rocm_gpu_guard: 2026-10-09T02:20:59+0900 monitor: 39 samples +rocm_gpu_guard: 2026-10-09T02:20:59+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..113b99b6d --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:39:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:39:31+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:39:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:39:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.055 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19711.80 ms (25.97 tok/s) + Decode: 13062.31 ms (9.80 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:40:45+0900 monitor: 39 samples +rocm_gpu_guard: 2026-10-09T02:40:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log new file mode 100644 index 000000000..0ddc91c4e --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:40:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:40:50+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:41:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:41:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.054 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 2.26 GB, active+cache 28.66 GB, phase peak 27.52 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.37 GB, active+cache 26.76 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19663.82 ms (26.04 tok/s) + Decode: 12839.24 ms (9.97 tok/s) + MLX peak memory: 27.52 GB +rocm_gpu_guard: 2026-10-09T02:42:18+0900 monitor: 48 samples +rocm_gpu_guard: 2026-10-09T02:42:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log new file mode 100644 index 000000000..9ae178084 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Mixtral-8x7B-Instruct-v0.1-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:42:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:42:23+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:42:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:42:38+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.065 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 26.40 GB, cache 0.37 GB, active+cache 26.77 GB, phase peak 27.51 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(32768), reserved 128 for generation) +[Memory] after measured pass: active 26.40 GB, cache 0.34 GB, active+cache 26.74 GB, phase peak 27.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 19878.37 ms (25.76 tok/s) + Decode: 12548.46 ms (10.20 tok/s) + Decode path: engine + MLX peak memory: 27.51 GB +rocm_gpu_guard: 2026-10-09T02:43:37+0900 monitor: 38 samples +rocm_gpu_guard: 2026-10-09T02:43:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..1b37a199e --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base-null.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T01:56:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:56:56+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:57:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:57:11+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.321 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1851.40 ms (276.55 tok/s) + Decode: 1697.24 ms (75.42 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:57:26+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T01:57:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log new file mode 100644 index 000000000..19ea57f93 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_base.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T01:55:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:55:45+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:56:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:56:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.327 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1827.53 ms (280.16 tok/s) + Decode: 1698.51 ms (75.36 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:56:16+0900 monitor: 11 samples +rocm_gpu_guard: 2026-10-09T01:56:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log new file mode 100644 index 000000000..66f2305b7 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:56:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:56:21+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:56:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:56:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.330 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.17 GB, active+cache 18.18 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.01 GB, cache 0.26 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1719.00 ms (297.85 tok/s) + Decode: 1695.18 ms (75.51 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:56:51+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T01:56:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..faad783e9 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base-null.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T02:16:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:16:12+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:16:27+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:16:27+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.330 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1819.72 ms (281.36 tok/s) + Decode: 1698.86 ms (75.34 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:16:42+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:16:42+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log new file mode 100644 index 000000000..04b8096e8 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_base.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T02:16:47+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:16:47+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:17:01+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:17:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.329 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.31 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.14 GB, active+cache 18.21 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1767.78 ms (289.63 tok/s) + Decode: 1703.63 ms (75.13 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:17:16+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:17:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log new file mode 100644 index 000000000..2d77f0b75 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:15:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:15:36+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:15:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:15:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.337 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.18 GB, active+cache 18.19 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.01 GB, cache 0.26 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 2000.43 ms (255.95 tok/s) + Decode: 1700.67 ms (75.26 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:16:07+0900 monitor: 11 samples +rocm_gpu_guard: 2026-10-09T02:16:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..301464bad --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base-null.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T02:35:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:35:21+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:35:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:35:36+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.332 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.31 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1878.15 ms (272.61 tok/s) + Decode: 1696.57 ms (75.45 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:35:52+0900 monitor: 11 samples +rocm_gpu_guard: 2026-10-09T02:35:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log new file mode 100644 index 000000000..89793f6f2 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_base.log @@ -0,0 +1,27 @@ +rocm_gpu_guard: 2026-10-09T02:35:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:35:57+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:36:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:36:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[NemotronH] Loading config... +[NemotronH] Config loaded: 52 layers (23 mamba, 6 attention, 0 mlp, 23 moe) +[NemotronH] Loading weights from safetensors... +[NemotronH] Building model... +[NemotronH] Model loaded successfully +[Load] wall: 0.333 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.30 GB, active+cache 18.32 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.07 GB, cache 0.21 GB, active+cache 18.27 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1794.91 ms (285.25 tok/s) + Decode: 1700.18 ms (75.29 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:36:27+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:36:27+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log new file mode 100644 index 000000000..38c5bd715 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:36:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:36:32+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:36:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:36:46+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.336 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 18.01 GB, cache 0.17 GB, active+cache 18.18 GB, phase peak 18.17 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 18.01 GB, cache 0.23 GB, active+cache 18.24 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1856.01 ms (275.86 tok/s) + Decode: 1694.83 ms (75.52 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:37:01+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:37:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log new file mode 100644 index 000000000..04c95815c --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:19:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:19:10+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:45:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:45:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.273 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.80 GB, active+cache 2.18 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 448.08 ms (4570.58 tok/s) + Decode: 590.32 ms (216.83 tok/s) + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T01:46:01+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-09T01:46:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log new file mode 100644 index 000000000..31b35ff19 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_base.log @@ -0,0 +1,25 @@ +rocm_gpu_guard: 2026-10-08T21:33:51+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-08T22:19:00+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2709s +rocm_gpu_guard: 2026-10-08T22:19:00+0900 waiting for 60s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-08T22:19:18+0900 idle streak reset after 12s: gpu=[] compilers=[1173803:cargo] +rocm_gpu_guard: 2026-10-08T22:23:33+0900 idle streak reset after 1s: gpu=[] compilers=[1334471:cargo] +rocm_gpu_guard: 2026-10-08T23:05:01+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-08T23:05:01+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.252 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.80 GB, active+cache 2.18 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 432.66 ms (4733.48 tok/s) + Decode: 594.48 ms (215.31 tok/s) + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-08T23:05:05+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-08T23:05:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log new file mode 100644 index 000000000..f449f5bf9 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r1_new.log @@ -0,0 +1,26 @@ +rocm_gpu_guard: 2026-10-08T23:05:15+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-08T23:13:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 521s +rocm_gpu_guard: 2026-10-08T23:13:56+0900 waiting for 60s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-08T23:14:31+0900 idle streak reset after 23s: gpu=[3262038:rocminfo] compilers=[3262000:cargo] +rocm_gpu_guard: 2026-10-08T23:23:02+0900 idle streak reset after 14s: gpu=[] compilers=[3571564:cargo,3571605:rustc,3571606:rustc,3571607:rustc,3571608:rustc,3571609:rustc] +rocm_gpu_guard: 2026-10-08T23:28:13+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-08T23:28:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.250 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 474.47 ms (4316.35 tok/s) + Decode: 580.45 ms (220.52 tok/s) + Decode path: engine + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-08T23:28:17+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-08T23:28:17+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log new file mode 100644 index 000000000..f3165af6c --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:05:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:05:19+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:05:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:05:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.195 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 460.43 ms (4447.97 tok/s) + Decode: 586.82 ms (218.13 tok/s) + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T02:05:38+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-09T02:05:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log new file mode 100644 index 000000000..34c9ffe01 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:05:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:05:43+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:05:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:05:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.199 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 453.61 ms (4514.86 tok/s) + Decode: 574.66 ms (222.74 tok/s) + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T02:06:02+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-09T02:06:02+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log new file mode 100644 index 000000000..c9068650d --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:04:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:04:57+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:05:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:05:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.193 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.99 GB, active+cache 1.37 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 450.84 ms (4542.64 tok/s) + Decode: 590.98 ms (216.59 tok/s) + Decode path: engine + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T02:05:14+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:05:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log new file mode 100644 index 000000000..aff771ccf --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:24:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:24:51+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:25:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:25:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.200 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.82 GB, active+cache 2.20 GB, phase peak 1.86 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 452.00 ms (4530.98 tok/s) + Decode: 588.40 ms (217.54 tok/s) + MLX peak memory: 1.86 GB +rocm_gpu_guard: 2026-10-09T02:25:08+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:25:08+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log new file mode 100644 index 000000000..ba360683d --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:25:13+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:25:13+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:25:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:25:28+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.205 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 1.81 GB, active+cache 2.19 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 1.00 GB, active+cache 1.39 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 462.41 ms (4428.99 tok/s) + Decode: 586.97 ms (218.07 tok/s) + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T02:25:32+0900 monitor: 3 samples +rocm_gpu_guard: 2026-10-09T02:25:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log new file mode 100644 index 000000000..4e6ad39a4 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp2048_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:25:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:25:37+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:25:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:25:52+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.198 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.88 GB, active+cache 1.27 GB, phase peak 1.84 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.96 GB, active+cache 1.35 GB, phase peak 1.85 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 454.49 ms (4506.18 tok/s) + Decode: 591.13 ms (216.53 tok/s) + Decode path: engine + MLX peak memory: 1.85 GB +rocm_gpu_guard: 2026-10-09T02:25:55+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:25:55+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..d4f5a5e65 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-08T21:22:36+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-08T21:32:09+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 573s +rocm_gpu_guard: 2026-10-08T21:32:09+0900 waiting for 60s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-08T21:33:38+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-08T21:33:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.192 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.77 GB, active+cache 1.15 GB, phase peak 1.04 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 70.41 ms (7271.58 tok/s) + Decode: 462.36 ms (276.84 tok/s) + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-08T21:33:41+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-08T21:33:41+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log new file mode 100644 index 000000000..35457a283 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_base.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-08T19:17:44+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-08T19:41:27+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 1423s +rocm_gpu_guard: 2026-10-08T19:41:27+0900 waiting for 60s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-08T19:42:09+0900 idle streak reset after 28s: gpu=[] compilers=[3313432:cargo] +rocm_gpu_guard: 2026-10-08T20:39:08+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-08T20:39:08+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.741 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.36 GB, active+cache 0.74 GB, phase peak 0.63 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 73.12 ms (7002.12 tok/s) + Decode: 465.33 ms (275.07 tok/s) + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-08T20:39:15+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-08T20:39:15+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log new file mode 100644 index 000000000..7112e48e7 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r1_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-08T20:39:25+0900 waiting for guard lock /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-08T21:20:56+0900 acquired guard lock /tmp/mlxcel-rocm-gpu-guard.lock after 2491s +rocm_gpu_guard: 2026-10-08T21:20:56+0900 waiting for 60s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-08T21:22:23+0900 idle for 60s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-08T21:22:23+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.256 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.26 GB, active+cache 0.64 GB, phase peak 0.60 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 59.61 ms (8589.40 tok/s) + Decode: 468.02 ms (273.49 tok/s) + Decode path: engine + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-08T21:22:26+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-08T21:22:26+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..a8b18a976 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:04:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:04:12+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:04:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:04:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.197 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.67 GB, active+cache 1.06 GB, phase peak 0.95 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 61.93 ms (8267.07 tok/s) + Decode: 464.92 ms (275.31 tok/s) + MLX peak memory: 1.47 GB +rocm_gpu_guard: 2026-10-09T02:04:29+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:04:29+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log new file mode 100644 index 000000000..85f974b44 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:04:34+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:04:34+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:04:49+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:04:49+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.192 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.61 GB, active+cache 1.00 GB, phase peak 0.87 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.47 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 68.65 ms (7458.21 tok/s) + Decode: 463.13 ms (276.38 tok/s) + MLX peak memory: 1.47 GB +rocm_gpu_guard: 2026-10-09T02:04:52+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:04:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log new file mode 100644 index 000000000..d92cd768f --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:03:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:03:49+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:04:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:04:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.211 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.26 GB, active+cache 0.64 GB, phase peak 0.60 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.26 GB, active+cache 0.65 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 59.82 ms (8559.12 tok/s) + Decode: 467.00 ms (274.09 tok/s) + Decode path: engine + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-09T02:04:06+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:04:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..504582ada --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:23:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:23:43+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:23:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:23:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.195 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.56 GB, active+cache 0.95 GB, phase peak 0.83 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 65.70 ms (7793.54 tok/s) + Decode: 462.61 ms (276.69 tok/s) + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-09T02:24:01+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:24:01+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log new file mode 100644 index 000000000..ae61fa0ba --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:24:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:24:06+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:24:20+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:24:20+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.198 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.55 GB, active+cache 0.93 GB, phase peak 0.82 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 68.96 ms (7424.31 tok/s) + Decode: 465.01 ms (275.26 tok/s) + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-09T02:24:23+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:24:23+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log new file mode 100644 index 000000000..3639461b2 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-0.6B-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:24:28+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:24:28+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:24:43+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:24:43+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-0.6B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.194 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 0.39 GB, cache 0.30 GB, active+cache 0.69 GB, phase peak 0.83 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 0.39 GB, cache 0.31 GB, active+cache 0.69 GB, phase peak 1.46 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 68.81 ms (7440.48 tok/s) + Decode: 465.67 ms (274.88 tok/s) + Decode path: engine + MLX peak memory: 1.46 GB +rocm_gpu_guard: 2026-10-09T02:24:46+0900 monitor: 2 samples +rocm_gpu_guard: 2026-10-09T02:24:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log new file mode 100644 index 000000000..7c2287d83 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:53:41+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:53:41+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:53:56+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:53:56+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.212 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.91 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7150.03 ms (286.43 tok/s) + Decode: 2200.86 ms (58.16 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T01:54:21+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T01:54:21+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log new file mode 100644 index 000000000..c90caa743 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:52:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:52:12+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:52:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:52:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.210 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.91 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7161.06 ms (285.99 tok/s) + Decode: 2205.06 ms (58.05 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T01:52:52+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T01:52:52+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log new file mode 100644 index 000000000..5130d3887 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:52:57+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:52:57+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:53:11+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:53:11+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.207 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7249.95 ms (282.48 tok/s) + Decode: 2196.16 ms (58.28 tok/s) + Decode path: engine + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T01:53:36+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T01:53:36+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log new file mode 100644 index 000000000..e00939b08 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:12:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:12:50+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:13:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:13:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.217 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7272.66 ms (281.60 tok/s) + Decode: 2200.73 ms (58.16 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:13:28+0900 monitor: 16 samples +rocm_gpu_guard: 2026-10-09T02:13:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log new file mode 100644 index 000000000..7fc69a5b3 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:13:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:13:33+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:13:48+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:13:48+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.251 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7144.89 ms (286.64 tok/s) + Decode: 2202.88 ms (58.11 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:14:13+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T02:14:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log new file mode 100644 index 000000000..7cf1a0ea4 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:11:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:11:54+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:12:09+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:12:09+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.221 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7053.70 ms (290.34 tok/s) + Decode: 2201.71 ms (58.14 tok/s) + Decode path: engine + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:12:44+0900 monitor: 24 samples +rocm_gpu_guard: 2026-10-09T02:12:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log new file mode 100644 index 000000000..d8f0ae959 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:31:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:31:48+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:32:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:32:03+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.217 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7198.39 ms (284.51 tok/s) + Decode: 2208.97 ms (57.95 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:32:28+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T02:32:28+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log new file mode 100644 index 000000000..1822eec3f --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:32:33+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:32:33+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:32:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:32:47+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.215 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.90 GB, active+cache 19.17 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.54 GB, active+cache 18.81 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7080.17 ms (289.26 tok/s) + Decode: 2192.55 ms (58.38 tok/s) + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:33:13+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T02:33:13+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log new file mode 100644 index 000000000..e7c91f7b0 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp2048_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:33:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:33:18+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:33:32+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:33:32+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.215 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 1.37 GB, active+cache 18.63 GB, phase peak 18.80 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 1.37 GB, active+cache 18.64 GB, phase peak 18.91 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 7148.04 ms (286.51 tok/s) + Decode: 2204.57 ms (58.06 tok/s) + Decode path: engine + MLX peak memory: 18.91 GB +rocm_gpu_guard: 2026-10-09T02:33:57+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T02:33:57+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..a7916b02e --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:51:37+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:51:37+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:51:52+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:51:52+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.209 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1806.26 ms (283.46 tok/s) + Decode: 2042.81 ms (62.66 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:52:07+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T01:52:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log new file mode 100644 index 000000000..20dccea75 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:50:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:50:11+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:50:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:50:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.216 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 2292.18 ms (223.37 tok/s) + Decode: 2046.99 ms (62.53 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:50:58+0900 monitor: 21 samples +rocm_gpu_guard: 2026-10-09T01:50:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log new file mode 100644 index 000000000..87bc3e5b4 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:51:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:51:03+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:51:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:51:17+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.216 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1889.81 ms (270.93 tok/s) + Decode: 2047.51 ms (62.52 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T01:51:32+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T01:51:32+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..0f456150f --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:10:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:10:44+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:10:59+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:10:59+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.208 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.77 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1605.19 ms (318.97 tok/s) + Decode: 2048.39 ms (62.49 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:11:14+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:11:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log new file mode 100644 index 000000000..634199821 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:11:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:11:19+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:11:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:11:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.213 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1736.22 ms (294.89 tok/s) + Decode: 2055.99 ms (62.26 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:11:49+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:11:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log new file mode 100644 index 000000000..484b481ff --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:10:10+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:10:10+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:10:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:10:25+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.220 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.39 GB, active+cache 17.66 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1964.26 ms (260.66 tok/s) + Decode: 2050.47 ms (62.42 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:10:39+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:10:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..47da1dbfb --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:30:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:30:03+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:30:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:30:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.213 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1982.61 ms (258.25 tok/s) + Decode: 2039.94 ms (62.75 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:30:34+0900 monitor: 11 samples +rocm_gpu_guard: 2026-10-09T02:30:34+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log new file mode 100644 index 000000000..f63e12679 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:30:39+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:30:39+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:30:54+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:30:54+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.220 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.65 GB, active+cache 17.92 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1807.65 ms (283.24 tok/s) + Decode: 2039.27 ms (62.77 tok/s) + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:31:09+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:31:09+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log new file mode 100644 index 000000000..f268c4011 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/Qwen3-30B-A3B-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:31:14+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:31:14+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:31:28+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:31:28+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/Qwen3-30B-A3B-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.223 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 17.27 GB, cache 0.38 GB, active+cache 17.65 GB, phase peak 17.66 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(40960), reserved 128 for generation) +[Memory] after measured pass: active 17.27 GB, cache 0.50 GB, active+cache 17.76 GB, phase peak 18.66 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1903.10 ms (269.03 tok/s) + Decode: 2041.74 ms (62.69 tok/s) + Decode path: engine + MLX peak memory: 18.66 GB +rocm_gpu_guard: 2026-10-09T02:31:43+0900 monitor: 10 samples +rocm_gpu_guard: 2026-10-09T02:31:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log new file mode 100644 index 000000000..90c19449e --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:49:44+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:49:44+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:49:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:49:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.630 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.04 GB, active+cache 4.02 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 702.59 ms (2914.94 tok/s) + Decode: 2046.94 ms (62.53 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T01:50:06+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:50:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log new file mode 100644 index 000000000..06a2380fd --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:48:50+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:48:50+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:49:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:49:04+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.633 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.27 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 720.27 ms (2843.38 tok/s) + Decode: 2037.11 ms (62.83 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T01:49:12+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:49:12+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log new file mode 100644 index 000000000..813cdab26 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r1_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T01:49:17+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:49:17+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:49:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:49:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.639 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 1.33 GB, active+cache 4.14 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 702.60 ms (2914.89 tok/s) + Decode: 2061.91 ms (62.08 tok/s) + Decode path: engine + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T01:49:39+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:49:39+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log new file mode 100644 index 000000000..310273323 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:09:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:09:16+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:09:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:09:31+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.668 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 703.54 ms (2911.00 tok/s) + Decode: 2046.37 ms (62.55 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:09:38+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:09:38+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log new file mode 100644 index 000000000..a1da50093 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:09:43+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:09:43+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:09:58+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:09:58+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.662 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.04 GB, active+cache 4.02 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 706.78 ms (2897.64 tok/s) + Decode: 2045.58 ms (62.57 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:10:05+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:10:05+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log new file mode 100644 index 000000000..d77e36575 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r2_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:08:49+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:08:49+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:09:04+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:09:04+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.649 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 1.42 GB, active+cache 4.22 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 709.08 ms (2888.25 tok/s) + Decode: 2074.55 ms (61.70 tok/s) + Decode path: engine + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:09:11+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:09:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log new file mode 100644 index 000000000..76c7e1a9e --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:28:42+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:28:42+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:28:57+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:28:57+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.648 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.27 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 703.00 ms (2913.23 tok/s) + Decode: 2043.34 ms (62.64 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:29:04+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:29:04+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log new file mode 100644 index 000000000..9fd239a73 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:29:09+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:29:09+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:29:24+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:29:24+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.648 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 2.47 GB, active+cache 5.28 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.97 GB, cache 1.21 GB, active+cache 4.18 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 706.11 ms (2900.41 tok/s) + Decode: 2042.94 ms (62.65 tok/s) + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:29:31+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:29:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log new file mode 100644 index 000000000..5bed9a721 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp2048_r3_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:29:36+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:29:36+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:29:51+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:29:51+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 2048 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.641 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 1.24 GB, active+cache 4.04 GB, phase peak 4.27 GB +[long-prompt] target=2048 tokens -> using 2048 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 1.25 GB, active+cache 4.05 GB, phase peak 4.52 GB +[Profile Results] + Prompt tokens: 2048 + Generated tokens: 128 + Prefill: 701.77 ms (2918.35 tok/s) + Decode: 2060.93 ms (62.11 tok/s) + Decode path: engine + MLX peak memory: 4.52 GB +rocm_gpu_guard: 2026-10-09T02:29:58+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:29:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..69eba7246 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:48:24+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:48:24+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:48:39+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:48:39+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.623 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.25 GB, active+cache 3.16 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 228.61 ms (2239.65 tok/s) + Decode: 1856.04 ms (68.96 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T01:48:45+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T01:48:45+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log new file mode 100644 index 000000000..5002b4b18 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:47:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:47:31+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:47:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:47:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.666 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.52 GB, active+cache 3.33 GB, phase peak 3.09 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.24 GB, active+cache 3.15 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 480.68 ms (1065.16 tok/s) + Decode: 1853.53 ms (69.06 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T01:47:53+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T01:47:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log new file mode 100644 index 000000000..b5b637887 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r1_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T01:47:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:47:58+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:48:13+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:48:13+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.660 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.38 GB, active+cache 3.18 GB, phase peak 3.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 0.37 GB, active+cache 3.18 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 231.93 ms (2207.57 tok/s) + Decode: 1852.54 ms (69.09 tok/s) + Decode path: engine + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T01:48:19+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T01:48:19+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..76a3c6c70 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:07:58+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:07:58+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:08:12+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:08:12+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.661 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.09 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.28 GB, active+cache 3.19 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 236.92 ms (2161.07 tok/s) + Decode: 1852.86 ms (69.08 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:08:18+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:08:18+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log new file mode 100644 index 000000000..390c92703 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:08:23+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:08:23+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:08:38+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:08:38+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.662 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.29 GB, active+cache 3.20 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 237.18 ms (2158.68 tok/s) + Decode: 1847.37 ms (69.29 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:08:44+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:08:44+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log new file mode 100644 index 000000000..43ebfe808 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r2_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:07:32+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:07:32+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:07:47+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:07:47+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.658 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.34 GB, active+cache 3.14 GB, phase peak 3.09 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 0.39 GB, active+cache 3.20 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 234.46 ms (2183.76 tok/s) + Decode: 1857.67 ms (68.90 tok/s) + Decode path: engine + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:07:53+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:07:53+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..6fb346d2a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:27:25+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:27:25+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:27:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:27:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.671 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.25 GB, active+cache 3.16 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 229.13 ms (2234.52 tok/s) + Decode: 1834.46 ms (69.78 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:27:46+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:27:46+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log new file mode 100644 index 000000000..29eb6b311 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:27:51+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:27:51+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:28:05+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:28:05+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.648 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.53 GB, active+cache 3.33 GB, phase peak 3.10 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.91 GB, cache 0.29 GB, active+cache 3.20 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 237.50 ms (2155.78 tok/s) + Decode: 1849.93 ms (69.19 tok/s) + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:28:11+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:28:11+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log new file mode 100644 index 000000000..6e00f6967 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gemma-3-4b-it-4bit_pp512_r3_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:28:16+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:28:16+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:28:31+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:28:31+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gemma-3-4b-it-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Warning: model.safetensors.index.json in models/mlx/gemma-3-4b-it-4bit references shards that don't match the on-disk files (likely a repackaged mlx-community quant). Falling back to all *.safetensors files in the directory. +[Load] wall: 0.638 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 2.80 GB, cache 0.38 GB, active+cache 3.18 GB, phase peak 3.09 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=None, reserved 128 for generation) +[Memory] after measured pass: active 2.80 GB, cache 0.39 GB, active+cache 3.20 GB, phase peak 3.92 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 229.49 ms (2231.04 tok/s) + Decode: 1865.00 ms (68.63 tok/s) + Decode path: engine + MLX peak memory: 3.92 GB +rocm_gpu_guard: 2026-10-09T02:28:37+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:28:37+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log new file mode 100644 index 000000000..3c51ef697 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:59:12+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:59:12+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:59:26+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:59:26+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.489 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 968.89 ms (528.44 tok/s) + Decode: 15015.86 ms (8.52 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T01:59:48+0900 monitor: 14 samples +rocm_gpu_guard: 2026-10-09T01:59:48+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log new file mode 100644 index 000000000..56373792d --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:57:31+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:57:31+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:57:46+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:57:46+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.570 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.82 GB, active+cache 13.08 GB, phase peak 12.88 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 969.43 ms (528.15 tok/s) + Decode: 14985.87 ms (8.54 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T01:58:22+0900 monitor: 24 samples +rocm_gpu_guard: 2026-10-09T01:58:22+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log new file mode 100644 index 000000000..1d75c1fda --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r1_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T01:58:27+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:58:27+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:58:42+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:58:42+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.566 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1004.76 ms (509.57 tok/s) + Decode: 15016.75 ms (8.52 tok/s) + Decode path: engine + MLX peak memory: 13.67 GB +rocm_gpu_guard: 2026-10-09T01:59:07+0900 monitor: 16 samples +rocm_gpu_guard: 2026-10-09T01:59:07+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log new file mode 100644 index 000000000..aba3ec3cd --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:18:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:18:19+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:18:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:18:33+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.555 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 973.88 ms (525.73 tok/s) + Decode: 15085.56 ms (8.48 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T02:18:58+0900 monitor: 16 samples +rocm_gpu_guard: 2026-10-09T02:18:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log new file mode 100644 index 000000000..111adbc8a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:19:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:19:03+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:19:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:19:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.502 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 971.68 ms (526.92 tok/s) + Decode: 15123.50 ms (8.46 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T02:19:40+0900 monitor: 15 samples +rocm_gpu_guard: 2026-10-09T02:19:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log new file mode 100644 index 000000000..c3d0c9d91 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r2_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:17:21+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:17:21+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:17:36+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:17:36+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.579 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 12.88 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1007.14 ms (508.37 tok/s) + Decode: 15144.73 ms (8.45 tok/s) + Decode path: engine + MLX peak memory: 13.60 GB +rocm_gpu_guard: 2026-10-09T02:18:14+0900 monitor: 25 samples +rocm_gpu_guard: 2026-10-09T02:18:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log new file mode 100644 index 000000000..4e6a5e281 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base-null.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:37:06+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:37:06+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:37:21+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:37:21+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.615 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.83 GB, active+cache 13.09 GB, phase peak 12.88 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 971.76 ms (526.88 tok/s) + Decode: 15159.36 ms (8.44 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T02:37:58+0900 monitor: 25 samples +rocm_gpu_guard: 2026-10-09T02:37:58+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log new file mode 100644 index 000000000..315b97ec5 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_base.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:38:03+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:38:03+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:38:18+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:38:18+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.546 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.26 GB, cache 1.90 GB, active+cache 13.16 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.84 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1097.23 ms (466.63 tok/s) + Decode: 15250.46 ms (8.39 tok/s) + MLX peak memory: 13.84 GB +rocm_gpu_guard: 2026-10-09T02:38:43+0900 monitor: 17 samples +rocm_gpu_guard: 2026-10-09T02:38:43+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log new file mode 100644 index 000000000..ad6ea9f63 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/gpt-oss-20b-MXFP4-Q4_pp512_r3_new.log @@ -0,0 +1,24 @@ +rocm_gpu_guard: 2026-10-09T02:38:48+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:38:48+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:39:03+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:39:03+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/gpt-oss-20b-MXFP4-Q4 -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +Quantization mode mxfp4: running on native ROCm kernels, no load-time conversion (backend quantization capability table) +[Load] wall: 0.499 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 11.28 GB, cache 0.27 GB, active+cache 11.55 GB, phase peak 13.67 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 11.30 GB, cache 0.28 GB, active+cache 11.57 GB, phase peak 13.60 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1002.98 ms (510.48 tok/s) + Decode: 15075.33 ms (8.49 tok/s) + Decode path: engine + MLX peak memory: 13.67 GB +rocm_gpu_guard: 2026-10-09T02:39:25+0900 monitor: 15 samples +rocm_gpu_guard: 2026-10-09T02:39:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log new file mode 100644 index 000000000..c46dd80b3 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:55:19+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:55:19+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:55:34+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:55:34+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.118 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 865.72 ms (591.41 tok/s) + Decode: 1435.18 ms (89.19 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T01:55:40+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T01:55:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log new file mode 100644 index 000000000..726c5040c --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T01:54:26+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:54:26+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:54:40+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:54:40+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.137 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1141.73 ms (448.44 tok/s) + Decode: 1426.12 ms (89.75 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T01:54:49+0900 monitor: 6 samples +rocm_gpu_guard: 2026-10-09T01:54:49+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log new file mode 100644 index 000000000..059b91b1d --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r1_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T01:54:54+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T01:54:54+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T01:55:08+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T01:55:08+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.122 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 869.89 ms (588.58 tok/s) + Decode: 1444.81 ms (88.59 tok/s) + Decode path: engine + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T01:55:14+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T01:55:14+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log new file mode 100644 index 000000000..4b892dd4a --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:14:45+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:14:45+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:15:00+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:15:00+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.126 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 894.81 ms (572.19 tok/s) + Decode: 1429.58 ms (89.54 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:15:06+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:15:06+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log new file mode 100644 index 000000000..3fcd771ef --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:15:11+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:15:11+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:15:25+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:15:25+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.121 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.76 GB, active+cache 7.71 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 894.08 ms (572.66 tok/s) + Decode: 1429.87 ms (89.52 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:15:31+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:15:31+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log new file mode 100644 index 000000000..7f9a607db --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r2_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:14:18+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:14:18+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:14:33+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:14:33+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.131 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 3.96 GB, cache 1.84 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1128.64 ms (453.64 tok/s) + Decode: 1438.29 ms (88.99 tok/s) + Decode path: engine + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:14:40+0900 monitor: 5 samples +rocm_gpu_guard: 2026-10-09T02:14:40+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log new file mode 100644 index 000000000..d82645bb6 --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base-null.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:34:02+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:34:02+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:34:17+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:34:17+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.131 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 6.61 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 1331.37 ms (384.57 tok/s) + Decode: 1427.41 ms (89.67 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:34:25+0900 monitor: 6 samples +rocm_gpu_guard: 2026-10-09T02:34:25+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log new file mode 100644 index 000000000..c855ae2bb --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_base.log @@ -0,0 +1,22 @@ +rocm_gpu_guard: 2026-10-09T02:34:30+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:34:30+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:34:45+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:34:45+0900 attempt 1/10: start: /tmp/claude-1001/-home-devops-Development-mlxcel/66a03ced-36c6-4e0a-9e39-155a3eb57353/scratchpad/issue-2192/pre-epic/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.123 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 3.20 GB, active+cache 7.16 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 4.02 GB, cache 1.78 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 868.14 ms (589.77 tok/s) + Decode: 1431.48 ms (89.42 tok/s) + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:34:51+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:34:51+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log new file mode 100644 index 000000000..ab486a6fb --- /dev/null +++ b/docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/rounds/raw/granite-4.0-h-tiny-4bit_pp512_r3_new.log @@ -0,0 +1,23 @@ +rocm_gpu_guard: 2026-10-09T02:34:56+0900 lock held by outer guard 2263193: not taking /tmp/mlxcel-rocm-gpu-guard.lock +rocm_gpu_guard: 2026-10-09T02:34:56+0900 waiting for 10s of idle GPU and no compiler +rocm_gpu_guard: 2026-10-09T02:35:10+0900 idle for 10s: kfd proc empty and no compiler in every 1 Hz sample +rocm_gpu_guard: 2026-10-09T02:35:10+0900 attempt 1/10: start: /home/devops/Development/wt-20261006T1212-issue-2192/target/release/mlxcel-bench-decode -m models/mlx/granite-4.0-h-tiny-4bit -p x -n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens 512 +[mlx-rocm] bound HIP device 0: gfx1151 (AMD Radeon 8060S Graphics) cus=20 warp=32 lds=64KB +[Memory] cache limit: 2.15 GB +[Load] wall: 0.121 s MLX peak: 0.00 GB +[Memory] after load: active 0.00 GB, cache 0.00 GB, active+cache 0.00 GB, phase peak 0.00 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[mlx-rocm] gfx1151 native_wmma=1 cus=20 warp=32 low_cu=0 host_WARP_SIZE=32 +[hipBLASLt caps] device 0 (gfx1151): bf16=1 fp8_e4m3=0 fp8_e5m2=0 int8=1 +[Memory] after warmup (prefill + warmup tokens): active 3.96 GB, cache 1.85 GB, active+cache 5.81 GB, phase peak 7.22 GB +[long-prompt] target=512 tokens -> using 512 tokens (max_context=Some(131072), reserved 128 for generation) +[Memory] after measured pass: active 3.96 GB, cache 1.84 GB, active+cache 5.80 GB, phase peak 7.31 GB +[Profile Results] + Prompt tokens: 512 + Generated tokens: 128 + Prefill: 855.19 ms (598.70 tok/s) + Decode: 1428.92 ms (89.58 tok/s) + Decode path: engine + MLX peak memory: 7.31 GB +rocm_gpu_guard: 2026-10-09T02:35:16+0900 monitor: 4 samples +rocm_gpu_guard: 2026-10-09T02:35:16+0900 attempt 1: CLEAN, exit 0 diff --git a/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md b/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md index f54f96dfa..6ea0d56d1 100644 --- a/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md +++ b/docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md @@ -126,7 +126,7 @@ ADR 0007's ROCm paragraph: the CLI and the server engine with dense storage must ### Method -`data/.../harness/bench_rounds.sh`: for each (model, prompt length) cell, three interleaved rounds of three arms, each a fresh `mlxcel-bench-decode` process at `scripts/bench_decode.sh`'s shape (`-n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens N`, greedy): `base` is the pre-epic binary (`4c44e317`, `CxxGenerator`), `new` is `main` (`DirectEngine`, `Decode path: engine`), and `base-null` is the pre-epic binary again, the method's noise floor. The arm order rotates each round. Every run was its own `scripts/rocm_gpu_guard.sh` command inside one host-wide lock hold (`data/.../harness/full_session.sh`): 10 s of idle GPU and no compiler before each run, a 1 Hz monitor during it, and all 99 runs were CLEAN on the first attempt (`data/.../guard/guard-logs.tar.gz`). The session ran 2026-10-09 01:19 to 02:43 KST. `data/.../harness/summarize_rounds.py` computes the medians and the paired per-round deltas below (median and range over the three rounds, percent of `base`). The threshold is ADR 0007's 1.0 percent on the paired decode delta. +`data/.../harness/bench_rounds.sh`: for each (model, prompt length) cell, three interleaved rounds of three arms, each a fresh `mlxcel-bench-decode` process at `scripts/bench_decode.sh`'s shape (`-n 128 --warmup-tokens 20 --ignore-eos --prompt-tokens N`, greedy): `base` is the pre-epic binary (`4c44e317`, `CxxGenerator`), `new` is `main` (`DirectEngine`, `Decode path: engine`), and `base-null` is the pre-epic binary again, the method's noise floor. The arm order rotates each round. Every run was its own `scripts/rocm_gpu_guard.sh` command inside one host-wide lock hold (`data/.../harness/full_session.sh`): 10 s of idle GPU and no compiler before each run, a 1 Hz monitor during it, and all 99 runs were CLEAN on the first attempt (`data/.../guard/logs/`). The session ran 2026-10-09 01:19 to 02:43 KST. `data/.../harness/summarize_rounds.py` computes the medians and the paired per-round deltas below (median and range over the three rounds, percent of `base`). The threshold is ADR 0007's 1.0 percent on the paired decode delta. ### Decode, `mlxcel generate` (DirectEngine) vs `CxxGenerator`