Model: GLM-5.2-UD-Q4_K_RoutedQ4K.gguf (official antirez/GLM-5.2-GGUF, 434 GB)
Machine: Mac Studio M3 Ultra, 512 GB, Metal backend
Symptom: short greedy prefill is fine, but any longer generation emits
fluent text with zero relation to the prompt. At --temp 0 the garbage
differs run to run (Wikipedia-style prose, source code, other languages)
— i.e. the decode is nondeterministic, pointing at a race / uninitialised
memory in the attention path.
Repro (no server needed):
./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ctx 4096 --tokens 240
-p "Was ist 2+2? Antworte nur mit der Zahl."
-> instead of "4", unrelated fluent text; different each run at temp 0.
git bisect (good=4771329, bad=b0982a1):
first bad commit = b0c31af "Improve GLM 5.3 attention memory and batching"
Parent 4771329 is clean. The commit refactors shared attention/batching
in ds4.c (labelled GLM 5.3) but regresses GLM 5.2 Metal decode.
Log shows the compact-DSA-KV path:
"GLM graph using compact DSA KV only; expanded full-attention KV cache is skipped"
Model: GLM-5.2-UD-Q4_K_RoutedQ4K.gguf (official antirez/GLM-5.2-GGUF, 434 GB)
Machine: Mac Studio M3 Ultra, 512 GB, Metal backend
Symptom: short greedy prefill is fine, but any longer generation emits
fluent text with zero relation to the prompt. At --temp 0 the garbage
differs run to run (Wikipedia-style prose, source code, other languages)
— i.e. the decode is nondeterministic, pointing at a race / uninitialised
memory in the attention path.
Repro (no server needed):
./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ctx 4096 --tokens 240
-p "Was ist 2+2? Antworte nur mit der Zahl."
-> instead of "4", unrelated fluent text; different each run at temp 0.
git bisect (good=4771329, bad=b0982a1):
first bad commit = b0c31af "Improve GLM 5.3 attention memory and batching"
Parent 4771329 is clean. The commit refactors shared attention/batching
in ds4.c (labelled GLM 5.3) but regresses GLM 5.2 Metal decode.
Log shows the compact-DSA-KV path:
"GLM graph using compact DSA KV only; expanded full-attention KV cache is skipped"