Skip to content

GLM 5.2 Q4 on Metal (M3 Ultra) produces fluent but unrelated output after commit b0c31af #932

Description

@richyy7

Model: GLM-5.2-UD-Q4_K_RoutedQ4K.gguf (official antirez/GLM-5.2-GGUF, 434 GB)
Machine: Mac Studio M3 Ultra, 512 GB, Metal backend

Symptom: short greedy prefill is fine, but any longer generation emits
fluent text with zero relation to the prompt. At --temp 0 the garbage
differs run to run (Wikipedia-style prose, source code, other languages)
— i.e. the decode is nondeterministic, pointing at a race / uninitialised
memory in the attention path.

Repro (no server needed):
./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ctx 4096 --tokens 240
-p "Was ist 2+2? Antworte nur mit der Zahl."
-> instead of "4", unrelated fluent text; different each run at temp 0.

git bisect (good=4771329, bad=b0982a1):
first bad commit = b0c31af "Improve GLM 5.3 attention memory and batching"
Parent 4771329 is clean. The commit refactors shared attention/batching
in ds4.c (labelled GLM 5.3) but regresses GLM 5.2 Metal decode.

Log shows the compact-DSA-KV path:
"GLM graph using compact DSA KV only; expanded full-attention KV cache is skipped"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions