Skip to content

Bump llama.cpp to b10582 (e85caa81e) - #87

Merged
nyo16 merged 2 commits into
masterfrom
llama-cpp-b10582
Aug 22, 2026
Merged

Bump llama.cpp to b10582 (e85caa81e)#87
nyo16 merged 2 commits into
masterfrom
llama-cpp-b10582

Conversation

@nyo16

@nyo16 nyo16 commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Bumps vendor/llama.cpp from b10435 (9e40df63b) to
b10582 (e85caa81e), 147 commits. Makefile's LLAMA_COMMIT moves with it, so a Hex source build clones the same
tree a git checkout uses.

No API change reaches the binding

c_src/llama_cpp_ex/llama_nif.cpp is untouched, and that is a measurement, not an assumption. Every header the NIF includes is byte-identical across the range:

Header Diff
include/llama.h empty
ggml/include/ggml-backend.h empty
common/chat.h empty
common/json-schema-to-grammar.h empty
common/speculative.h empty
ggml/include/ggml-rpc.h RPC_PROTO_MINOR_VERSION 0 → 1

The RPC bump (5.0.0 → 5.1.0) is use_count propagation so backends can fuse ops behind RPC (af5172627). Same major, no signature change, and nothing in this repo pins the
protocol version.

llama_model_default_params() / llama_context_default_params() are unchanged, so no default shifted under a field we leave alone — the silent failure mode this checklist exists
for.

Two upstream MTP commits, both no-ops for us

  • f466cfa38 (spec : avoid binding reference to null pointer) skips a null dp.result when dp.drafting is false. The NIF sets dp.drafting = true on the seq immediately
    before every common_speculative_draft, so it never took that path.
  • 2c6b141ef (common : fix draft-mtp with embeddings) forces embedding = false / pooling_type = UNSPECIFIED inside common_base_params_to_speculative. The NIF does not call
    it — it builds target and draft contexts itself.

Upstream defects: all three still stand

Re-checked as a source diff, per docs/release-guide.md:

  1. ARM -mcpu=native probeggml/src/ggml-cpu/CMakeLists.txt did change, but only OpenMP target variables, KleidiAI SME2 GEMV sources, and IntelLLVM fast-math gating.
    Nothing near the probe, so LLAMA_CPU_ARM_ARCH stays.
  2. -sm tensor with a non-CUDA memberggml_backend_cuda_comm_init untouched. #26502 moved the tensor-split meta backend and was reverted in f20395dae.
  3. ggml_backend_rpc_start_server returns void — signature unchanged, so rpc_preflight_bind / rpc_wait_until_listening stay.

The guide's re-check note now records e85caa81e and, for the first time, a defect whose file did move — with why the movement is irrelevant.

Verified

macOS 25.5 / M1 Max, Metal, no RPC, at e85caa81e. Models from ~/Downloads, one run per tag as test/test_helper.exs documents:

Run Model(s) Result
mix test none 428 passed, 149 excluded
--include smoke --include embeddings Qwen3.5-0.8B-UD-Q4_K_XL, Qwen3-Embedding-0.6B-f16 548 passed
--include mtp Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL 3;176m 439 passed
--include mtp_sidecar Qwen3.8-27B-Q4_K_M + mtp-Qwen3.8-27B-Q4_0 434 passed156;163;176m
--include slow Qwen3.5-0.8B-UD-Q4_K_XL 438 passed

Plus mix format --check-formatted clean, mix dialyzer 0 errors, and the Hex source build path: mix hex.build → fresh extract → LLAMA_BACKEND=cpu mix compile clones and
links e85caa81e, matching the submodule — the check that catches a drifted LLAMA_COMMIT.

Not run: --include rpc_live (needs a reachable worker and an LLAMA_RPC=1 build) and the DGX Spark CUDA legs. The CUDA figure quoted in the changelog is still the b10435
measurement, attributed as such.

Docs fix picked up on the way

docs/release-guide.md step 3 told you to run the suite with LLAMA_MODEL_PATH / LLAMA_EMBEDDING_MODEL_PATH. Those were removed in v0.8.42, so following the guide literally runs
the suite with no model loaded at all and silently exercises none of the inference paths a bump most needs. Replaced with the LLAMA_SMOKE_* vars, one invocation per tag, and a
note on why GGML_METAL_NO_RESIDENCY=1 is there.

Not in this PR

No @version bump and no tag, matching how #86 shipped: the changelog entry lands under ## Unreleased and the release commit does the rest.

nyo16 added 2 commits August 22, 2026 10:01
147 commits past b10435. No API change reaches the binding: include/llama.h,
ggml-backend.h, common/chat.h, common/json-schema-to-grammar.h and
common/speculative.h are byte-identical across the range, so llama_nif.cpp is
untouched. ggml-rpc.h moves only RPC_PROTO_MINOR_VERSION 0 -> 1 (use_count
propagation, af5172627); same major, no signature change, and nothing here pins
the protocol version.

Two upstream commits land on the MTP path and both are no-ops for us: f466cfa38
skips a null dp.result when dp.drafting is false (the NIF sets drafting = true
immediately before every common_speculative_draft), and 2c6b141ef fixes
draft-mtp under --embeddings inside common_base_params_to_speculative, which the
NIF does not call -- it builds both contexts itself.

All three upstream defects in docs/release-guide.md still stand, re-checked as a
source diff: ggml_backend_rpc_start_server still returns void,
ggml_backend_cuda_comm_init untouched (the tensor-split meta backend change was
reverted in f20395dae), and the ggml-cpu CMake diff is OpenMP target variables,
KleidiAI SME2 GEMV sources and IntelLLVM fast-math gating -- nothing near the
-mcpu=native probe.

Verified on macOS (Metal, no RPC), models from ~/Downloads:
  mix test                                        428 passed, 149 excluded
  --include smoke --include embeddings            548 passed
  --include mtp (Qwen3.6-35B-A3B-MTP)             439 passed
  --include mtp_sidecar (Qwen3.8-27B + mtp head)  434 passed
  --include slow                                  438 passed
  mix format --check-formatted                    clean
  mix dialyzer                                    0 errors
  hex source build (LLAMA_BACKEND=cpu)            clones e85caa81e, links

docs/release-guide.md: the test commands named LLAMA_MODEL_PATH /
LLAMA_EMBEDDING_MODEL_PATH, which were removed in v0.8.42 -- following the guide
literally runs the suite with no model at all. Replaced with the LLAMA_SMOKE_*
vars and one invocation per tag.
Ran all seven default-excluded tags, not just the model ones.

Default build, macOS/Metal/M1 Max:
  mix test                                              428 passed, 149 excluded
  smoke + embeddings + slow + mtp (one run)              569 passed, 8 excluded
  mtp_sidecar (Qwen3.8-27B + mtp-*-Q4_0)                 434 passed
LLAMA_RPC=1 build, live local LlamaCppEx.RPC.Server on 127.0.0.1:50052:
  rpc_live                                              429 passed, 148 excluded
  smoke + embeddings + slow + mtp                        569 passed, 8 excluded
  mtp_sidecar                                            434 passed

1. :mtp_cancel no longer aborts the VM. Four runs per build, same model
(Qwen3.6-35B-A3B-MTP) and same machine: b10435 aborted 4/4 -- one exit 134 on
GGML_ASSERT(buf != NULL && "tensor buffer not set"), three exit 139 -- and
b10582 aborted 0/4, failing 3 with {:error, "prompt decode failed: code=-1"} /
"verify decode failed: code=-1" and passing 1. The race is unchanged and
unfixed; only its consequence moved, from killing the BEAM to returning an
error. The module comment claimed "does not fail, it aborts the VM", which is
now false, so it says what was measured instead. The tag stays: a flaky failure
still does not belong in a green run.

2. --include rpc_live must run alone. Combining it with the model tags aborted
the VM at ggml-rpc.cpp:576 "Remote RPC server crashed or returned malformed
response" inside llama_kv_cache's constructor -- with the worker still
listening afterwards. Cause: the live test's RPC.add_server/1 mutates the
process-global ggml device registry, and llama.cpp puts RPC devices at the front
of the placement list built from it, so a later test loading a model with
n_gpu_layers: -1 places layers and KV cache on the worker. A peer-side alloc
failure is RPC_STATUS_ASSERT, which is GGML_ABORT. Documented in
test/test_helper.exs beside the tag, and the release guide's test block now
covers rpc_live and mtp_cancel instead of stopping at mtp_sidecar.
@nyo16
nyo16 merged commit d5aad66 into master Aug 22, 2026
9 checks passed
@nyo16
nyo16 deleted the llama-cpp-b10582 branch August 22, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant