Bump llama.cpp to b10582 (e85caa81e) - #87
Merged
Merged
Conversation
147 commits past b10435. No API change reaches the binding: include/llama.h, ggml-backend.h, common/chat.h, common/json-schema-to-grammar.h and common/speculative.h are byte-identical across the range, so llama_nif.cpp is untouched. ggml-rpc.h moves only RPC_PROTO_MINOR_VERSION 0 -> 1 (use_count propagation, af5172627); same major, no signature change, and nothing here pins the protocol version. Two upstream commits land on the MTP path and both are no-ops for us: f466cfa38 skips a null dp.result when dp.drafting is false (the NIF sets drafting = true immediately before every common_speculative_draft), and 2c6b141ef fixes draft-mtp under --embeddings inside common_base_params_to_speculative, which the NIF does not call -- it builds both contexts itself. All three upstream defects in docs/release-guide.md still stand, re-checked as a source diff: ggml_backend_rpc_start_server still returns void, ggml_backend_cuda_comm_init untouched (the tensor-split meta backend change was reverted in f20395dae), and the ggml-cpu CMake diff is OpenMP target variables, KleidiAI SME2 GEMV sources and IntelLLVM fast-math gating -- nothing near the -mcpu=native probe. Verified on macOS (Metal, no RPC), models from ~/Downloads: mix test 428 passed, 149 excluded --include smoke --include embeddings 548 passed --include mtp (Qwen3.6-35B-A3B-MTP) 439 passed --include mtp_sidecar (Qwen3.8-27B + mtp head) 434 passed --include slow 438 passed mix format --check-formatted clean mix dialyzer 0 errors hex source build (LLAMA_BACKEND=cpu) clones e85caa81e, links docs/release-guide.md: the test commands named LLAMA_MODEL_PATH / LLAMA_EMBEDDING_MODEL_PATH, which were removed in v0.8.42 -- following the guide literally runs the suite with no model at all. Replaced with the LLAMA_SMOKE_* vars and one invocation per tag.
Ran all seven default-excluded tags, not just the model ones.
Default build, macOS/Metal/M1 Max:
mix test 428 passed, 149 excluded
smoke + embeddings + slow + mtp (one run) 569 passed, 8 excluded
mtp_sidecar (Qwen3.8-27B + mtp-*-Q4_0) 434 passed
LLAMA_RPC=1 build, live local LlamaCppEx.RPC.Server on 127.0.0.1:50052:
rpc_live 429 passed, 148 excluded
smoke + embeddings + slow + mtp 569 passed, 8 excluded
mtp_sidecar 434 passed
1. :mtp_cancel no longer aborts the VM. Four runs per build, same model
(Qwen3.6-35B-A3B-MTP) and same machine: b10435 aborted 4/4 -- one exit 134 on
GGML_ASSERT(buf != NULL && "tensor buffer not set"), three exit 139 -- and
b10582 aborted 0/4, failing 3 with {:error, "prompt decode failed: code=-1"} /
"verify decode failed: code=-1" and passing 1. The race is unchanged and
unfixed; only its consequence moved, from killing the BEAM to returning an
error. The module comment claimed "does not fail, it aborts the VM", which is
now false, so it says what was measured instead. The tag stays: a flaky failure
still does not belong in a green run.
2. --include rpc_live must run alone. Combining it with the model tags aborted
the VM at ggml-rpc.cpp:576 "Remote RPC server crashed or returned malformed
response" inside llama_kv_cache's constructor -- with the worker still
listening afterwards. Cause: the live test's RPC.add_server/1 mutates the
process-global ggml device registry, and llama.cpp puts RPC devices at the front
of the placement list built from it, so a later test loading a model with
n_gpu_layers: -1 places layers and KV cache on the worker. A peer-side alloc
failure is RPC_STATUS_ASSERT, which is GGML_ABORT. Documented in
test/test_helper.exs beside the tag, and the release guide's test block now
covers rpc_live and mtp_cancel instead of stopping at mtp_sidecar.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps
vendor/llama.cppfromb10435(9e40df63b) tob10582(e85caa81e), 147 commits.Makefile'sLLAMA_COMMITmoves with it, so a Hex source build clones the sametree a git checkout uses.
No API change reaches the binding
c_src/llama_cpp_ex/llama_nif.cppis untouched, and that is a measurement, not an assumption. Every header the NIF includes is byte-identical across the range:include/llama.hggml/include/ggml-backend.hcommon/chat.hcommon/json-schema-to-grammar.hcommon/speculative.hggml/include/ggml-rpc.hRPC_PROTO_MINOR_VERSION0 → 1The RPC bump (5.0.0 → 5.1.0) is
use_countpropagation so backends can fuse ops behind RPC (af5172627). Same major, no signature change, and nothing in this repo pins theprotocol version.
llama_model_default_params()/llama_context_default_params()are unchanged, so no default shifted under a field we leave alone — the silent failure mode this checklist existsfor.
Two upstream MTP commits, both no-ops for us
f466cfa38(spec : avoid binding reference to null pointer) skips a nulldp.resultwhendp.draftingis false. The NIF setsdp.drafting = trueon the seq immediatelybefore every
common_speculative_draft, so it never took that path.2c6b141ef(common : fix draft-mtp with embeddings) forcesembedding = false/pooling_type = UNSPECIFIEDinsidecommon_base_params_to_speculative. The NIF does not callit — it builds target and draft contexts itself.
Upstream defects: all three still stand
Re-checked as a source diff, per
docs/release-guide.md:-mcpu=nativeprobe —ggml/src/ggml-cpu/CMakeLists.txtdid change, but only OpenMP target variables, KleidiAI SME2 GEMV sources, and IntelLLVM fast-math gating.Nothing near the probe, so
LLAMA_CPU_ARM_ARCHstays.-sm tensorwith a non-CUDA member —ggml_backend_cuda_comm_inituntouched. #26502 moved the tensor-split meta backend and was reverted inf20395dae.ggml_backend_rpc_start_serverreturnsvoid— signature unchanged, sorpc_preflight_bind/rpc_wait_until_listeningstay.The guide's re-check note now records
e85caa81eand, for the first time, a defect whose file did move — with why the movement is irrelevant.Verified
macOS 25.5 / M1 Max, Metal, no RPC, at
e85caa81e. Models from~/Downloads, one run per tag astest/test_helper.exsdocuments:mix test--include smoke --include embeddings--include mtp--include mtp_sidecarmtp-Qwen3.8-27B-Q4_0--include slowPlus
mix format --check-formattedclean,mix dialyzer0 errors, and the Hex source build path:mix hex.build→ fresh extract →LLAMA_BACKEND=cpu mix compileclones andlinks
e85caa81e, matching the submodule — the check that catches a driftedLLAMA_COMMIT.Not run:
--include rpc_live(needs a reachable worker and anLLAMA_RPC=1build) and the DGX Spark CUDA legs. The CUDA figure quoted in the changelog is still the b10435measurement, attributed as such.
Docs fix picked up on the way
docs/release-guide.mdstep 3 told you to run the suite withLLAMA_MODEL_PATH/LLAMA_EMBEDDING_MODEL_PATH. Those were removed in v0.8.42, so following the guide literally runsthe suite with no model loaded at all and silently exercises none of the inference paths a bump most needs. Replaced with the
LLAMA_SMOKE_*vars, one invocation per tag, and anote on why
GGML_METAL_NO_RESIDENCY=1is there.Not in this PR
No
@versionbump and no tag, matching how #86 shipped: the changelog entry lands under## Unreleasedand the release commit does the rest.