Add vision support to LlamaLanguageModel via mtmd

[james-333i]

Image segments threw unsupportedFeature because the backend had no multimodal path, even though the prebuilt llama.cpp binaries ship the mtmd library. This accepts an mmprojPath at initialization, replaces image segments with the mtmd media marker during prompt formatting, and evaluates text and image chunks through mtmd_helper_eval_chunks before sampling continues. Both respond and streaming support images, and models without a projector keep rejecting image input. Stacks on #195; the squash-merge will sort out the shared commit.

[mattt]

Hi @james-333i. Thanks for these four — I read the whole stack today. #195 is in, so this needs a rebase first (main moved in LlamaLanguageModel.swift with #205). Notes per PR, all small except one:

• Add vision support to LlamaLanguageModel via mtmd #213: would you honor the threads option in mtmdParams.n_threads, and leave a comment that the mtmd_helper_bitmap_init_from_buf signature is pinned to the current llama.swift (master already adds an options argument)?
• Render the Gemma 4 chat format when template detection fails #214: renderGemma4Prompt can't be unit-tested as a private instance method. Could you make it static (or internal) and add a test that renders a short transcript?
• Reuse a per-session context across chat exchanges #215: this is the one I want to talk through. LlamaLanguageModel is @unchecked Sendable with no lock, and acquireSessionContext frees the cached context when a different session comes in. Two sessions sharing one model could free a context the other is still decoding on, where before each generation owned its own. Either a lock around the cache or a documented one-generation-at-a-time rule would do; which would you prefer? Also worth a line in the clearCachedContext() docs that the context (and its KV) now lives as long as the model.
• Add tool calling to LlamaLanguageModel, including streaming #216: this bundles @noorbhatia's Support tool calling in MLXLanguageModel.streamResponse() #181, so it waits on the Make LanguageModelSession transcript observable while streaming responses #103 decision. Two things on your side meanwhile: the Linux legs fail on CFGetTypeID/CFBooleanGetTypeID in LlamaToolCallFormat.swift (not in swift-corelibs-foundation), and the last streamed snapshot can shrink (the .stop path yields emittedBase + roundVisible, then the post-loop yield goes back to emittedBase). Stopping at the first </tool_call> also means only one call per round; if that's intentional, say so in the docs.

The LlamaToolCallFormat tests are great — thank you for making that part pure.

[james-333i]

@mattt Thanks for the thorough review. All four are rebased onto main and updated.

#213: n_threads now follows the model's threads setting, and the bitmap helper call carries a note that it is pinned to the current llama.swift signature.

#214: renderGemma4Prompt is static and internal now, with a small suite rendering a short transcript and the assistant prefill.

#215: I went with the lock, but with checkout semantics rather than a bare mutex, since a lock around the accessor alone would not protect a context for the length of a generation. A generation checks the cached context out for its whole run. A concurrent generation for another session gets a transient context that is never cached, so nothing can free a context that is still decoding. clearCachedContext() during a run marks the context to be freed on release. The docs now say the context and its KV live as long as the model.

#216: the boolean check is Darwin-conditional with an objCType fallback for corelibs-foundation, and the last snapshot no longer shrinks: the round's visible text is folded into the base before the .stop yield. One call per round is intentional and now documented on callTerminator. Models that want several calls issue them across consecutive rounds. Happy to widen it to multiple calls per round as a follow-up if you would rather have that.