Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

The easiest way to use Ollama in .NET

✨ Kubectl plugin to create manifests with LLMs

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.

Social and customizable AI writing assistant! ✍️

Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with multi-GPU tensor-split, MoE expert placement, measured flag tuning (AI Tune), hardware-matched HuggingFace downloads, and crash recovery. An Ollama alternative for multi-GPU rigs.

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

A local and uncensored AI entity.

LLM RAG Application with Cross-Encoders Re-ranking for YouTube video 🎥

Fast LLM swapping with sleep/wake support, compatible with vllm, llama.cpp, etc. llama-swap fork.

Benchmark system for testing opencode with various LLM models, measuring speed (latency) and correctness (accuracy).

Lightweight Python tool using Optuna for tuning llama.cpp flags: towards optimal tok/s for your machine

User friendly GUI for Llama.cpp for easy configuration and launching.

Run Pi coding agent isolated in a Docker Sandbox microVM with a local llama-server as the inference backend

Privacy-first desktop AI companion with 5 pillars: unified chat, OpenClaw Gateway, evolving avatar, desktop control, and vision. Auth0 with encrypted tunneling.

Rust SDK for building AI agents with local OpenAI-compatible servers (LMStudio, Ollama, llama.cpp, vLLM). Features streaming, tools, hooks, retry logic, and comprehensive examples.

Run gguf LLM models in Latest Version TextGen-webui and koboldcpp