diff --git a/.mintignore b/.mintignore index f4d7f21d..27ad499e 100644 --- a/.mintignore +++ b/.mintignore @@ -1,2 +1,4 @@ MODEL-MATRIX.md DESIGN.md +CLAUDE.md +INTERACTIVE_QUICKSTART_WIDGET.md diff --git a/.mintlify/Assistant.md b/.mintlify/Assistant.md new file mode 100644 index 00000000..38839a88 --- /dev/null +++ b/.mintlify/Assistant.md @@ -0,0 +1,27 @@ +# Liquid Docs assistant instructions + +## Speed, memory, and hardware + +The docs don't list measured speed, RAM use, or download size for every model and device. When someone asks how fast a model is, how much memory it needs, how big the download is, or whether it runs on a specific device (phone, Raspberry Pi, Jetson, laptop, GPU): + +- Give any figure that the model's own page states, and cite that page. +- Point them to [Pipette](https://pipette.liquid.ai), Liquid AI's benchmarking site with speed and memory measured on real devices. +- For download size, start with the model's documentation page and follow its exact Hugging Face link for the requested format and precision (HF, GGUF, MLX, ONNX, or another listed format). Preserve the repository owner and name; conversions may be published by `mlx-community` or `onnx-community`, not only `LiquidAI`. Don't construct repository URLs from the model name. +- Link the selected repository's Files tab. Quote a size only if it is stated in the docs or verified in that file listing. If the format or precision is unspecified, explain that download size depends on those choices and link the available formats. +- Count the selected weights and any required shards, projectors, codecs, or other companion files described by the model's setup instructions. A repository's total size may include several alternative quantizations; it is not the download size for one choice. Download size is not runtime RAM usage. + +Don't estimate tokens per second or RAM use yourself. + +## Benchmark scores and model comparisons + +When someone asks for benchmark results, quality scores, or a comparison with other models: + +- Start with the model's documentation page and cite any published evaluation results it provides. +- Follow the exact Hugging Face model-card links or evaluation sources linked from that page. Don't construct repository URLs from the model name. +- Don't assume every model has a Hugging Face repository or published benchmark scores. For API-only models such as d1-ultra and d1.1, use published evaluation sources linked from their documentation when available. If no published results are available in those sources, say so. +- Don't make up or estimate scores. + +## Questions about this assistant + +If someone asks which model powers you, say you're the Liquid Docs assistant, provided by Mintlify, and that you don't run on a Liquid Foundation Model. +To try LFMs, point them to the [Liquid Playground](https://playground.liquid.ai) to chat with the models in the browser, the [Liquid API quickstart](/api/quickstart). diff --git a/deployment/on-device/leap-sdk-changelog.mdx b/deployment/on-device/leap-sdk-changelog.mdx index 09a259b9..9081e4fd 100644 --- a/deployment/on-device/leap-sdk-changelog.mdx +++ b/deployment/on-device/leap-sdk-changelog.mdx @@ -1,7 +1,6 @@ --- title: "Changelog" description: "Release notes for the LEAP SDK, including the 0.9.x → 0.10.x Kotlin Multiplatform transition." -noindex: true --- diff --git a/deployment/on-device/ollama.mdx b/deployment/on-device/ollama.mdx index 766dad5c..724b49a9 100644 --- a/deployment/on-device/ollama.mdx +++ b/deployment/on-device/ollama.mdx @@ -10,7 +10,7 @@ description: "Deploy Liquid AI LFM models with Ollama. Pull, configure, and serv Ollama uses GGUF models and supports GPU acceleration (CUDA, Metal, ROCm). - The official Ollama v0.17.0 (latest stable) from [ollama.com](https://ollama.com) fails with a `missing tensor 'output_norm.weight'` error on the `lfm2moe` architecture. This affects all LFM MoE models (e.g. LFM2-24B-A2B, LFM2-8A-A1B). To run any LFM MoE model you specifically need [v0.17.1-rc0](https://github.com/ollama/ollama/releases/tag/v0.17.1-rc0) or later. + The official Ollama v0.17.0 (latest stable) from [ollama.com](https://ollama.com) fails with a `missing tensor 'output_norm.weight'` error on the `lfm2moe` architecture. This affects all LFM MoE models (e.g. LFM2-24B-A2B, LFM2-8B-A1B). To run any LFM MoE model you specifically need [v0.17.1-rc0](https://github.com/ollama/ollama/releases/tag/v0.17.1-rc0) or later.
diff --git a/deployment/on-device/sdk/advanced-features.mdx b/deployment/on-device/sdk/advanced-features.mdx index 73d10a3c..5482647a 100644 --- a/deployment/on-device/sdk/advanced-features.mdx +++ b/deployment/on-device/sdk/advanced-features.mdx @@ -1,7 +1,6 @@ --- title: "Advanced Features" description: "GenerationOptions, JSONSchemaGenerator, function-calling type references — same surface everywhere." -noindex: true --- diff --git a/deployment/on-device/sdk/ai-agent-usage-guide.mdx b/deployment/on-device/sdk/ai-agent-usage-guide.mdx index a2060d00..524ec225 100644 --- a/deployment/on-device/sdk/ai-agent-usage-guide.mdx +++ b/deployment/on-device/sdk/ai-agent-usage-guide.mdx @@ -1,7 +1,6 @@ --- title: "AI Agent Usage Guide" description: "End-to-end recipes for building AI agents with the LEAP SDK — same patterns across iOS, macOS, Android, JVM, and native." -noindex: true --- diff --git a/deployment/on-device/sdk/cloud-ai-comparison.mdx b/deployment/on-device/sdk/cloud-ai-comparison.mdx index 5684f823..b6e94a6b 100644 --- a/deployment/on-device/sdk/cloud-ai-comparison.mdx +++ b/deployment/on-device/sdk/cloud-ai-comparison.mdx @@ -1,7 +1,6 @@ --- title: "Cloud AI Comparison" description: "Mapping LEAP SDK concepts to cloud chat-completion APIs like OpenAI." -noindex: true --- diff --git a/deployment/on-device/sdk/constrained-generation.mdx b/deployment/on-device/sdk/constrained-generation.mdx index a39fa3fe..63bde8c2 100644 --- a/deployment/on-device/sdk/constrained-generation.mdx +++ b/deployment/on-device/sdk/constrained-generation.mdx @@ -1,7 +1,6 @@ --- title: "Constrained Generation" description: "Generate structured JSON output with compile-time validation — same approach on every platform." -noindex: true --- diff --git a/deployment/on-device/sdk/conversation-generation.mdx b/deployment/on-device/sdk/conversation-generation.mdx index aac8ad96..b2d2b2fa 100644 --- a/deployment/on-device/sdk/conversation-generation.mdx +++ b/deployment/on-device/sdk/conversation-generation.mdx @@ -1,7 +1,6 @@ --- title: "Conversation & Generation" description: "Reference for ModelRunner, Conversation, MessageResponse, and GenerationOptions — same API on every platform." -noindex: true --- diff --git a/deployment/on-device/sdk/desktop-platforms.mdx b/deployment/on-device/sdk/desktop-platforms.mdx index 01e96219..6ad20f39 100644 --- a/deployment/on-device/sdk/desktop-platforms.mdx +++ b/deployment/on-device/sdk/desktop-platforms.mdx @@ -1,7 +1,6 @@ --- title: "Desktop & Native Platforms" description: "Run the LEAP SDK on JVM desktop, native Linux, native Windows, and macOS — same API as Android and iOS." -noindex: true --- diff --git a/deployment/on-device/sdk/function-calling.mdx b/deployment/on-device/sdk/function-calling.mdx index ba5e5d2b..9e1656f5 100644 --- a/deployment/on-device/sdk/function-calling.mdx +++ b/deployment/on-device/sdk/function-calling.mdx @@ -1,7 +1,6 @@ --- title: "Function Calling" description: "Tool use with LeapFunction — same API on every platform, with Hermes and Pythonic parsers." -noindex: true --- diff --git a/deployment/on-device/sdk/messages-content.mdx b/deployment/on-device/sdk/messages-content.mdx index 34547e07..30d5500a 100644 --- a/deployment/on-device/sdk/messages-content.mdx +++ b/deployment/on-device/sdk/messages-content.mdx @@ -1,7 +1,6 @@ --- title: "Messages & Content" description: "ChatMessage, ChatMessageContent, audio format requirements — same shape on every platform." -noindex: true --- diff --git a/deployment/on-device/sdk/model-loading.mdx b/deployment/on-device/sdk/model-loading.mdx index 956f3786..122d24e1 100644 --- a/deployment/on-device/sdk/model-loading.mdx +++ b/deployment/on-device/sdk/model-loading.mdx @@ -1,7 +1,6 @@ --- title: "Model Loading" description: "Reference for ModelDownloader, LeapDownloader, loadModel, loadSimpleModel, and KV cache reuse." -noindex: true --- diff --git a/deployment/on-device/sdk/openai-client.mdx b/deployment/on-device/sdk/openai-client.mdx index e50be65f..1936aff5 100644 --- a/deployment/on-device/sdk/openai-client.mdx +++ b/deployment/on-device/sdk/openai-client.mdx @@ -1,7 +1,6 @@ --- title: "OpenAI-Compatible Client" description: "Lightweight client for OpenAI-compatible chat completions APIs — ideal for hybrid on-device + cloud routing." -noindex: true --- diff --git a/deployment/on-device/sdk/overview.mdx b/deployment/on-device/sdk/overview.mdx index fdbc29a4..3cff5cb2 100644 --- a/deployment/on-device/sdk/overview.mdx +++ b/deployment/on-device/sdk/overview.mdx @@ -1,7 +1,6 @@ --- title: "LEAP SDK (archived)" description: "The LEAP SDK is deprecated. This archived reference is kept for teams still shipping it; new projects should use llama.cpp directly." -noindex: true --- diff --git a/deployment/on-device/sdk/quick-start.mdx b/deployment/on-device/sdk/quick-start.mdx index 60640ed1..c96d3e6b 100644 --- a/deployment/on-device/sdk/quick-start.mdx +++ b/deployment/on-device/sdk/quick-start.mdx @@ -1,7 +1,6 @@ --- title: "Quick Start" description: "Install the LEAP SDK on iOS, macOS, Android, JVM, Linux, or Windows — same API everywhere." -noindex: true --- diff --git a/deployment/on-device/sdk/utilities.mdx b/deployment/on-device/sdk/utilities.mdx index d3352828..b0d78c48 100644 --- a/deployment/on-device/sdk/utilities.mdx +++ b/deployment/on-device/sdk/utilities.mdx @@ -1,7 +1,6 @@ --- title: "Utilities" description: "Error handling, serialization, Android-specific downloader internals, and a putting-it-together example." -noindex: true --- diff --git a/deployment/on-device/sdk/voice-assistant.mdx b/deployment/on-device/sdk/voice-assistant.mdx index bec83e02..293e305f 100644 --- a/deployment/on-device/sdk/voice-assistant.mdx +++ b/deployment/on-device/sdk/voice-assistant.mdx @@ -1,7 +1,6 @@ --- title: "Voice Assistant Widget" description: "Drop-in Compose Multiplatform voice UI — runs on iOS, macOS, Android, and JVM Desktop." -noindex: true --- diff --git a/deployment/tools/model-bundling/authentication.mdx b/deployment/tools/model-bundling/authentication.mdx index 9f97f180..5ec6a1c6 100644 --- a/deployment/tools/model-bundling/authentication.mdx +++ b/deployment/tools/model-bundling/authentication.mdx @@ -1,9 +1,12 @@ --- title: "Authentication" description: "Authentication commands for the LEAP Bundle CLI" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## `login` Authenticate with the LEAP platform using your API token. diff --git a/deployment/tools/model-bundling/bundle-creation.mdx b/deployment/tools/model-bundling/bundle-creation.mdx index 5029b5b0..01e871f4 100644 --- a/deployment/tools/model-bundling/bundle-creation.mdx +++ b/deployment/tools/model-bundling/bundle-creation.mdx @@ -1,9 +1,12 @@ --- title: "Bundle Creation" description: "Commands for creating and validating bundle requests in the LEAP Bundle CLI" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## `create` Create a new bundle request by uploading a model directory. diff --git a/deployment/tools/model-bundling/bundle-management.mdx b/deployment/tools/model-bundling/bundle-management.mdx index 35a69fc5..0efa897c 100644 --- a/deployment/tools/model-bundling/bundle-management.mdx +++ b/deployment/tools/model-bundling/bundle-management.mdx @@ -1,9 +1,12 @@ --- title: "Bundle Management" description: "Commands for listing and canceling bundle requests in the LEAP Bundle CLI" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## `list` List bundle requests or get details for a specific request. diff --git a/deployment/tools/model-bundling/changelog.mdx b/deployment/tools/model-bundling/changelog.mdx index 215e2030..dacaa1a1 100644 --- a/deployment/tools/model-bundling/changelog.mdx +++ b/deployment/tools/model-bundling/changelog.mdx @@ -2,9 +2,12 @@ title: "LEAP Bundle CLI Changelog" sidebarTitle: "Changelog" description: "Release history for leap-bundle, the Liquid AI model bundling CLI tool." -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## `v0.9.0` - 2026-01-02 **New features** diff --git a/deployment/tools/model-bundling/configuration.mdx b/deployment/tools/model-bundling/configuration.mdx index 5e2c6437..336be0c1 100644 --- a/deployment/tools/model-bundling/configuration.mdx +++ b/deployment/tools/model-bundling/configuration.mdx @@ -1,9 +1,12 @@ --- title: "Configuration" description: "Configuration commands and file format for the LEAP Bundle CLI" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## `config` View CLI configuration settings. diff --git a/deployment/tools/model-bundling/data-privacy.mdx b/deployment/tools/model-bundling/data-privacy.mdx index 65be8c2e..e37bd289 100644 --- a/deployment/tools/model-bundling/data-privacy.mdx +++ b/deployment/tools/model-bundling/data-privacy.mdx @@ -2,9 +2,12 @@ title: "Data Privacy" description: "This page outlines how the LEAP Model Bundling Service handles your data, what information we collect and delete." sidebar_position: 3 -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ### Bundle Request Information When you create a bundle request using `leap-bundle create`, we store the following information: diff --git a/deployment/tools/model-bundling/download.mdx b/deployment/tools/model-bundling/download.mdx index 16f21fa6..231dc1ba 100644 --- a/deployment/tools/model-bundling/download.mdx +++ b/deployment/tools/model-bundling/download.mdx @@ -1,9 +1,12 @@ --- title: "Download" description: "Download commands for bundle requests and LEAP models in the LEAP Bundle CLI" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + The `download` command supports two modes of operation: diff --git a/deployment/tools/model-bundling/quick-start.mdx b/deployment/tools/model-bundling/quick-start.mdx index 5861c3f5..d49534ae 100644 --- a/deployment/tools/model-bundling/quick-start.mdx +++ b/deployment/tools/model-bundling/quick-start.mdx @@ -2,7 +2,6 @@ title: "LEAP Bundle CLI Quick Start" sidebarTitle: "Quick Start" description: "The Bundling Service helps users create and manage model bundles for Liquid Edge AI Platform (LEAP). Currently users interact with it through a command-line interface (CLI)." -noindex: true --- diff --git a/deployment/tools/model-bundling/reference.mdx b/deployment/tools/model-bundling/reference.mdx index aa3afe0f..0f5ba5af 100644 --- a/deployment/tools/model-bundling/reference.mdx +++ b/deployment/tools/model-bundling/reference.mdx @@ -1,9 +1,12 @@ --- title: "Reference" description: "Reference information for the LEAP Bundle CLI including limitations, error handling, and exit codes" -noindex: true --- + + The LEAP Model Bundling Service is deprecated and no longer maintained. For new deployments, download GGUF files from [Hugging Face](https://huggingface.co/LiquidAI) and run them with [llama.cpp](/deployment/on-device/llama-cpp). + + ## All Commands | Command | Description | diff --git a/docs.json b/docs.json index 4f1a6907..317a3d4e 100644 --- a/docs.json +++ b/docs.json @@ -11,6 +11,7 @@ "name": "Liquid AI Docs", "description": "Documentation for Liquid AI Foundation Models (LFM)", "seo": { + "indexing": "all", "organization": { "name": "Liquid AI", "url": "https://www.liquid.ai", @@ -619,6 +620,38 @@ { "source": "/lfm/fine-tuning", "destination": "/lfm/fine-tuning/overview" + }, + { + "source": "/lfm/getting-started/intro", + "destination": "/lfm/getting-started/welcome" + }, + { + "source": "/lfm/inference/lmstudio", + "destination": "/deployment/on-device/lm-studio" + }, + { + "source": "/lfm/key-concepts/models", + "destination": "/lfm/models/complete-library" + }, + { + "source": "/deployment/tools/model-bundling", + "destination": "/deployment/on-device/llama-cpp" + }, + { + "source": "/lfm/models/lfm3-1-7b", + "destination": "/lfm/models/lfm3-1.7b" + }, + { + "source": "/lfm/getting-started/text-generation-and-prompting", + "destination": "/lfm/key-concepts/text-generation-and-prompting" + }, + { + "source": "/lfm/frameworks/leap", + "destination": "/deployment/on-device/llama-cpp/migrating-from-leap-sdk" + }, + { + "source": "/lfm/getting-started/vision", + "destination": "/lfm/key-concepts/vision-capabilities" } ], "ai": { diff --git a/examples/laptop-examples/audio-car-cockpit.mdx b/examples/laptop-examples/audio-car-cockpit.mdx index b8519eb6..811ab3fb 100644 --- a/examples/laptop-examples/audio-car-cockpit.mdx +++ b/examples/laptop-examples/audio-car-cockpit.mdx @@ -7,6 +7,10 @@ description: "Voice-controlled car cockpit demo combining LFM2.5-Audio for speec Browse the complete example on GitHub + +This demo uses LFM2.5-Audio-1.5B and LFM2-1.2B-Tool, both of which are deprecated. For new projects, see [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) for speech and [LFM2.5-1.2B-Instruct](/lfm/models/lfm25-1.2b-instruct) for tool calling. The demo and commands below use the original models. + + ## What's inside? Combines [LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B-GGUF) in TTS and STT modes with [LFM2-1.2B-Tool](https://huggingface.co/LiquidAI/LFM2-1.2B-Tool-GGUF) within a mockup of a car cockpit, letting the user control the car functionalities by voice. diff --git a/examples/laptop-examples/audio-to-text-in-real-time.mdx b/examples/laptop-examples/audio-to-text-in-real-time.mdx index eae02332..783b5db2 100644 --- a/examples/laptop-examples/audio-to-text-in-real-time.mdx +++ b/examples/laptop-examples/audio-to-text-in-real-time.mdx @@ -7,7 +7,11 @@ description: "Real-time audio transcription using LFM2.5-Audio-1.5B with llama.c Browse the complete example on GitHub -This example demonstrates how to use the [LFM2.5-Audio-1.5B](https://docs.liquid.ai/lfm/models/lfm25-audio-1.5b) model with llama.cpp to transcribe audio files locally in real-time. + +This example uses LFM2.5-Audio-1.5B, which is deprecated. For new projects, use [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b). + + +This example demonstrates how to use the [LFM2.5-Audio-1.5B](/lfm/models/lfm25-audio-1.5b) model with llama.cpp to transcribe audio files locally in real-time. Intelligent audio assistants on the edge are possible, and this repository is just one step towards that. @@ -135,7 +139,7 @@ For example, we can use ### What is LFM2.5-350M? -[LFM2.5-350M](https://docs.liquid.ai/lfm/models/lfm25-350m) is a small text-to-text model that can be used for tasks like text cleaning. To achieve optimal performance for your particular use case, you need to optimize your system and user prompts. +[LFM2.5-350M](/lfm/models/lfm25-350m) is a small text-to-text model that can be used for tasks like text cleaning. To achieve optimal performance for your particular use case, you need to optimize your system and user prompts. ## Need help? diff --git a/examples/laptop-examples/invoice-extractor-tool-with-liquid-nanos.mdx b/examples/laptop-examples/invoice-extractor-tool-with-liquid-nanos.mdx index d0c51d07..82e925b5 100644 --- a/examples/laptop-examples/invoice-extractor-tool-with-liquid-nanos.mdx +++ b/examples/laptop-examples/invoice-extractor-tool-with-liquid-nanos.mdx @@ -7,6 +7,10 @@ description: "Extract structured payment details from invoice PDFs using LFM2.5- Browse the complete example on GitHub + +**Newer model available:** This example uses LFM2.5-VL-1.6B. For new projects, see [LFM3-VL-2B](/lfm/models/lfm3-vl-2b). + + A Python CLI that extracts payment details from invoice PDFs. ![Invoice extraction: a utility bill image goes through LFM2.5-VL-1.6B on llama.cpp and comes out as structured JSON](https://github.com/Liquid4All/cookbook/raw/main/examples/invoice-parser/media/diagram.jpg) @@ -23,7 +27,7 @@ In this example, you will learn how to: * **Set up local AI inference** using llama.cpp to run Liquid models entirely on your machine without requiring cloud services or API keys * **Build a file monitoring system** that automatically processes new files dropped into a directory -* **Extract structured output from images** using [LFM2.5-VL-1.6B](https://docs.liquid.ai/lfm/models/lfm25-vl-1.6b), a small vision-language model. +* **Extract structured output from images** using [LFM2.5-VL-1.6B](/lfm/models/lfm25-vl-1.6b), a small vision-language model. ## Environment setup diff --git a/examples/web/audio-webgpu-demo.mdx b/examples/web/audio-webgpu-demo.mdx index a12bc98c..9cf82a59 100644 --- a/examples/web/audio-webgpu-demo.mdx +++ b/examples/web/audio-webgpu-demo.mdx @@ -7,6 +7,10 @@ description: "Run LFM2.5-Audio-1.5B in the browser with WebGPU for speech recogn Browse the complete example on Hugging Face + +This demo uses LFM2.5-Audio-1.5B, which is deprecated. For new projects, see [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b). The instructions below use the original model. + + This example showcases the **LFM2.5-Audio-1.5B** model running entirely within a web browser using WebGPU and ONNX Runtime Web technology. The demo provides three powerful audio processing modes: automatic speech recognition, text-to-speech synthesis, and interleaved conversations. ## What's Inside? diff --git a/examples/web/hand-voice-racer.mdx b/examples/web/hand-voice-racer.mdx index 5aaac370..7a21f84f 100644 --- a/examples/web/hand-voice-racer.mdx +++ b/examples/web/hand-voice-racer.mdx @@ -15,6 +15,10 @@ description: "Browser-based driving game controlled by hand gestures and voice c allowFullScreen > + +This example uses LFM2.5-Audio-1.5B, which is deprecated. For new projects, use [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b). + + **A browser driving game you control with your hands and voice, powered by models running fully local.** Steer by holding both hands up like a steering wheel. Speak commands to accelerate, brake, toggle headlights, and play music. No cloud calls, no server round-trips. Everything runs in your browser tab. @@ -24,7 +28,7 @@ Steer by holding both hands up like a steering wheel. Speak commands to accelera Two models run in parallel, entirely client-side: - **[MediaPipe Hand Landmarker](https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker)** tracks your hand positions via webcam at ~30 fps. The angle between your two wrists drives the steering. -- **[LFM2.5-Audio-1.5B](https://docs.liquid.ai/lfm/models/lfm25-audio-1.5b)** runs in a Web Worker with ONNX Runtime Web. It listens for speech via the [Silero VAD](https://github.com/snakers4/silero-vad) and transcribes each utterance on-device. Matched keywords control game state. +- **[LFM2.5-Audio-1.5B](/lfm/models/lfm25-audio-1.5b)** runs in a Web Worker with ONNX Runtime Web. It listens for speech via the [Silero VAD](https://github.com/snakers4/silero-vad) and transcribes each utterance on-device. Matched keywords control game state. The audio model loads from Hugging Face and is cached in IndexedDB after the first run, so subsequent starts are instant. diff --git a/guides/migration-guide.mdx b/guides/migration-guide.mdx index 63d7fb24..814320f8 100644 --- a/guides/migration-guide.mdx +++ b/guides/migration-guide.mdx @@ -3,7 +3,6 @@ title: "Migration Guide" description: "Move from Qwen, Llama, or Gemma to LFMs while avoiding the common migration pitfalls." --- -import { ChatMLExample } from "/snippets/key-concepts/chatml-example.mdx"; import { ToolCallTokenExample } from "/snippets/key-concepts/tool-call-token-example.mdx"; This guide is for teams already running Qwen, Llama, or Gemma who want to evaluate Liquid Foundation Models (LFMs) as replacements. LFMs load and serve through the same frameworks you already use, so most migrations are a model ID and configuration change. The details that usually matter are: @@ -17,22 +16,25 @@ Use this alongside [Use Case Evaluation](/guides/use-case-evaluation) for model ## Model mapping -Pick the LFM in the same deployment class as your current model. Compare within class; benchmarking a 1.2B LFM against a 32B Qwen-family model does not tell you whether it fits your deployment. +Start with LFM3 for general text, vision, audio, or omni workloads. Compare models within your deployment's memory, latency, and quality constraints. Smaller and specialized older models remain useful when they fit those constraints better. | If you run | Evaluate | Notes | | --- | --- | --- | -| Small text models, `<1B` | [LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) or [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) | Classification, extraction, routing, and tight memory budgets. | -| Mid-size text models, `1B-4B` | [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) or [LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) | Default migration range; choose the Thinking variant when reasoning quality is worth the added latency. | -| Large text models, `4B-13B` | [LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B) | Best fit for stronger chat, tool use, and agentic workloads; 1.5B active parameters. | -| Extra-large text models, `14B+` | [LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B) | Highest-capability LFM option for larger-model replacement evaluations. | -| Vision-language models | [LFM2.5-VL-450M](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) or [LFM2.5-VL-1.6B](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B) | Image understanding; use `-Extract` variants for image to strict JSON. | -| Whisper plus separate TTS | [LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) | ASR, TTS, and speech-to-speech in one model. | +| General text models, `1B-4B` | [LFM3-1.7B](/lfm/models/lfm3-1.7b) | Starting point for chat, local agents, tool use, and structured output; 256K context. | +| Vision-language models | [LFM3-VL-2B](/lfm/models/lfm3-vl-2b) | Image and video understanding, visual reasoning, and OCR. | +| Whisper plus separate TTS | [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) | ASR, TTS, and speech-to-speech in one model. | +| Separate vision and speech models | [LFM3-Omni-2B](/lfm/models/lfm3-omni-2b) | Text, image, video, and audio input with text or speech output. | +| Small text models, `<1B` | [LFM2.5-230M](/lfm/models/lfm25-230m) or [LFM2.5-350M](/lfm/models/lfm25-350m) | Classification, extraction, routing, and tight memory budgets. | +| Larger text models, `4B-13B` | [LFM2.5-8B-A1B](/lfm/models/lfm25-8b-a1b) | MoE alternative for reasoning and agentic workloads; 1.5B active parameters. | +| Larger text models, `14B+` | [LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b) | Larger reasoning MoE alternative; evaluate quality, latency, and memory on your task. | + +For image-to-JSON extraction, also evaluate the specialized [LFM2.5-VL-Extract models](/lfm/models/liquid-nanos). See the [Complete Model Library](/lfm/models/complete-library) for the full catalog. ## Deployment runtimes -Across runtimes, avoid carrying over hand-written Qwen, Llama, or Gemma prompt templates. Use the model-provided chat template, and configure the `lfm2` tool-call parser in vLLM or SGLang when tool use is in scope. +Across runtimes, avoid carrying over hand-written Qwen, Llama, or Gemma prompt templates. Use the model-provided chat template and a runtime build that supports your checkpoint. For tool use, LFM3 uses `qwen3_coder` parsing in vLLM and SGLang; LFM2 and LFM2.5 use `lfm2`. Follow the selected model's quickstart for the full launch configuration. For GPU inference, see [Transformers](/deployment/gpu-inference/transformers), [vLLM](/deployment/gpu-inference/vllm), [SGLang](/deployment/gpu-inference/sglang), or [zml/llmd](/deployment/gpu-inference/zml-llmd). @@ -40,11 +42,9 @@ For edge and on-device inference, see [llama.cpp](/deployment/on-device/llama-cp ## Chat Template -LFMs use a ChatML-style format: - - +LFMs use a ChatML-style format, with generation-specific tokens and reasoning behavior. LFM3 templates normally open a thinking-model assistant turn with ``; use the tokenizer or processor to format the prompt. -Roles are `system`, `user`, `assistant`, and `tool`. Vision-language models use an `` sentinel. +Roles are `system`, `user`, `assistant`, and `tool`. Image and video markers differ by model generation; let the processor insert them. Migration notes: @@ -62,33 +62,45 @@ See the [Prompting Guide](/lfm/key-concepts/text-generation-and-prompting) for r ## Tool Use -LFMs natively emit a Pythonic tool-call list, not OpenAI-style JSON: +LFM3 emits XML tool calls, using a superset of the `qwen3_coder` format: + +```text + + + +12345 + + + +``` + +LFM2 and LFM2.5 use the earlier tool-call format, typically a Pythonic call list: - + -If you swap an agent to an LFM and let a generic JSON parser run, tool calls can appear broken. The format is different. +Serving runtimes can parse these native formats into OpenAI-compatible `tool_calls`. A client that reads raw generated text must handle the format for its selected model. What to do: 1. Pass tools through [`tokenizer.apply_chat_template(messages, tools=[...])`](https://huggingface.co/docs/transformers/main/en/chat_templating#using-applychattemplate) or the serving runtime's supported tool mechanism. -2. Parse the Pythonic call list between `<|tool_call_start|>` and `<|tool_call_end|>`. -3. On vLLM or SGLang, configure the `lfm2` tool-call parser instead of a generic JSON parser. +2. Prefer the serving runtime's structured `tool_calls`. If you parse raw text, handle LFM3 XML or the older `<|tool_call_start|>` / `<|tool_call_end|>` format as appropriate. +3. On vLLM or SGLang, use `qwen3_coder` for LFM3 and `lfm2` for LFM2/LFM2.5, with the other flags from the model's quickstart. 4. Return results as `tool` role messages, with JSON content if useful. -5. If you fine-tune for tool calling, train on the native Pythonic format and convert to any internal domain-specific language after parsing. +5. If you fine-tune for tool calling, format the data with the selected model's own chat template and native tool-call format. See [Tool Use](/lfm/key-concepts/tool-use) for the full tool-calling workflow. ## Fine-tuning -LFMs are standard Hugging Face causal LMs, so TRL and Unsloth-style pipelines carry over with two LFM-specific corrections. +Use a training workflow that supports your model architecture. The [LFM3-1.7B model card](https://huggingface.co/LiquidAI/LFM3-1.7B#fine-tuning) documents a TRL path. Check framework support before reusing an older training recipe. -First, update LoRA `target_modules`. LFM2 and LFM2.5 use a conv-attention hybrid with different module names than Llama-family models: +First, choose LoRA `target_modules` for the selected model. Inspect `model.named_modules()` or use its training recipe; module names differ across LFM generations. The following example applies to LFM2 and LFM2.5 only: ```python # LFM2 / LFM2.5 target_modules = ["w1", "w2", "w3", "q_proj", "k_proj", "v_proj", "out_proj", "in_proj"] -# Llama/Qwen/Gemma configurations are wrong for LFMs: +# These Llama/Qwen/Gemma module names do not apply to this LFM2/LFM2.5 recipe: # o_proj/gate_proj/up_proj/down_proj do not exist. ``` @@ -96,7 +108,7 @@ Before a long run, call `model.print_trainable_parameters()`. You should see mil Second, format training examples with the LFM chat template. Training data formatted with your previous model's template creates a silent distribution mismatch. -Everything else transfers directly. Use [Liquid Finetune](/lfm/fine-tuning/liquid-finetune), [TRL](/lfm/fine-tuning/trl), [Unsloth](/lfm/fine-tuning/unsloth), or [Halo](/lfm/fine-tuning/halo) depending on your existing workflow. +Use [Liquid Finetune](/lfm/fine-tuning/liquid-finetune), [TRL](/lfm/fine-tuning/trl), [Unsloth](/lfm/fine-tuning/unsloth), or [Halo](/lfm/fine-tuning/halo) according to the selected model and the framework's documented support. See [Fine-tuning Overview](/lfm/fine-tuning/overview) for the main fine-tuning workflow. diff --git a/guides/use-case-evaluation.mdx b/guides/use-case-evaluation.mdx index f928bbda..39043e1c 100644 --- a/guides/use-case-evaluation.mdx +++ b/guides/use-case-evaluation.mdx @@ -7,11 +7,14 @@ Use this guide when you want to validate LFMs on an actual product task. The goa ## Recommended starter models -- **Default:** [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) -- **Best quality:** [LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B) or [LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B) -- **Reasoning:** [LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) -- **Tightest footprint:** [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) -- **Vision and audio:** [LFM2.5-VL-1.6B](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B) or [LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) +- **Default text and local agents:** [LFM3-1.7B](/lfm/models/lfm3-1.7b) +- **Vision:** [LFM3-VL-2B](/lfm/models/lfm3-vl-2b) +- **Audio and speech:** [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) +- **Combined vision and audio:** [LFM3-Omni-2B](/lfm/models/lfm3-omni-2b) +- **Larger reasoning alternatives:** [LFM2.5-8B-A1B](/lfm/models/lfm25-8b-a1b) or [LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b), when they fit your hardware and task +- **Tightest footprint:** [LFM2.5-230M](/lfm/models/lfm25-230m) or [LFM2.5-350M](/lfm/models/lfm25-350m) + +Start with LFM3 for general use. Keep smaller or specialized older models in the evaluation when their memory needs, capabilities, or runtime support suit your deployment. Compare quality on your own task rather than treating model size or generation as a universal ranking. ## Integration @@ -21,7 +24,7 @@ Read these three pages before judging quality: - [Chat template](/lfm/key-concepts/chat-template): LFMs use a ChatML-style format. Always use `apply_chat_template` or the server's chat endpoint instead of hand-writing prompts. - [Text generation and prompting](/lfm/key-concepts/text-generation-and-prompting): start from LFM sampling defaults instead of carrying over another model's generation parameters. -- [Tool use](/lfm/key-concepts/tool-use): LFMs emit a Pythonic tool-call format. If your harness assumes OpenAI-style JSON tool calls, read the [migration guide](/guides/migration-guide) first. +- [Tool use](/lfm/key-concepts/tool-use): LFM3 emits XML tool calls; LFM2 and LFM2.5 use their earlier tool-call format. Configure the serving runtime's parser for your model so your harness receives structured tool calls. Read the [migration guide](/guides/migration-guide) before changing models. ## What to evaluate diff --git a/lfm/getting-started/welcome.mdx b/lfm/getting-started/welcome.mdx index f8eaa0a1..ba634b51 100644 --- a/lfm/getting-started/welcome.mdx +++ b/lfm/getting-started/welcome.mdx @@ -76,7 +76,7 @@ Built on a new tri-hybrid architecture, LFM3 sets a new standard in quality, spe Classification, routing, scoring, and extraction with zero generated tokens: d1-ultra and d1.1 via the Liquid API, open-weight d1-2B, d1-omni-2B, and d1-omni-600M - Task-specific models for extraction, retrieval, summarization, and translation + Task-specific models for extraction, retrieval for RAG (embeddings and reranking), summarization, and translation diff --git a/lfm/help/contributing.mdx b/lfm/help/contributing.mdx index d545dc51..a7a0757c 100644 --- a/lfm/help/contributing.mdx +++ b/lfm/help/contributing.mdx @@ -1,6 +1,7 @@ --- title: "Contributing to Docs" description: "Guidelines for contributing to Liquid AI documentation." +noindex: true --- We welcome contributions to improve our documentation! Here's how you can help. diff --git a/lfm/help/deprecations.mdx b/lfm/help/deprecations.mdx index 2f4f883b..270e0b8c 100644 --- a/lfm/help/deprecations.mdx +++ b/lfm/help/deprecations.mdx @@ -4,7 +4,7 @@ description: "Deprecated LFM models and SDKs with their recommended replacements --- -Deprecated models remain available for download on Hugging Face, but they are no longer actively maintained and receive no further updates. We recommend migrating to the suggested replacement for new projects when one is listed. Each listed replacement is a newer model for the same use case. Size and resource needs can differ: for example, LFM2.5-1.2B-Instruct is larger than LFM2-700M, and LFM2.5-2.6B has more parameters (2.70B vs 2.57B) and a larger vocabulary (128,000 vs 65,536 tokens) than LFM2-2.6B. Check the replacement's model page before migrating. +Deprecated models remain available for download on Hugging Face, but they are no longer actively maintained and receive no further updates. We recommend migrating to the suggested replacement for new projects when one is listed. Each listed replacement is a newer model for the same use case. Size and resource needs can differ, so check the replacement's parameters, context length, and runtime requirements before migrating. ## Deprecated models @@ -56,7 +56,7 @@ Deprecated models remain available for download on Hugging Face, but they are no LFM2-2.6B - LFM2.5-2.6B + LFM3-1.7B ✓ ✓ ✓ @@ -64,7 +64,7 @@ Deprecated models remain available for download on Hugging Face, but they are no LFM2-2.6B-Exp - LFM2.5-2.6B + LFM3-1.7B ✓ ✓ ✓ @@ -130,7 +130,7 @@ Deprecated models remain available for download on Hugging Face, but they are no LFM2-Audio-1.5B - LFM2.5-Audio-1.5B + LFM3-Audio-2B ✓ ✓ ✗ diff --git a/lfm/help/faqs.mdx b/lfm/help/faqs.mdx index db6b05d8..dfefe047 100644 --- a/lfm/help/faqs.mdx +++ b/lfm/help/faqs.mdx @@ -6,11 +6,11 @@ description: "Frequently asked questions about LFM models and deployment." ## General Questions -LFM (Liquid Foundation Models) are a family of efficient models built on a new hybrid architecture designed for fast training and inference. Open-weight models range from 230M to 24B parameters ([LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b) is the largest) and support text, vision, and audio. The family also includes embedding, encoder, and [decision models](/lfm/models/decision-models). +LFM (Liquid Foundation Models) are Liquid AI's efficient models for local and hosted deployment. [LFM3](/lfm/models/complete-library) is the latest generation, with text, vision, audio, omni, and decision models. The library also includes smaller and specialized LFM2.5 and LFM2 models, including embedding and encoder models. Open-weight models range from 230M to 24B parameters. -Most LFM text, vision, and audio models support a 32K token context length. [LFM2.5-8B-A1B](/lfm/models/lfm25-8b-a1b) supports 128K tokens, and [LFM2.5-2.6B](/lfm/models/lfm25-2.6b) and [LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b) support 131K. Encoder-based models have shorter limits (8K for the LFM2.5 encoders, 16K for d1-omni-600M). Each model page lists its exact limit. +[LFM3-1.7B](/lfm/models/lfm3-1.7b) supports 256K tokens, and [LFM3-VL-2B](/lfm/models/lfm3-vl-2b) supports 128K. Other limits vary by model: many LFM text, vision, and audio models support 32K, [LFM2.5-8B-A1B](/lfm/models/lfm25-8b-a1b) supports 128K, and [LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b) supports 131K. Encoder-based models have shorter limits (8K for the LFM2.5 encoders, 16K for d1-omni-600M). Each model page lists its exact limit; hosted API limits can differ from the open-weight checkpoints. @@ -26,18 +26,23 @@ LFM models are compatible with the frameworks below. Support varies by model (fo ## Model Selection -- **General chat/instruction following**: LFM2.5-1.2B-Instruct (recommended) -- **Vision tasks**: LFM2.5-VL-1.6B -- **Audio/speech**: LFM2.5-Audio-1.5B -- **Extraction from images**: LFM2.5-VL-1.6B-Extract or LFM2.5-VL-450M-Extract -- **Extraction from text**: LFM2.5-1.2B-Instruct, LFM2.5-350M, or LFM2.5-230M -- **Classification, routing, and scoring**: [decision models](/lfm/models/decision-models) such as d1-3B -- **Edge deployment**: LFM2.5-230M or LFM2.5-350M for the smallest footprint -- **Highest performance**: LFM2.5-24B-A2B (MoE architecture, 131K context) +- **General chat/instruction following**: [LFM3-1.7B](/lfm/models/lfm3-1.7b) +- **Vision tasks**: [LFM3-VL-2B](/lfm/models/lfm3-vl-2b) +- **Audio/speech**: [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) +- **Combined vision and audio**: [LFM3-Omni-2B](/lfm/models/lfm3-omni-2b) +- **Extraction from text**: [LFM3-1.7B](/lfm/models/lfm3-1.7b) with structured output +- **Classification, routing, and scoring**: [decision models](/lfm/models/decision-models) such as [d1-2B](/lfm/models/d1-2b) +- **Specialized extraction from images**: [LFM2.5-VL-1.6B-Extract](/lfm/models/lfm25-vl-1.6b-extract) or [LFM2.5-VL-450M-Extract](/lfm/models/lfm25-vl-450m-extract) +- **Smallest footprint**: [LFM2.5-230M](/lfm/models/lfm25-230m) or [LFM2.5-350M](/lfm/models/lfm25-350m) for tight memory budgets +- **Larger reasoning model**: [LFM2.5-24B-A2B](/lfm/models/lfm25-24b-a2b) (MoE architecture, 131K context); evaluate it on your task when a larger model fits your hardware - -LFM2.5 models are updated versions with improved training that deliver higher performance on the same hybrid architecture. Some LFM2.5 models also extend the context window or vocabulary. We recommend using LFM2.5 variants when available. + +- **LFM3** is the latest generation. Its tri-hybrid architecture combines gated short convolutions, sliding-window attention, and global attention. The family includes text, vision, audio, omni, and decision models, with updated chat templates and tool-call formats. +- **LFM2.5** builds on the LFM2 architecture with improved training. Some models extend the context window or vocabulary. It remains useful for smaller footprints, larger MoE models, and specialized tasks. +- **LFM2** is the earlier generation. Some specialized models remain useful; deprecated models and their replacements are listed on the [Deprecations](/lfm/help/deprecations) page. + +Start with LFM3 for general use, then consider an older model when its size, specialization, or runtime support better fits your needs. Use the selected model's own [chat template](/lfm/key-concepts/chat-template), [tool-call format](/lfm/key-concepts/tool-use), and generation settings. diff --git a/lfm/help/troubleshooting.mdx b/lfm/help/troubleshooting.mdx index b95d1494..74520ebb 100644 --- a/lfm/help/troubleshooting.mdx +++ b/lfm/help/troubleshooting.mdx @@ -5,20 +5,20 @@ description: "Common issues and solutions when working with LFM models." ## Installation Issues - -Ensure you have the latest version of transformers installed: + +Install the Transformers version required by the selected model's quickstart. For LFM3, start with an up-to-date installation: ```bash -pip install transformers>=4.55.0 +python -m pip install -U transformers torch accelerate ``` -If you're using an older version, the LFM model classes may not be available. +Some checkpoints require `trust_remote_code=True` when loading the model, tokenizer, or processor. Follow the model page's example; an older Transformers installation may lack the required model classes or APIs. Try these solutions in order: -1. **Use a smaller model**: Try LFM2-350M instead of LFM2-1.2B +1. **Use a smaller model**: If [LFM3-1.7B](/lfm/models/lfm3-1.7b) exceeds your memory budget, evaluate [LFM2.5-350M](/lfm/models/lfm25-350m) or [LFM2.5-230M](/lfm/models/lfm25-230m) for a smaller footprint 2. **Enable quantization**: ```python model = AutoModelForCausalLM.from_pretrained( @@ -41,30 +41,27 @@ model.gradient_checkpointing_enable() - Try downloading with `snapshot_download`: ```python from huggingface_hub import snapshot_download -snapshot_download("LiquidAI/LFM2.5-1.2B-Instruct") +snapshot_download("LiquidAI/LFM3-1.7B") ``` ## Inference Issues -Adjust generation parameters: +Start with the generation parameters on your model's page. For general use with LFM3-1.7B: ```python outputs = model.generate( **inputs, - max_new_tokens=512, - temperature=0.7, - top_p=0.9, + max_new_tokens=2048, + temperature=0.1, + top_k=50, do_sample=True, repetition_penalty=1.1 ) ``` -Key parameters to tune: -- `temperature`: Lower (0.3-0.5) for factual, higher (0.7-1.0) for creative -- `top_p`: 0.9 is a good default -- `repetition_penalty`: 1.1-1.2 helps avoid loops +Other models and tasks have different presets. [LFM3-1.7B](/lfm/models/lfm3-1.7b) lists separate settings for math and science; use the selected model's settings before tuning on your own examples. Also verify its [chat template](/lfm/key-concepts/chat-template). @@ -89,11 +86,11 @@ Increase `max_new_tokens`: ```python outputs = model.generate( **inputs, - max_new_tokens=1024, # Increase this value + max_new_tokens=2048, # Adjust to your task and model ) ``` -Check that your input isn't too long - LFM models support 32k context, but very long inputs leave less room for output. +Check the selected model's context limit and leave room for both the input and generated output, including reasoning tokens. [LFM3-1.7B](/lfm/models/lfm3-1.7b) supports 256K tokens and [LFM3-VL-2B](/lfm/models/lfm3-vl-2b) supports 128K; other checkpoints and hosted APIs can have different limits. A generation cap can cut off reasoning before the final answer. ## Fine-tuning Issues @@ -113,7 +110,7 @@ Memory optimization strategies: 1. **Use QLoRA** instead of full fine-tuning 2. **Reduce batch size** and increase gradient accumulation 3. **Enable gradient checkpointing** -4. **Use a smaller model** (LFM2-350M for experiments) +4. **Use a smaller model** such as [LFM2.5-350M](/lfm/models/lfm25-350m) for experiments with tight memory budgets ```python training_args = TrainingArguments( diff --git a/lfm/models/complete-library.mdx b/lfm/models/complete-library.mdx index 21ced3ca..ae2db79f 100644 --- a/lfm/models/complete-library.mdx +++ b/lfm/models/complete-library.mdx @@ -47,7 +47,7 @@ Start with the model family that matches your input and output shape, then choos - Task-specific models for extraction, summarization, RAG, and translation. + Task-specific models for extraction, summarization, retrieval for RAG (embeddings and reranking), and translation. @@ -113,10 +113,10 @@ Quantization reduces model size and speeds up inference with minimal quality los | [LFM2.5-VL-1.6B](/lfm/models/lfm25-vl-1.6b) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-GGUF) | [✓](https://huggingface.co/mlx-community/LFM2.5-VL-1.6B-8bit) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-ONNX) | ✗ | Yes (TRL) | | [LFM2.5-VL-450M](/lfm/models/lfm25-vl-450m) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-GGUF) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-MLX-8bit) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-ONNX) | ✗ | Yes (TRL) | | **Audio Models** | | | | | | | -| [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B-GGUF) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B-MLX-8bit) | ✗ | ✗ | Yes | +| [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B-GGUF) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B-MLX-8bit) | [✓](https://huggingface.co/LiquidAI/LFM3-Audio-2B-ONNX) | ✗ | Yes | | [LFM2.5-Audio-1.5B-JP](/lfm/models/lfm25-audio-1.5b-jp) | [✓](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B-JP) | [✓](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B-JP-GGUF) | ✗ | ✗ | ✗ | Yes ([liquid-audio](https://github.com/Liquid4All/liquid-audio#finetuning)) | | **Omni Models** | | | | | | | -| [LFM3-Omni-2B](/lfm/models/lfm3-omni-2b) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B-GGUF) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B-MLX-8bit) | ✗ | ✗ | Yes | +| [LFM3-Omni-2B](/lfm/models/lfm3-omni-2b) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B-GGUF) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B-MLX-8bit) | [✓](https://huggingface.co/LiquidAI/LFM3-Omni-2B-ONNX) | ✗ | Yes | | **Liquid Nanos** | | | | | | | | [LFM2.5-VL-1.6B-Extract](/lfm/models/lfm25-vl-1.6b-extract) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-Extract) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-Extract-GGUF) | ✗ | ✗ | ✗ | Yes (TRL) | | [LFM2.5-VL-450M-Extract](/lfm/models/lfm25-vl-450m-extract) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-Extract) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-Extract-GGUF) | ✗ | ✗ | ✗ | Yes (TRL) | diff --git a/lfm/models/lfm2-2.6b-exp.mdx b/lfm/models/lfm2-2.6b-exp.mdx index a7f39c0a..50e35636 100644 --- a/lfm/models/lfm2-2.6b-exp.mdx +++ b/lfm/models/lfm2-2.6b-exp.mdx @@ -1,6 +1,6 @@ --- title: "LFM2-2.6B-Exp" -description: "RL-only post-trained 2.6B model with improved math and reasoning capabilities (deprecated - use LFM2.5-2.6B instead)" +description: "RL-only post-trained 2.6B model with improved math and reasoning capabilities (deprecated - use LFM3-1.7B instead)" --- import { TextTransformers } from "/snippets/quickstart/text-transformers.mdx"; @@ -11,7 +11,7 @@ import { TextLlamacpp } from "/snippets/quickstart/text-llamacpp.mdx"; ← Back to Text Models -This model is deprecated. Use [LFM2.5-2.6B](/lfm/models/lfm25-2.6b) for improved agentic capabilities and a 131K context window. +This model is deprecated. For new projects, use [LFM3-1.7B](/lfm/models/lfm3-1.7b), which reasons and calls tools natively with a 256K context window. LFM2-2.6B-Exp is an experimental checkpoint of LFM2-2.6B post-trained with pure reinforcement learning on instruction following, knowledge, and math. diff --git a/lfm/models/lfm2-2.6b.mdx b/lfm/models/lfm2-2.6b.mdx index 76b71cca..5fe08155 100644 --- a/lfm/models/lfm2-2.6b.mdx +++ b/lfm/models/lfm2-2.6b.mdx @@ -1,6 +1,6 @@ --- title: "LFM2-2.6B" -description: "Highly capable 2.6B parameter model for deployment on most phones and laptops (deprecated - use LFM2.5-2.6B instead)" +description: "Highly capable 2.6B parameter model for deployment on most phones and laptops (deprecated - use LFM3-1.7B instead)" --- import { TextTransformers } from "/snippets/quickstart/text-transformers.mdx"; @@ -11,7 +11,7 @@ import { TextLlamacpp } from "/snippets/quickstart/text-llamacpp.mdx"; ← Back to Text Models -This model is deprecated. Use [LFM2.5-2.6B](/lfm/models/lfm25-2.6b) for improved agentic capabilities and a 131K context window. +This model is deprecated. For new projects, use [LFM3-1.7B](/lfm/models/lfm3-1.7b), which reasons and calls tools natively with a 256K context window. LFM2-2.6B is a versatile mid-sized model delivering strong performance across chat, reasoning, and tool-calling tasks. Optimized for deployment on consumer devices including phones and laptops. diff --git a/lfm/models/lfm2-audio-1.5b.mdx b/lfm/models/lfm2-audio-1.5b.mdx index 12ca6b34..d0625643 100644 --- a/lfm/models/lfm2-audio-1.5b.mdx +++ b/lfm/models/lfm2-audio-1.5b.mdx @@ -1,12 +1,12 @@ --- title: "LFM2-Audio-1.5B" -description: "1.5B audio model (deprecated - use LFM2.5-Audio-1.5B instead)" +description: "1.5B audio model (deprecated - use LFM3-Audio-2B instead)" --- ← Back to Audio Models -This model is deprecated. Use [LFM2.5-Audio-1.5B](/lfm/models/lfm25-audio-1.5b) for improved ASR, TTS, and CPU-friendly inference. +This model is deprecated. For new projects, use [LFM3-Audio-2B](/lfm/models/lfm3-audio-2b) for speech recognition, speech synthesis, and voice chat. For Japanese voice chat, see [LFM2.5-Audio-1.5B-JP](/lfm/models/lfm25-audio-1.5b-jp). LFM2-Audio-1.5B was the original fully interleaved audio/text model. It has been superseded by LFM2.5-Audio-1.5B, which features a custom LFM-based audio detokenizer and improved performance. diff --git a/lfm/models/lfm25-colbert-350m.mdx b/lfm/models/lfm25-colbert-350m.mdx index 01c015c2..71b44200 100644 --- a/lfm/models/lfm25-colbert-350m.mdx +++ b/lfm/models/lfm25-colbert-350m.mdx @@ -15,6 +15,8 @@ LFM2.5-ColBERT-350M is a late-interaction retrieval model. It creates 128-dimens Use LFM2.5-ColBERT-350M when you want stronger retrieval or reranking quality and can afford a larger per-token index. Use [LFM2.5-Embedding-350M](/lfm/models/lfm25-embedding-350m) when you need the smallest, fastest dense-vector index. + +It is a late-interaction model, not a cross-encoder: it encodes queries and documents separately and compares their token vectors with MaxSim, so document vectors can be indexed ahead of time. ## Specifications @@ -49,42 +51,12 @@ Use LFM2.5-ColBERT-350M when you want stronger retrieval or reranking quality an ## Quick Start -Run the model with Sentence Transformers (`MultiVectorEncoder`), with PyLate for indexing, retrieval, and reranking, or with llama.cpp through the GGUF checkpoint. +Choose **Retrieval** to search an indexed collection or **Reranking** to reorder candidate documents. - - **Install:** - ```bash - pip install "sentence-transformers>=6.0.0" - ``` - - **Run:** - ```python - from sentence_transformers import MultiVectorEncoder - - model = MultiVectorEncoder("LiquidAI/LFM2.5-ColBERT-350M", trust_remote_code=True) - - query = "Which planet is known as the Red Planet?" - documents = [ - "Venus is often called Earth's twin because of its similar size and proximity.", - "Mars, known for its reddish appearance, is often referred to as the Red Planet.", - "Jupiter, the largest planet in our solar system, has a prominent red spot.", - "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.", - ] - - query_embeddings = model.encode_query([query]) - document_embeddings = model.encode_document(documents) - print(query_embeddings[0].shape, document_embeddings[0].shape) - # (32, 128) (17, 128) - - # MaxSim late-interaction scoring (the Mars document ranks highest) - scores = model.similarity(query_embeddings, document_embeddings) - print(scores) - # tensor([[27.1621, 28.2578, 27.7266, 28.1992]]) - ``` - + + Use PyLate to index a collection and retrieve relevant documents. - **Install:** ```bash pip install -U pylate @@ -138,46 +110,93 @@ Run the model with Sentence Transformers (`MultiVectorEncoder`), with PyLate for - ```python - from pylate import models, rank - - model = models.ColBERT( - model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M", - trust_remote_code=True, - ) - - queries = ["Which city is Japan's capital?"] - documents = [[ - "Paris is the capital of France.", - "Tokyo is the capital of Japan.", - "Berlin is the capital of Germany.", - ]] - document_ids = [["fr", "jp", "de"]] - - query_embeddings = model.encode(queries, is_query=True) - document_embeddings = model.encode(documents, is_query=False) - - reranked = rank.rerank( - documents_ids=document_ids, - queries_embeddings=query_embeddings, - documents_embeddings=document_embeddings, - ) - print(reranked) - ``` - - - - **Start llama-server:** - ```bash - llama-server -hf LiquidAI/LFM2.5-ColBERT-350M-GGUF --embeddings - ``` - - The `/embedding` endpoint returns one vector per token. The client applies the ColBERT pre- and post-processing defined in the model's `config_sentence_transformers.json`: - - - Prefix queries with `[Q] ` and documents with `[D] `. - - Pad queries to 32 tokens with the pad token. Truncate documents to 512 tokens and drop the vectors of punctuation tokens listed in `skiplist_words`. - - L2-normalize each token vector, then score query/document pairs with MaxSim. - - The [GGUF model card](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M-GGUF) has a complete client script (`colbert-rerank.py`) that does this against a local `llama-server`. + Choose a library or runtime to score and rank your candidate documents. + + + + **Install:** + ```bash + pip install "sentence-transformers>=6.0.0" + ``` + + **Run:** + ```python + from sentence_transformers import MultiVectorEncoder + + model = MultiVectorEncoder("LiquidAI/LFM2.5-ColBERT-350M", trust_remote_code=True) + + query = "Which planet is known as the Red Planet?" + documents = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.", + ] + + query_embeddings = model.encode_query([query]) + document_embeddings = model.encode_document(documents) + print(query_embeddings[0].shape, document_embeddings[0].shape) + # (32, 128) (17, 128) + + # MaxSim late-interaction scoring (the Mars document ranks highest) + scores = model.similarity(query_embeddings, document_embeddings) + print(scores) + # tensor([[27.1621, 28.2578, 27.7266, 28.1992]]) + + for index in scores[0].argsort(descending=True).tolist(): + print(f"{scores[0, index].item():.4f} {documents[index]}") + ``` + + + + **Install:** + ```bash + pip install -U pylate + ``` + + **Rerank candidates:** + ```python + from pylate import models, rank + + model = models.ColBERT( + model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M", + trust_remote_code=True, + ) + + queries = ["Which city is Japan's capital?"] + documents = [[ + "Paris is the capital of France.", + "Tokyo is the capital of Japan.", + "Berlin is the capital of Germany.", + ]] + document_ids = [["fr", "jp", "de"]] + + query_embeddings = model.encode(queries, is_query=True) + document_embeddings = model.encode(documents, is_query=False) + + reranked = rank.rerank( + documents_ids=document_ids, + queries_embeddings=query_embeddings, + documents_embeddings=document_embeddings, + ) + print(reranked) + ``` + + + + **Start llama-server:** + ```bash + llama-server -hf LiquidAI/LFM2.5-ColBERT-350M-GGUF --embeddings + ``` + + The `/embedding` endpoint returns one vector per token. The client applies the ColBERT pre- and post-processing defined in the model's `config_sentence_transformers.json`: + + - Prefix queries with `[Q] ` and documents with `[D] `. + - Pad queries to 32 tokens with the pad token. Truncate documents to 512 tokens and drop the vectors of punctuation tokens listed in `skiplist_words`. + - L2-normalize each token vector, then score query/document pairs with MaxSim. + + The [GGUF model card](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M-GGUF) has a complete client script (`colbert-rerank.py`) that does this against a local `llama-server`. + + diff --git a/lfm/models/lfm25-embedding-350m.mdx b/lfm/models/lfm25-embedding-350m.mdx index 05df6ab6..ca7a25f8 100644 --- a/lfm/models/lfm25-embedding-350m.mdx +++ b/lfm/models/lfm25-embedding-350m.mdx @@ -1,6 +1,6 @@ --- title: "LFM2.5-Embedding-350M" -description: "350M dense bi-encoder model for multilingual retrieval and semantic search" +description: "350M multilingual text embedding model (dense bi-encoder) for vector search and retrieval in RAG (retrieval-augmented generation) pipelines" --- ← Back to Liquid Nanos @@ -53,7 +53,7 @@ The model was trained with asymmetric prefixes: `query: ` for queries and `docum - + **Install:** ```bash pip install -U sentence-transformers "transformers<5.12" diff --git a/lfm/models/lfm3-audio-2b.mdx b/lfm/models/lfm3-audio-2b.mdx index a62e9441..8993b487 100644 --- a/lfm/models/lfm3-audio-2b.mdx +++ b/lfm/models/lfm3-audio-2b.mdx @@ -13,6 +13,7 @@ LFM3-Audio-2B is a compact, fully featured audio model that runs locally on lapt HF GGUF MLX + ONNX
## Specifications diff --git a/lfm/models/lfm3-omni-2b.mdx b/lfm/models/lfm3-omni-2b.mdx index 877080dd..730898f8 100644 --- a/lfm/models/lfm3-omni-2b.mdx +++ b/lfm/models/lfm3-omni-2b.mdx @@ -13,6 +13,7 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und HF GGUF MLX + ONNX ## Specifications diff --git a/lfm/models/liquid-nanos.mdx b/lfm/models/liquid-nanos.mdx index 3f72dcfa..0349e37b 100644 --- a/lfm/models/liquid-nanos.mdx +++ b/lfm/models/liquid-nanos.mdx @@ -1,6 +1,6 @@ --- title: "Liquid Nanos" -description: "Liquid Nanos are task-specific LFMs for extraction, summarization, retrieval, and translation, small enough for local AI or high-volume serving." +description: "Liquid Nanos are task-specific LFMs for extraction, summarization, retrieval for RAG (embeddings and reranking), and translation, small enough for local AI or high-volume serving." icon: "sparkles" --- diff --git a/link-snapshot.yaml b/link-snapshot.yaml index 87a9741c..7c3ebcde 100644 --- a/link-snapshot.yaml +++ b/link-snapshot.yaml @@ -84,6 +84,7 @@ active: - /deployment/on-device/sdk/quick-start - /deployment/on-device/sdk/utilities - /deployment/on-device/sdk/voice-assistant + - /deployment/tools/model-bundling - /deployment/tools/model-bundling/authentication - /deployment/tools/model-bundling/bundle-creation - /deployment/tools/model-bundling/bundle-management @@ -147,8 +148,12 @@ active: - /lfm/fine-tuning/overview - /lfm/fine-tuning/trl - /lfm/fine-tuning/unsloth + - /lfm/frameworks/leap - /lfm/getting-started/connect-ai-tools + - /lfm/getting-started/intro - /lfm/getting-started/model-license + - /lfm/getting-started/text-generation-and-prompting + - /lfm/getting-started/vision - /lfm/getting-started/welcome - /lfm/help/connect-ai-tools - /lfm/help/contributing @@ -160,12 +165,14 @@ active: - /lfm/inference - /lfm/inference/llama-cpp - /lfm/inference/lm-studio + - /lfm/inference/lmstudio - /lfm/inference/mlx - /lfm/inference/ollama - /lfm/inference/transformers - /lfm/inference/vllm - /lfm/key-concepts/audio-capabilities - /lfm/key-concepts/chat-template + - /lfm/key-concepts/models - /lfm/key-concepts/omni-capabilities - /lfm/key-concepts/speculative-decoding - /lfm/key-concepts/text-generation-and-prompting @@ -220,6 +227,7 @@ active: - /lfm/models/lfm25-vl-3b - /lfm/models/lfm25-vl-450m - /lfm/models/lfm25-vl-450m-extract + - /lfm/models/lfm3-1-7b - /lfm/models/lfm3-1.7b - /lfm/models/lfm3-audio-2b - /lfm/models/lfm3-omni-2b diff --git a/style.css b/style.css index 88387732..2fcc4b35 100644 --- a/style.css +++ b/style.css @@ -115,12 +115,6 @@ a[href*="discord.gg"]:hover img { color: var(--lfm-fg-inverse) !important; } -/* Hide the "Ask Assistant" box in the desktop header. The assistant stays on: - it opens with Cmd/Ctrl+I, and the mobile header keeps its icon. */ -#assistant-entry { - display: none !important; -} - /* Mobile drawer: reuse Mintlify's close button inside the header. No dedicated close-button hook is exposed; its accessible label is the fallback. */ #mobile-nav > button[aria-label="Close navigation"] { diff --git a/styles.js b/styles.js index fd11dbce..f82d34a2 100644 --- a/styles.js +++ b/styles.js @@ -120,12 +120,6 @@ color: var(--lfm-fg-inverse) !important; } - /* Hide the "Ask Assistant" box in the desktop header. The assistant stays on: - it opens with Cmd/Ctrl+I, and the mobile header keeps its icon. */ - #assistant-entry { - display: none !important; - } - /* Mobile drawer: reuse Mintlify's close button inside the header. No dedicated close-button hook is exposed; its accessible label is the fallback. */ #mobile-nav > button[aria-label="Close navigation"] {