Skip to content

model_selection.md

Latest commit

 

History

History
137 lines (101 loc) · 10.4 KB

model_selection.md

File metadata and controls

137 lines (101 loc) · 10.4 KB
Raw

FunASR Model Selection Guide

Transformers-first Nano evaluation

For Chinese, English and Japanese transcription through Hugging Face APIs, start with native Transformers 5.17.0, the official FunAudioLLM/Fun-ASR-Nano-2512-hf checkpoint and the CPU example. The toolkit and service paths below remain separate; the native export does not add timestamps, speakers or an HTTP endpoint.

Use this guide when you are choosing a first model, comparing FunASR with Whisper or a cloud ASR provider, or deciding which model alias to expose through the OpenAI-compatible API.

Fast default path

For GPU evaluation of Chinese, English, Japanese, or Chinese dialects and regional accents, start with the flagship Fun-ASR-Nano (SenseVoice encoder + Qwen3 decoder). Compare it on your own audio before choosing a production model:

from funasr import AutoModel

model = AutoModel(model="FunAudioLLM/Fun-ASR-Nano-2512", device="cuda")
result = model.generate(input="meeting.wav")
print(result[0]["text"])

For non-autoregressive multilingual ASR with emotion/event tags, or a CPU evaluation path, start with SenseVoice-Small. The example below adds separate VAD and speaker-processing stages for meeting transcripts; speaker diarization is not a SenseVoice output from the same recognition pass:

from funasr import AutoModel

model = AutoModel(
    model="iic/SenseVoiceSmall",
    vad_model="fsmn-vad",
    spk_model="cam++",
    device="cuda",  # use "cpu" for a portable smoke test
)
result = model.generate(input="meeting.wav")

SenseVoice produces transcription and rich tags. fsmn-vad locates speech; cam++ produces speaker embeddings, which the pipeline clusters into anonymous labels within a recording. These labels do not identify registered people and are not stable identities across recordings.

Switch to Paraformer when your workload is Mandarin-only and you want character-level timestamps or hotwords.

Decision table

Need Start with Why Next doc
Fast multilingual private transcription SenseVoice-Small Strong default with ASR, emotion tags, audio event tags, and CPU viability. README quick start
Mandarin production ASR Paraformer-Large Mature Chinese ASR path with VAD and punctuation. Tutorial
English-only route in the OpenAI API example paraformer-en alias Smaller English route for API compatibility checks. OpenAI API example
LLM-based ASR or Chinese/English/Japanese + dialect experiments Fun-ASR-Nano Evaluate the Python path first; split-engine and native vLLM use different loading contracts. Choose a vLLM path
Offline long-form ASR with anonymous diarization MOSS-Transcribe-Diarize One offline request returns transcription, timestamps, and per-recording anonymous speaker labels; it does not identify known people and needs no external VAD or speaker model. MOSS deployment guide
Live captions or call-center streams Runtime WebSocket service Designed for long-lived streaming sessions and partial results. Runtime service docs
Batch archive processing SenseVoice-Small or Paraformer-Large Stable offline transcription path; caller owns manifests, retries, and logs. Batch ASR example
Migration from Whisper/cloud ASR SenseVoice-Small first, then benchmark alternatives Gives a strong baseline before deeper model-specific tuning. Migration guide

OpenAI-compatible API aliases

The examples/openai_api server exposes short aliases so application teams do not need to know model repository IDs:

  • sensevoice uses iic/SenseVoiceSmall for multilingual HTTP transcription on CPU/GPU. Returned text has rich tags removed.
  • paraformer uses paraformer-zh with VAD and punctuation for a Mandarin-oriented route.
  • paraformer-en uses paraformer-en with VAD for English transcription in OpenAI-style clients.
  • fun-asr-nano uses FunAudioLLM/Fun-ASR-Nano-2512 for evaluating Chinese, English, Japanese, and Chinese dialect/accent coverage. Select a compatible runtime when evaluating vLLM acceleration.
  • moss-transcribe-diarize uses the third-party OpenMOSS-Team/MOSS-Transcribe-Diarize model for offline transcription and anonymous per-recording speaker labels. Prepare its separate dependencies and reviewed remote code using the MOSS guide; request verbose_json for structured segments. It does not require an external VAD/speaker model and does not identify known people.

These aliases describe the example server, which loads AutoModel. They do not configure native vLLM or automatically select AutoModelVLLM. The packaged funasr-server has a separate loader and backend selection; do not copy an alias or a performance result between services without checking the corresponding HTTP guide.

The example HTTP service cleans both top-level text and segment text in verbose_json; that format does not restore emotion/event tags. If you need the original tags, use the Python SDK and preserve the returned text before display postprocessing. See the raw-tag recipe.

For Ascend NPU deployments, treat fun-asr-nano separately from SenseVoice / Paraformer. The Fun-ASR-Nano PyTorch AutoModel path has community compatibility evidence on 310P3 after the NPU autocast fix, but it was much slower than CPU in that smoke test; AutoModelVLLM still depends on vLLM-Ascend operator support and has hit Qwen3 rotary / TransData failures. Use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime for production unless you are actively validating an Ascend backend.

Check the live service before wiring clients:

curl http://localhost:8000/v1/models
python examples/openai_api/smoke_test.py --base-url http://localhost:8000 --model sensevoice

For SDK, JavaScript, workflow, Postman, OpenAPI, Docker, and Kubernetes paths, start from the OpenAI API example.

Runtime choice by workload

Workload Runtime path Notes
Notebook or one-off evaluation Python AutoModel Shortest path for install, model download, and output-shape checks.
Internal HTTP service OpenAI-compatible API Reuse OpenAI-style clients, Dify, n8n, LangChain, AutoGen, and HTTP nodes.
Repeatable local container demo Docker Compose API CPU-first smoke test; adapt the image before using CUDA.
Internal cluster service Kubernetes API template Private ClusterIP, persistent model cache, /health probes, and port-forward smoke test.
Live audio Runtime WebSocket service Validate chunk size, VAD, endpointing, reconnects, and client backpressure with real audio.
LLM-based ASR throughput Choose split-engine or native vLLM below Match the checkpoint, loading API and tested environment; this is not a Paraformer backend.

See the deployment matrix when you are choosing between these paths.

Choose the vLLM Checkpoint and Interface

Path Checkpoint and interface Read next
FunASR split-engine Base FunAudioLLM/Fun-ASR-Nano-2512 assets through AutoModelVLLM; FunASR handles the audio side and vLLM the decoder. Split-engine preparation and limits
Official native vLLM Converted FunAudioLLM/Fun-ASR-Nano-2512-vllm snapshot through vLLM's native model implementation and /v1/audio/transcriptions. Not an AutoModelVLLM load. Official functional validation
Historical community native vLLM Community allendou/Fun-ASR-Nano-2512-vllm, tested on 2026-08-13. Its timings belong to that checkpoint and environment. Historical community record

The official record pins a model revision and an existing environment; it is not a clean-install recipe, a sustained-load benchmark, or proof of /v1/realtime streaming. Do not reuse the historical community timings for the official model. For MOSS, follow its own deployment guide: the Nano checkpoints and validation above do not establish MOSS runtime compatibility. Choose model, checkpoint, interface and environment together before testing your own workload.

Benchmark before committing

Do not choose a model from a single clean demo file. Use a small representative set first:

  • 20-50 audio files that cover short clips, long meetings, silence, noise, overlapping speakers, domain vocabulary, and target languages.
  • Record model name, model revision, FunASR version, device, CPU/GPU type, CUDA/PyTorch version, runtime path, batch size, and whether warmup/model download time is excluded.
  • Track quality with your normal WER/CER or human review process, not only transcript readability.
  • Track latency, throughput, memory, failures, and upload size limits together.
  • Keep at least one public sample for smoke tests and at least one private realistic sample for deployment validation.

For migration work, use the migration benchmark example and the migration guide.

Practical recommendations

  • With a GPU, evaluate Fun-ASR-Nano for Chinese, English, Japanese, and Chinese dialects/accents. Include difficult audio, context and proper nouns in your own comparison. For the separate 31-language checkpoint, use Fun-ASR-MLT-Nano.
  • On CPU, or for multilingual + emotion workloads, use SenseVoice-Small (fast non-autoregressive, CPU-viable).
  • Use Paraformer when your production traffic is primarily Mandarin and you want timestamps or hotwords.
  • For offline long recordings that need anonymous per-recording speaker labels in the same request, use MOSS-Transcribe-Diarize; it is not a realtime WebSocket or known-person identification path.
  • Use the streaming runtime when partial results and long-lived connections matter more than a single final transcript.
  • Keep model aliases stable in production runbooks so benchmark results and bug reports are reproducible.
  • Open a Deployment Help issue with model, device, command, logs, audio duration, and runtime path when you get stuck.