fix(voxcpm1): extract tokenizer/config directly from GGUF for standalone packages

[jasonchen31]

Summary

Fixes failed to load model resources using builtin model spec for family 'voxcpm1' source 'gguf': missing model package file 'tokenizer_config': /.../VoxCPM1-GGUF/tokenizer_config.json for standalone LLama-style GGUFs.

Published VoxCPM1-GGUF (voxcpm-0.5b-q8_0-audiovae-f16.gguf) is general.architecture=llama with embedded_sidecars=false. It embeds:

• tokenizer.ggml.tokens (73448) / tokenizer.ggml.merges (104297) / tokenizer.ggml.token_type / bos=1
• voxcpm_lm_config_*, voxcpm_encoder_config_*, voxcpm_dit_config_*, voxcpm_audio_vae_config_*, voxcpm_patch_size etc.

Runtime previously required external sidecars via engine::model_spec::load_resource_bundle with model_specs/voxcpm1.json:263 sources[gguf].files: {config, tokenizer_json, tokenizer_config} → src/framework/model_spec/package.cpp:283 threw before VoxCPM1TextTokenizer could fallback.

Changes

git diff origin/main --stat – 4 files, 520 insertions(+), 70 deletions(-)

src/community_models/voxcpm1/assets.cpp

• Add engine/framework/io/filesystem.h, ggml.h, gguf.h, <optional>
• Helpers locate_gguf() (via open_tensor_source("weights") / find_directory_gguf), gguf_get_i64 / gguf_get_f32_or / gguf_get_str_or / gguf_get_i64_array / gguf_get_f32_array
• parse_config_from_gguf(gguf_path) – reads all voxcpm_* KV (voxcpm_architecture, lm_config incl. longrope factors, encoder/dit/audio_vae, patch_size, feat_dim, max_length, device/dtype, sample_rate 16000/44100)
• parse_config(resources) – has_file("config") ? JSON else locate_gguf() → GGUF
• Relaxed validate_weight_anchors for locenc.special_token (element-count check [1024] vs [1,1,1,1024])
• load_voxcpm1_assets() wraps load_resource_bundle in try/catch(missing tokenizer_config|tokenizer_json|config.json) → prepare_model_directory minimal ResourceBundle with weights/audiovae_weights + optional sidecars via try_add

src/community_models/voxcpm1/tokenizer_text.cpp

• Add engine/framework/assets/tensor_source.h, ggml.h, gguf.h, <optional>
• locate_gguf_for_tokenizer(), load_tokenizer_from_gguf(gguf_path) – vocab from tokens, merges, token_type==3 → special_tokens (23 control incl. <|audio_start|>), fallback ensures audio_start/end, bos=1/eos=2/unk=0
• Constructor checks has_file("tokenizer_json" && "tokenizer_config") else GGUF

src/community_models/voxcpm1/minicpm.cpp:200

• Flexible locenc.special_token load: require_metadata → try [1,1,1,hidden] then [hidden]

src/community_models/voxcpm1/audiovae.cpp

• load_snake() tries {1,channels,1} → {channels,1} → {channels}
• load_sr_condition() synthesizes scale=1/bias=0 when sr_cond_model.* missing (custom GGUF)
• load_wn_conv1d() fallback for omitted leading 1 (decoder.model.7 [96,7] vs [1,96,7]) with element-count check

Verification – local sense_asr

Model: models/SenseVoice-Small-GGUF/sensevoice-small-q8-audiocpp-v1.gguf (243M, family=sense_asr, CPU)

# TTS (16kHz, RMS 0.057)
audiocpp_cli --task tts --family voxcpm1 --model /tmp/single_gguf_test \
  --text "Hello world, this is a test." --out /tmp/verify.wav
audiocpp_cli --task asr --family sense_asr --model models/SenseVoice-Small-GGUF \
  --audio /tmp/verify.wav
# → Hello world, this is a test.

# Clone (V1 prompt-continuation requires reference_text)
audiocpp_cli --task tts --family voxcpm1 --model /tmp/single_gguf_test \
  --text "This VoxCPM1 path test clones the reference speaker for a short review sentence." \
  --voice-ref assets/resources/sample_16k.wav \
  --request-option reference_text="Some call me nature." --out /tmp/clone.wav
audiocpp_cli --task asr --family sense_asr --model models/SenseVoice-Small-GGUF \
  --audio /tmp/clone.wav
# → This Vox C PM1 Pa test clones the reference speaker for a short review sentence.

# Original issue path
audiocpp_cli --inspect --family voxcpm1 --model /tmp/single_gguf_test
# family=voxcpm1 variant=voxcpm model_root=/tmp/single_gguf_test (no longer throws)

Notes

• model_specs/voxcpm1.json unchanged (git diff shows 0 for this file) per review.
• Build: cmake -B /tmp/build_fix && cmake --build /tmp/build_fix --parallel 8 (CPU), bin/audiocpp_cli / bin/audiocpp_server link OK.

Related Issue

voxcpm1 model extract the tokenizer directly from the gguf files. However somehow it report:

failed to load model resources using builtin model spec for family 'voxcpm1' source 'gguf': missing model package file 'tokenizer_config': /mnt/data/chatgpt/models/audio-cpp/VoxCPM1-GGUF/tokenizer_config.json"`

[0xShug0]

@jasonchen31 Thanks for the PR! I’ll be AFK most of the day today and will look at the issue tonight. I think it may be a conversion-stage issue and could have a simple fix.

[0xShug0]

@jasonchen31 I'm a little confused here. The voxcpm1 gguf from audio.cpp HF repo does contain the tokenizer and can work without the PR. Looks like you try to support non audio.cpp-native, third-party GGUFs? I’m not sure we should spend much effort supporting non-native GGUFs.

[jasonchen31]

@0xShug0 You are right. I was not aware of the change to the gguf model. After downloading the audio.cpp native model, it works. Only that I notice that the voxcpm1 cannot be download with webui. So I created another PR #424 which also fix the Contonese mis-triggering issue.