From 0862fc29feb27ebccd5548c3ca594365155b58f0 Mon Sep 17 00:00:00 2001 From: KakaruHayate Date: Sat, 22 Aug 2026 18:59:33 +0800 Subject: [PATCH] docs: add OpenUtau .oudep install guide (old-API -> v0.1.0 old_-prefixed pkg) + fix stale ggml_backend paths --- BUILDING.md | 10 +++++- README.md | 51 +++++++++++++++++++--------- README_CN.md | 50 ++++++++++++++++++--------- examples/external_consumer/README.md | 8 ++--- scripts/README.md | 22 ++++++------ 5 files changed, 93 insertions(+), 48 deletions(-) diff --git a/BUILDING.md b/BUILDING.md index 749de8f..81b55d5 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -203,7 +203,15 @@ build/bin/game_ggml_cli extract input.wav \ # Serve mode (for OpenUtau integration) build/bin/game_ggml_cli serve game_medium.gguf -# Then write binary request frames to stdin (see src/cli/main.cpp for protocol) +# Then write binary request frames to stdin (see src/cli/main.cpp for protocol). + +# API spec: the serve protocol (VRES request frames, notes-JSON responses) +# lives in src/cli/main.cpp (`serve` command). Packages in releases from +# v0.1.1+ speak this current protocol. +# +# > Early OpenUtau builds use the OLD API spec: install the `old_`-prefixed +# > .oudep from the v0.1.0 release (README → "OpenUtau integration"), not the +# > latest package. Do not remove the v0.1.0 old assets — old OpenUtau needs them. ``` ## CUDA compatibility and CI scope diff --git a/README.md b/README.md index 9cd04a7..3d1afcc 100644 --- a/README.md +++ b/README.md @@ -58,10 +58,10 @@ waveform (44100 Hz mono) ## Build ```bash -cmake -S ggml_backend -B ggml_backend/build \ +cmake -S . -B build \ -DCMAKE_BUILD_TYPE=Release \ -DGAME_GGML_BUILD_TESTS=ON -cmake --build ggml_backend/build -j +cmake --build build -j ``` Options: @@ -77,10 +77,10 @@ Options: ## Convert a PyTorch checkpoint ```bash -pip install -r ggml_backend/scripts/requirements.txt -python ggml_backend/scripts/convert_pt_to_gguf.py \ +pip install -r scripts/requirements.txt +python scripts/convert_pt_to_gguf.py \ --model-dir GAME-pt-1.0-medium \ - -o ggml_backend/assets/game_medium.gguf + -o game_medium.gguf ``` The script reads `model.pt` + `config.yaml` + `lang_map.json` from the given @@ -90,14 +90,14 @@ directory and writes a single GGUF file containing all 671 tensors (FP32) and Inspect the result: ```bash -./ggml_backend/build/bin/game_ggml_cli inspect ggml_backend/assets/game_medium.gguf +./build/bin/game_ggml_cli inspect game_medium.gguf ``` ## Run inference ```bash -./ggml_backend/build/bin/game_ggml_cli extract input.wav \ - -m ggml_backend/assets/game_medium.gguf \ +./build/bin/game_ggml_cli extract input.wav \ + -m game_medium.gguf \ --output-formats mid,txt,csv \ --output-dir out/ \ --tempo 120 \ @@ -228,6 +228,25 @@ configs produce identical note output): launches load precompiled PSOs instead of recompiling — no reliance on driver-level caches. +## OpenUtau integration (.oudep) + +Packages for OpenUtau are distributed as `.oudep` archives (a zip of +`game_ggml_cli` + `game_medium.gguf` + `config.json` + `oudep.yaml`) from the +[GitHub Releases](https://github.com/KakaruHayate/game.cpp/releases). Install +the package for your platform; OpenUtau unpacks it itself. + +| OpenUtau build | Package to install | +|---|---| +| **Current** (new serve-API protocol) | `game_ggml-.oudep` or `-q8` — from the **latest** release (e.g. `v0.1.3`). This package speaks the current serve protocol (`src/cli/main.cpp`, `game_ggml_cli serve`). | +| **Early** (old API spec) | `old_game_ggml-.oudep` — only shipped on the [`v0.1.0` release](https://github.com/KakaruHayate/game.cpp/releases/tag/v0.1.0). | + +> If your (older) OpenUtau build fails to talk to the engine, you are on the +> old API spec — download the **`old_`-prefixed** `.oudep` from v0.1.0, not the +> newest release. + +Platforms: `windows-x64-vulkan`, `linux-x64-vulkan`, +`macos-arm64-metal`, `macos-x64-metal` (old-prefixed set). + ## Reproducing the benchmark ```bash @@ -238,17 +257,17 @@ y, _ = librosa.load('28.wav', sr=44100, mono=True) sf.write('/tmp/28_44100.wav', y, 44100, subtype='PCM_16')" # 2. Capture PyTorch's D3PM RNG stream (also produces a reference MIDI) -python3 ggml_backend/scripts/align_demo.py /tmp/28_44100.wav \ +python3 scripts/align_demo.py /tmp/28_44100.wav \ -m GAME-pt-1.0-medium/model.pt \ - -g ggml_backend/assets/game_medium.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_medium.gguf \ + --cli build/bin/game_ggml_cli \ -l zh -o /tmp/align_out # 3. Run the 3-per-side subprocess-isolated benchmark -python3 ggml_backend/scripts/benchmark_align.py /tmp/28_44100.wav \ +python3 scripts/benchmark_align.py /tmp/28_44100.wav \ -m GAME-pt-1.0-medium/model.pt \ - -g ggml_backend/assets/game_medium.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_medium.gguf \ + --cli build/bin/game_ggml_cli \ --rng /tmp/align_out/align_rng.bin \ -l zh -o /tmp/bench_out --runs 3 ``` @@ -256,7 +275,7 @@ python3 ggml_backend/scripts/benchmark_align.py /tmp/28_44100.wav \ ## Using as a third-party library ```cmake -add_subdirectory(path/to/GAME/ggml_backend) +add_subdirectory(path/to/game.cpp) add_executable(my_app main.cpp) target_link_libraries(my_app PRIVATE game_ggml::game_ggml) @@ -305,7 +324,7 @@ standalone CMake project that builds against the library. ## Tests ```bash -ctest --test-dir ggml_backend/build --output-on-failure +ctest --test-dir build --output-on-failure ``` The suite has 37 tests covering: diff --git a/README_CN.md b/README_CN.md index 5d85860..e9d34b1 100644 --- a/README_CN.md +++ b/README_CN.md @@ -48,10 +48,10 @@ waveform (44100 Hz mono) ## 构建 ```bash -cmake -S ggml_backend -B ggml_backend/build \ +cmake -S . -B build \ -DCMAKE_BUILD_TYPE=Release \ -DGAME_GGML_BUILD_TESTS=ON -cmake --build ggml_backend/build -j +cmake --build build -j ``` 选项： @@ -67,10 +67,10 @@ cmake --build ggml_backend/build -j ## 转换 PyTorch checkpoint ```bash -pip install -r ggml_backend/scripts/requirements.txt -python ggml_backend/scripts/convert_pt_to_gguf.py \ +pip install -r scripts/requirements.txt +python scripts/convert_pt_to_gguf.py \ --model-dir GAME-pt-1.0-medium \ - -o ggml_backend/assets/game_medium.gguf + -o game_medium.gguf ``` 脚本读取指定目录下的 `model.pt` + `config.yaml` + `lang_map.json`，写出单个 GGUF（含全部 671 个 tensor（FP32）与 74 个 metadata KV）。 @@ -78,14 +78,14 @@ python ggml_backend/scripts/convert_pt_to_gguf.py \ 检查结果： ```bash -./ggml_backend/build/bin/game_ggml_cli inspect ggml_backend/assets/game_medium.gguf +./build/bin/game_ggml_cli inspect game_medium.gguf ``` ## 运行推理 ```bash -./ggml_backend/build/bin/game_ggml_cli extract input.wav \ - -m ggml_backend/assets/game_medium.gguf \ +./build/bin/game_ggml_cli extract input.wav \ + -m game_medium.gguf \ --output-formats mid,txt,csv \ --output-dir out/ \ --tempo 120 \ @@ -188,6 +188,24 @@ GPU 后端（Vulkan/Metal/CUDA）同样默认**开启**：设备侧缓存判定 - CPU 用 `-q8` 包（省内存、速度基本持平）。 - 冷启动：GPU 首次推理要编译 shader（Vulkan/Metal）。NVIDIA 驱动会跨进程缓存，仅首次调用多耗几秒。 +## OpenUtau 集成（.oudep） + +供 OpenUtau 使用的插件以 `.oudep` 压缩包分发（zip，内含 `game_ggml_cli` + +`game_medium.gguf` + `config.json` + `oudep.yaml`），见 +[GitHub Releases](https://github.com/KakaruHayate/game.cpp/releases)。按你的 +平台安装即可，OpenUtau 会自动解包。 + +| OpenUtau 版本 | 应安装的包 | +|---|---| +| **新版**（新的 serve-API 协议） | `game_ggml-<平台>.oudep` 或 `-q8`——取**最新** Release（如 `v0.1.3`）。该包使用当前的 serve 协议（`src/cli/main.cpp`，`game_ggml_cli serve`）。 | +| **早期**（旧 API 规范） | `old_game_ggml-<平台>.oudep` —— 仅存于 [`v0.1.0` Release](https://github.com/KakaruHayate/game.cpp/releases/tag/v0.1.0)。 | + +> 如果（较旧的）OpenUtau 连不上引擎，说明你走的是旧 API 规范——请下载 +> **`old_` 前缀**的 `.oudep`（v0.1.0），而不是最新 Release 里的包。 + +平台：`windows-x64-vulkan`、`linux-x64-vulkan`、 +`macos-arm64-metal`、`macos-x64-metal`（old 前缀集合）。 + ## 复现基准测试 ```bash @@ -198,17 +216,17 @@ y, _ = librosa.load('28.wav', sr=44100, mono=True) sf.write('/tmp/28_44100.wav', y, 44100, subtype='PCM_16')" # 2. 捕获 PyTorch 的 D3PM RNG 流（同时产出参考 MIDI） -python3 ggml_backend/scripts/align_demo.py /tmp/28_44100.wav \ +python3 scripts/align_demo.py /tmp/28_44100.wav \ -m GAME-pt-1.0-medium/model.pt \ - -g ggml_backend/assets/game_medium.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_medium.gguf \ + --cli build/bin/game_ggml_cli \ -l zh -o /tmp/align_out # 3. 运行 3-per-side 子进程隔离基准 -python3 ggml_backend/scripts/benchmark_align.py /tmp/28_44100.wav \ +python3 scripts/benchmark_align.py /tmp/28_44100.wav \ -m GAME-pt-1.0-medium/model.pt \ - -g ggml_backend/assets/game_medium.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_medium.gguf \ + --cli build/bin/game_ggml_cli \ --rng /tmp/align_out/align_rng.bin \ -l zh -o /tmp/bench_out --runs 3 ``` @@ -216,7 +234,7 @@ python3 ggml_backend/scripts/benchmark_align.py /tmp/28_44100.wav \ ## 作为第三方库使用 ```cmake -add_subdirectory(path/to/GAME/ggml_backend) +add_subdirectory(path/to/game.cpp) add_executable(my_app main.cpp) target_link_libraries(my_app PRIVATE game_ggml::game_ggml) @@ -261,7 +279,7 @@ int main() { ## 测试 ```bash -ctest --test-dir ggml_backend/build --output-on-failure +ctest --test-dir build --output-on-failure ``` 套件共 37 个测试，覆盖：后端初始化、GGUF I/O round-trip、每个算子（RMSNorm、Linear、LayerScale、Embedding、GLU-FFN、CgMLP、三种模式 RoPE、Attention、PAC、EBF block）、Encoder/Segmenter/Estimator 端到端 vs PyTorch 参考 dump、D3PM 8-step 注入 RNG 逐位一致（容忍 Metal FP32 漂移引起的 ≤2/100 边界翻转）、mel 频谱 vs `lib.feature.mel.StretchableMelSpectrogram`、Slicer、MIDI 写入器、TXT/CSV 文本写入器、全流水线逐位 E2E。 diff --git a/examples/external_consumer/README.md b/examples/external_consumer/README.md index 46ec140..0bad534 100644 --- a/examples/external_consumer/README.md +++ b/examples/external_consumer/README.md @@ -12,9 +12,9 @@ examples/external_consumer/ ```bash # From the repo root: -cmake -S ggml_backend/examples/external_consumer -B /tmp/consumer +cmake -S examples/external_consumer -B /tmp/consumer cmake --build /tmp/consumer -j -/tmp/consumer/my_app ggml_backend/assets/game_small.gguf /path/to/input.wav +/tmp/consumer/my_app game_small.gguf /path/to/input.wav ``` ## What it illustrates @@ -29,10 +29,10 @@ cmake --build /tmp/consumer -j broader format support. The model's backend (CPU / Metal / CUDA / Vulkan) is inherited from the parent -`ggml_backend/` build configuration. To override, pass the relevant flags +`your build configuration. To override, pass the relevant flags when you configure: ```bash -cmake -S ggml_backend/examples/external_consumer -B /tmp/consumer \ +cmake -S examples/external_consumer -B /tmp/consumer \ -DGAME_GGML_METAL=OFF # CPU-only build ``` diff --git a/scripts/README.md b/scripts/README.md index a68c140..233dba1 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1,4 +1,4 @@ -# `ggml_backend/scripts/` +# `scripts/` Python helpers that complement the C++ binaries. All scripts assume they are run from the repo root and have `torch`, `numpy`, `gguf`, `pyyaml`, @@ -15,28 +15,28 @@ run from the repo root and have `torch`, `numpy`, `gguf`, `pyyaml`, ## Typical workflow ```bash -pip install -r ggml_backend/scripts/requirements.txt +pip install -r scripts/requirements.txt # one-time: produce the GGUF for the C++ backend -python ggml_backend/scripts/convert_pt_to_gguf.py \ +python scripts/convert_pt_to_gguf.py \ --model-dir GAME-pt-1.0-small \ - -o ggml_backend/assets/game_small.gguf + -o game_small.gguf # each time you touch a C++ op: regenerate the reference dumps so tests pass -python ggml_backend/scripts/dump_reference.py --category all --out ggml_backend/tests/data +python scripts/dump_reference.py --category all --out tests/data # one-off: verify bit-exact alignment on a real clip -python ggml_backend/scripts/align_demo.py /path/to/your.wav \ +python scripts/align_demo.py /path/to/your.wav \ -m GAME-pt-1.0-small/model.pt \ - -g ggml_backend/assets/game_small.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_small.gguf \ + --cli build/bin/game_ggml_cli \ -l zh -o /tmp/align_out # one-off: speed + memory benchmark (reuses align_rng.bin from align_demo) -python ggml_backend/scripts/benchmark_align.py /path/to/your.wav \ +python scripts/benchmark_align.py /path/to/your.wav \ -m GAME-pt-1.0-small/model.pt \ - -g ggml_backend/assets/game_small.gguf \ - --cli ggml_backend/build/bin/game_ggml_cli \ + -g game_small.gguf \ + --cli build/bin/game_ggml_cli \ --rng /tmp/align_out/align_rng.bin \ -l zh -o /tmp/bench_out --runs 3 ```