From 13bfa1cb4f4a46f6b6a084e44bd0600dc45b4d18 Mon Sep 17 00:00:00 2001 From: sunerpy Date: Sat, 24 Oct 2026 16:52:50 +0800 Subject: [PATCH] =?UTF-8?q?docs:=20=E8=87=AA=E5=BB=BA=E8=AF=86=E5=88=AB?= =?UTF-8?q?=E6=9C=8D=E5=8A=A1=E9=A1=B5=E7=9A=84=E6=80=A7=E8=83=BD=E8=A1=A8?= =?UTF-8?q?=E6=8B=86=E6=88=90=E4=B8=A4=E5=BC=A0=EF=BC=8C=E6=89=8B=E6=9C=BA?= =?UTF-8?q?=E4=B8=8A=E6=9B=B4=E6=98=93=E8=AF=BB?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/site/recognition/self-host.md | 21 +++++++++++++-------- docs/site/zh/recognition/self-host.md | 21 +++++++++++++-------- 2 files changed, 26 insertions(+), 16 deletions(-) diff --git a/docs/site/recognition/self-host.md b/docs/site/recognition/self-host.md index 2248095..24e902f 100644 --- a/docs/site/recognition/self-host.md +++ b/docs/site/recognition/self-host.md @@ -97,15 +97,20 @@ vLLM's and Hugging Face's usage statistics are turned off. ## Performance and tuning -On an AWS g6.xlarge (NVIDIA L4, 24 GB) and a g4dn.xlarge (NVIDIA T4, 16 GB) in the Seoul region, with the default settings (measured 2026-10-24, sent on the server itself): +On an AWS g6.xlarge (NVIDIA L4, 24 GB) and a g4dn.xlarge (NVIDIA T4, 16 GB) in the Seoul region, with the default settings, recognising one recording took, in seconds (measured 2026-10-24, sent on the server itself): | Recording | L4 · 1.7B | L4 · 0.6B | T4 · 1.7B | | --- | --- | --- | --- | -| 5.3 s | 0.37 s | 0.15 s | 0.41 s | -| 7.3 s | 0.50 s | 0.20 s | 0.53 s | -| 14.3 s | 0.73 s | 0.30 s | 0.79 s | -| 8 recordings of 14.3 s sent at once, all done | 0.85 s | 0.38 s | 1.16 s | -| 16 recordings of 14.3 s sent at once, all done | 1.63 s | 0.71 s | 2.28 s | +| 5.3 s | 0.37 | 0.15 | 0.41 | +| 7.3 s | 0.50 | 0.20 | 0.53 | +| 14.3 s | 0.73 | 0.30 | 0.79 | + +Recordings of 14.3 s sent at once were all done after, in seconds: + +| Sent at once | L4 · 1.7B | L4 · 0.6B | T4 · 1.7B | +| --- | --- | --- | --- | +| 8 | 0.85 | 0.38 | 1.16 | +| 16 | 1.63 | 0.71 | 2.28 | The recordings sent at once went to vLLM directly, not through the gateway, which allows each address 8 connections. From another computer, add the time it takes to upload the recording. 0.6B is faster, but in these recordings it got more names and English words wrong. @@ -113,7 +118,7 @@ The recordings sent at once went to vLLM directly, not through the gateway, whic vLLM takes the share of GPU memory set by `--gpu-memory` when it starts; what the model does not need goes to recognising several recordings at once. The memory taken does not change with the number of requests. On an NVIDIA L4 with 24 GB (measured 2026-10-24): -| Model | `--gpu-memory` | Memory taken | Recordings recognised at once, in total | +| Model | `--gpu-memory` | Memory taken | At once | | --- | --- | --- | --- | | 1.7B | 0.85 | 18.3 GB | about 120 minutes | | 1.7B | 0.40 | 8.1 GB | about 33 minutes | @@ -122,7 +127,7 @@ vLLM takes the share of GPU memory set by `--gpu-memory` when it starts; what th | 0.6B | 0.25 | 4.7 GB | about 24 minutes | | 0.6B | 0.20 | 3.6 GB | about 14 minutes | -Every setting started and recognised normally; the lowest recognised a single recording as fast as the default. The total length is estimated from what was measured: a recording of 14.3 seconds and its text took 245 tokens. A card with less than 8 GB uses 0.6B by default. +**At once** is the total length of the recordings that can be recognised at the same time, estimated from what was measured: a recording of 14.3 seconds and its text took 245 tokens. Every setting started and recognised normally; the lowest recognised a single recording as fast as the default. A card with less than 8 GB uses 0.6B by default. ### Settings diff --git a/docs/site/zh/recognition/self-host.md b/docs/site/zh/recognition/self-host.md index cb3fee9..951350a 100644 --- a/docs/site/zh/recognition/self-host.md +++ b/docs/site/zh/recognition/self-host.md @@ -97,15 +97,20 @@ vLLM 和 Hugging Face 的使用统计默认关闭。 ## 性能与调优 -在 AWS 首尔区域的 g6.xlarge（NVIDIA L4 24 GB）和 g4dn.xlarge（NVIDIA T4 16 GB）上，使用默认设置（2026-10-24 实测，在服务器本机发送）： +在 AWS 首尔区域的 g6.xlarge（NVIDIA L4 24 GB）和 g4dn.xlarge（NVIDIA T4 16 GB）上，使用默认设置，识别一段录音的用时（秒；2026-10-24 实测，在服务器本机发送）： | 录音 | L4 · 1.7B | L4 · 0.6B | T4 · 1.7B | | --- | --- | --- | --- | -| 5.3 秒 | 0.37 秒 | 0.15 秒 | 0.41 秒 | -| 7.3 秒 | 0.50 秒 | 0.20 秒 | 0.53 秒 | -| 14.3 秒 | 0.73 秒 | 0.30 秒 | 0.79 秒 | -| 8 段 14.3 秒的录音同时发送，全部完成 | 0.85 秒 | 0.38 秒 | 1.16 秒 | -| 16 段 14.3 秒的录音同时发送，全部完成 | 1.63 秒 | 0.71 秒 | 2.28 秒 | +| 5.3 秒 | 0.37 | 0.15 | 0.41 | +| 7.3 秒 | 0.50 | 0.20 | 0.53 | +| 14.3 秒 | 0.73 | 0.30 | 0.79 | + +同时发送多段 14.3 秒的录音时，全部完成的用时（秒）： + +| 同时发送 | L4 · 1.7B | L4 · 0.6B | T4 · 1.7B | +| --- | --- | --- | --- | +| 8 段 | 0.85 | 0.38 | 1.16 | +| 16 段 | 1.63 | 0.71 | 2.28 | 同时发送的录音直接发给 vLLM，没有经过网关：网关对每个地址最多允许 8 个连接。从其他电脑使用时，还需要加上上传录音的时间。0.6B 更快，但在这几段录音中，人名和英文词的错误更多。 @@ -113,7 +118,7 @@ vLLM 和 Hugging Face 的使用统计默认关闭。 vLLM 启动时按 `--gpu-memory` 的比例占用显存，其中模型之外的部分用于同时识别多段录音。占用的显存不随请求多少而变化。在 NVIDIA L4 24 GB 上（2026-10-24 实测）： -| 模型 | `--gpu-memory` | 占用显存 | 可同时识别的录音总长 | +| 模型 | `--gpu-memory` | 占用显存 | 可同时识别 | | --- | --- | --- | --- | | 1.7B | 0.85 | 18.3 GB | 约 120 分钟 | | 1.7B | 0.40 | 8.1 GB | 约 33 分钟 | @@ -122,7 +127,7 @@ vLLM 启动时按 `--gpu-memory` 的比例占用显存，其中模型之外的 | 0.6B | 0.25 | 4.7 GB | 约 24 分钟 | | 0.6B | 0.20 | 3.6 GB | 约 14 分钟 | -每种设置都能正常启动和识别，表中最低的设置识别单段录音的速度与默认设置相同。录音总长按实测用量估算：一段 14.3 秒的录音及其文字共占 245 个 token。显存小于 8 GB 的显卡默认使用 0.6B。 +「可同时识别」是同一时刻可以识别的录音总长，按实测用量估算：一段 14.3 秒的录音及其文字共占 245 个 token。每种设置都能正常启动和识别，表中最低的设置识别单段录音的速度与默认设置相同。显存小于 8 GB 的显卡默认使用 0.6B。 ### 设置