diff options
Diffstat (limited to 'app/docs/backend-sglomni.md')
| -rw-r--r-- | app/docs/backend-sglomni.md | 10 |
1 files changed, 7 insertions, 3 deletions
diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md index bf7d3a9..4f553f9 100644 --- a/app/docs/backend-sglomni.md +++ b/app/docs/backend-sglomni.md @@ -60,7 +60,7 @@ inside the start timeout. | `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | | | `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality | | `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | | -| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | no config file needed | +| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + raised generation cap (requests carry `max_new_tokens=12288`; the engine's 2048-frame default is ~27 s and clamps per-request values) | | `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | | | `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz | | `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` | @@ -155,6 +155,7 @@ app/envs/sglomni/bin/hf download bosonai/higgs-audio-v3-tts-4b # 5. Start a server manually (the managed flow does this for you) app/envs/sglomni/bin/sgl-omni serve \ --model-path bosonai/higgs-audio-v3-tts-4b \ + --config app/backends/sglomni/configs/higgs_audio_v3_tts.yaml \ --port 8100 # 6. Convert @@ -162,9 +163,12 @@ python audiobook.py --backend sglomni --model higgs_audio_v3_tts \ --api-url http://127.0.0.1:8100 ``` -Models that take a vendored config file (all but Higgs and ZONOS2) add +Models that take a vendored config file (all but ZONOS2) add `--config app/backends/sglomni/configs/<key>.yaml` to the serve command — -the managed spec builds this from the catalog automatically. +the managed spec builds this from the catalog automatically. Higgs's config +also exists for a reason: the upstream pipeline budgets 98% of the card +across its colocated stages, which OOMs on a 24 GB GPU once anything else +uses VRAM, and its engine caps requests at ~27 s of speech. ## CLI examples |