diff options
Diffstat (limited to 'app/docs/backend-sglomni.md')
| -rw-r--r-- | app/docs/backend-sglomni.md | 36 |
1 files changed, 35 insertions, 1 deletions
diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md index 4f553f9..58e5ce5 100644 --- a/app/docs/backend-sglomni.md +++ b/app/docs/backend-sglomni.md @@ -60,7 +60,7 @@ inside the start timeout. | `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | | | `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality | | `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | | -| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + raised generation cap (requests carry `max_new_tokens=12288`; the engine's 2048-frame default is ~27 s and clamps per-request values) | +| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + a raised generation cap (requests carry `max_new_tokens=3000` — upstream pins the thinker engine's context length at 4096, so prompt + generation must fit 4095 tokens, ~40 s of speech — and sub-requests cap at 80 words, with a pre-run popup offering the CHUNK_SIZE clamp) | | `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | | | `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz | | `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` | @@ -108,6 +108,28 @@ Unlike `audio.cpp` (server-side voice presets) the reference clip travels Reference clips live in the project's `voices/` directory (10-20 seconds of clean speech recommended). +### Higgs Audio v3's request window + +The Higgs server pins its thinker engine's context length at 4096 tokens +(not configurable), and the scheduler rejects any request whose prompt — +including the reference-audio tokens — plus `max_new_tokens` exceeds that +window ("Request requires more tokens than the thinker KV cache can hold", +on every GPU: the KV pool side is never the binding constraint). One +request can therefore narrate at most ~40 s of speech, so: + +- Requests carry `max_new_tokens=3000` (the vendored config raises the + engine's 2048-frame default to match), and sub-requests cap at 80 words. + Before a run with Higgs, a popup offers to clamp CHUNK_SIZE for the run + ("Set Chunk to 80"), keep the configured size (audio may cut off + mid-chunk), or cancel; CLI runs answer on the console (`[S/t/c]`, and a + closed stdin clamps so unattended runs never truncate silently). +- When a request is still rejected (a long reference clip, for example), + the client refits `max_new_tokens` to the capacity the server reports + and resends once — the same request never retries as-is. +- Voice cloning with a *long* reference clip eats into the same window: + keep references around 10-20 seconds (as recommended above) for the + most usable generation headroom. + ## Server lifecycle The hub and the CLI start and stop the managed instance around each run @@ -136,6 +158,18 @@ CLI's `--api-url`) at an `sgl-omni` instance. The hub discovers it via model from `GET /v1/models`; models uploaded to that server via `POST /v1/audio/voices` appear in its Voice menu. +## Troubleshooting + +**The generate screen sits on "Status: starting", and the server log says +`Port 8100 is already in use ... Using port 37183 instead`.** Something +else already holds the configured port (often a stale `sgl-omni` from an +earlier attempt). The upstream launcher does not fail — it silently moves +the server to a random port, where no client ever looks for it. The app +detects this in the boot log, aborts the boot, and names both ports. +Find and stop the process holding the port +(`ss -tlnp 'sport = :8100'`), or move this server to a free port +(Settings → SGLang-Omni port), then start again. + ## Manual setup ```bash |