From d04a2c53b926ccde0d582dbf4a7360dc0f072205 Mon Sep 17 00:00:00 2001
From: historia <historiavg@proton.me>
Date: Wed, 2 Sep 2026 22:53:07 -0400
Subject: fix: warn before using a likely too-big chunk size for sglang-omni
 models

---
 app/docs/backend-sglomni.md | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

(limited to 'app/docs')

diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md
index 4f553f9..58e5ce5 100644
--- a/app/docs/backend-sglomni.md
+++ b/app/docs/backend-sglomni.md
@@ -60,7 +60,7 @@ inside the start timeout.
 | `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | |
 | `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality |
 | `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | |
-| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + raised generation cap (requests carry `max_new_tokens=12288`; the engine's 2048-frame default is ~27 s and clamps per-request values) |
+| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + a raised generation cap (requests carry `max_new_tokens=3000` — upstream pins the thinker engine's context length at 4096, so prompt + generation must fit 4095 tokens, ~40 s of speech — and sub-requests cap at 80 words, with a pre-run popup offering the CHUNK_SIZE clamp) |
 | `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | |
 | `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz |
 | `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` |
@@ -108,6 +108,28 @@ Unlike `audio.cpp` (server-side voice presets) the reference clip travels
 Reference clips live in the project's `voices/` directory (10-20 seconds of
 clean speech recommended).
 
+### Higgs Audio v3's request window
+
+The Higgs server pins its thinker engine's context length at 4096 tokens
+(not configurable), and the scheduler rejects any request whose prompt —
+including the reference-audio tokens — plus `max_new_tokens` exceeds that
+window ("Request requires more tokens than the thinker KV cache can hold",
+on every GPU: the KV pool side is never the binding constraint). One
+request can therefore narrate at most ~40 s of speech, so:
+
+- Requests carry `max_new_tokens=3000` (the vendored config raises the
+  engine's 2048-frame default to match), and sub-requests cap at 80 words.
+  Before a run with Higgs, a popup offers to clamp CHUNK_SIZE for the run
+  ("Set Chunk to 80"), keep the configured size (audio may cut off
+  mid-chunk), or cancel; CLI runs answer on the console (`[S/t/c]`, and a
+  closed stdin clamps so unattended runs never truncate silently).
+- When a request is still rejected (a long reference clip, for example),
+  the client refits `max_new_tokens` to the capacity the server reports
+  and resends once — the same request never retries as-is.
+- Voice cloning with a *long* reference clip eats into the same window:
+  keep references around 10-20 seconds (as recommended above) for the
+  most usable generation headroom.
+
 ## Server lifecycle
 
 The hub and the CLI start and stop the managed instance around each run
@@ -136,6 +158,18 @@ CLI's `--api-url`) at an `sgl-omni` instance. The hub discovers it via
 model from `GET /v1/models`; models uploaded to that server via
 `POST /v1/audio/voices` appear in its Voice menu.
 
+## Troubleshooting
+
+**The generate screen sits on "Status: starting", and the server log says
+`Port 8100 is already in use ... Using port 37183 instead`.** Something
+else already holds the configured port (often a stale `sgl-omni` from an
+earlier attempt). The upstream launcher does not fail — it silently moves
+the server to a random port, where no client ever looks for it. The app
+detects this in the boot log, aborts the boot, and names both ports.
+Find and stop the process holding the port
+(`ss -tlnp 'sport = :8100'`), or move this server to a free port
+(Settings → SGLang-Omni port), then start again.
+
 ## Manual setup
 
 ```bash
-- 
cgit v1.2.3