feat: sglang backend support

1 files changed, 177 insertions, 0 deletions

diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md
new file mode 100644
index 0000000..13c8876
--- /dev/null
+++ b/app/docs/backend-sglomni.md
@@ -0,0 +1,177 @@
+# SGLang-Omni Backend
+
+`--backend sglomni` talks to an [SGLang-Omni](https://github.com/sgl-project/sglang-omni)
+server (`sgl-omni serve`), which hosts **one TTS model per server process** behind the
+OpenAI-compatible `POST /v1/audio/speech` endpoint. The backend is installed as a pip
+package into its own managed venv (`app/envs/sglomni`) and its models are pre-downloaded
+into the standard HuggingFace cache — the same flow as the `qwen` and `faster` backends.
+
+## Requirements
+
+- **Linux with an NVIDIA GPU** (recent driver). SGLang-Omni's serving stack is
+  CUDA-only: there are no Windows builds, and Apple Silicon is limited to ASR.
+  The setup wizard warns loudly when `nvidia-smi` does not answer.
+- **Compute capability matters per model.** ZONOS2's default pipeline
+  quantizes its MoE experts to FP8 at load time, and the FP8 Triton kernels
+  only compile on compute capability 8.9+ (RTX 4090/5090, Hopper). On older
+  GPUs (Ampere: RTX 30xx, A100) the managed spec automatically launches the
+  vendored bf16 config (`app/backends/sglomni/configs/zonos2_bf16.yaml`)
+  instead, which runs ZONOS2 in bf16 at roughly twice the MoE VRAM — the
+  status line tags the model `zonos2 (bf16 fallback)` and the install/boot
+  output says so. That config also raises the engine's
+  `mem_fraction_static` to 0.70: the builder's 0.5 default leaves no room
+  for the KV cache once the bf16 weights (~11.5 GB) are resident on a
+  24 GB card, and the server aborts with "Loaded weights leave no GPU
+  memory for the KV cache". Practical floor: a ~16 GB-class card; all
+  stages stay colocated on GPU 0.
+- **Python 3.10, 3.11 or 3.12** for the backend venv. `sglang-omni` requires
+  `>=3.10,<3.13` while the app itself is version-agnostic, so the wizard
+  resolves a compatible interpreter automatically:
+  1. the app venv's interpreter when it is already 3.10-3.12 (no download);
+  2. a `python3.12`/`python3.11`/`python3.10` found on `PATH`;
+  3. otherwise it pip-installs `uv` into the app venv and provisions a managed
+     standalone CPython 3.12 into `app/envs/pythons` (checksum-verified, no root
+     required — this is the zero-prerequisites path on hosts like stock Arch).
+- **Disk space**: the venv alone is ~10 GB (torch, sglang, flash-attn, flashinfer,
+  CUDA-13 wheels); each model adds 1-10 GB of weights in `~/.cache/huggingface/hub`.
+- `ffmpeg` (already a project prerequisite).
+
+The venv pip-installs `sglang-omni` with `--pre` (its dependency stack includes
+prerelease components). The verified version at the time of writing is **0.1.4**.
+
+## Model Catalog
+
+Each catalog entry is one installable model. Both the setup wizard and the
+hub's **Configure Backends… → Configure SGLang-Omni** screen use the same
+checkbox tree (grouped by upstream org, like the audio.cpp one), with the
+already-installed models pre-checked: checking a model installs it — running
+its companion-package recipe, then pre-downloading its weights with the
+venv's `hf` CLI (resumable, cancelable) — and unchecking one removes its
+cached weights after a confirm (a managed server hosting that model is
+stopped first). The whole diff runs as one task-view pass, removals before
+downloads, and the tree re-opens afterwards reflecting the state on disk.
+Models can also be left uninstalled — the first server start for one fetches
+its weights implicitly, but a pre-download keeps the managed server's boot
+inside the start timeout.
+
+| Catalog key | Model | Voice | Notes |
+| --- | --- | --- | --- |
+| `qwen3_tts_0_6b_customvoice` | Qwen3-TTS 0.6B CustomVoice | built-in speakers | lightest model |
+| `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | |
+| `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality |
+| `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | |
+| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | no config file needed |
+| `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | |
+| `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz |
+| `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` |
+| `dots_tts_mf` | dots.tts (MeanFlow) | clone (reference required) | |
+| `fish_s2_pro` | Fish Speech S2-Pro | default voice or clone | needs ~24 GB VRAM (known OOM on a single RTX 3090, upstream issue #359) |
+| `zonos2` | ZONOS2 | clone (reference required) | 44.1 kHz; FP8 pipeline falls back to bf16 on GPUs below compute capability 8.9 |
+
+In the hub's **Generate Audiobooks** form the Model picker reads as a table,
+like the audio.cpp one: each entry's label is padded to the widest one and its
+capabilities are rendered as fixed columns so every capability word lines up
+down its own column — `tts` in the first column (entries that voice plain
+text with a preset or built-in default voice), `clone` in the second (the
+entry clones a reference clip), `design` in the third (Qwen3-TTS VoiceDesign).
+Clone entries that also narrate without a reference (Higgs Audio v3, MOSS-TTS
+Local, Fish Speech S2-Pro) carry both `tts  clone`.
+
+### Companion packages
+
+Some models need extra packages in the backend venv before their server
+starts; installing a model through this tool runs its recipe automatically.
+
+| Model family | Companion packages (as upstream instructs) | System packages |
+| --- | --- | --- |
+| Qwen3-TTS (all four) | `sox`, `einops`, `qwen-tts==0.1.1` — all with `--no-deps`: the qwen-tts demo pins Transformers 4, which would replace sglang-omni's pinned 5.x stack (sglang-omni shims the API differences) | the `sox` **binary** (e.g. `sudo pacman -S sox`, `sudo apt install sox`) |
+| Fish Speech S2-Pro, ZONOS2 | `descript-audiotools==0.7.2`, `descript-audio-codec==1.0.0` (Descript DAC codec) | `ffmpeg` on the server's `PATH` |
+
+A missing system package never blocks a download — the wizard prints the
+remediation and the weights install anyway — but the server will fail to
+synthesize with that model until the package is present.
+
+## How voice cloning works
+
+Unlike `audio.cpp` (server-side voice presets) the reference clip travels
+**with each request** as `ref_audio` + `ref_text`:
+
+- The transcript comes from `--transcription`, or a local Whisper
+  transcription of the clip (the qwen backend's flow). Without any
+  transcript the request is sent without `ref_text`, which lowers quality
+  for models that use it.
+- On a locally-managed (or loopback `--api-url`) server the clip is sent as
+  its **local path** — the server reads the file directly.
+- Against a remote server the clip is **inlined as a base64 data URL**, so
+  `--api-url` runs need no server-side file setup.
+
+Reference clips live in the project's `voices/` directory (10-20 seconds of
+clean speech recommended).
+
+## Server lifecycle
+
+The hub and the CLI start and stop the managed instance around each run
+(like every backend): without `--api-url` the CLI boots the selected model's
+server, converts, and stops it again; a server already answering at the
+configured endpoint is used as-is and left running. Because one process
+hosts one model, a run whose selected model differs from the hosted one
+restarts a server this tool started — a foreign server hosting another
+model refuses the run with an actionable message instead.
+
+The server boots a multi-stage pipeline (preprocessing → TTS generation →
+vocoder) and may pull companion weights on first start, so its start
+timeout is larger than the other backends' (20 minutes). Pre-downloading
+models keeps cold boots well inside it.
+
+External servers: point the **Settings → SGLang-Omni remote URL** (or the
+CLI's `--api-url`) at an `sgl-omni` instance. The hub discovers it via
+`GET /health` (`{"status": "healthy", "stages": [...]}`) and lists its
+model from `GET /v1/models`; models uploaded to that server via
+`POST /v1/audio/voices` appear in its Voice menu.
+
+## Manual setup
+
+```bash
+# 1. A compatible venv (any of: system python3.10-3.12, or uv-managed)
+uv venv --seed --python 3.12 app/envs/sglomni
+
+# 2. The package (into that venv)
+app/envs/sglomni/bin/python -m pip install --pre sglang-omni
+
+# 3. A model's companion packages (example: Qwen3-TTS)
+app/envs/sglomni/bin/python -m pip install --no-deps sox einops
+app/envs/sglomni/bin/python -m pip install --no-deps qwen-tts==0.1.1
+
+# 4. Model weights (example)
+app/envs/sglomni/bin/hf download bosonai/higgs-audio-v3-tts-4b
+
+# 5. Start a server manually (the managed flow does this for you)
+app/envs/sglomni/bin/sgl-omni serve \
+  --model-path bosonai/higgs-audio-v3-tts-4b \
+  --port 8100
+
+# 6. Convert
+python audiobook.py --backend sglomni --model higgs_audio_v3_tts \
+  --api-url http://127.0.0.1:8100
+```
+
+Models that take a vendored config file (all but Higgs and ZONOS2) add
+`--config app/backends/sglomni/configs/<key>.yaml` to the serve command —
+the managed spec builds this from the catalog automatically.
+
+## CLI examples
+
+```bash
+# Zero-shot narration with Higgs Audio v3
+python audiobook.py --backend sglomni --model higgs_audio_v3_tts
+
+# Voice cloning from a reference clip (transcript transcribed locally)
+python audiobook.py --backend sglomni --model moss_tts_local --clone voices/narrator.wav
+
+# Built-in speakers (Qwen3-TTS CustomVoice)
+python audiobook.py --backend sglomni --model qwen3_tts_0_6b_customvoice --voice Vivian
+
+# Voice design
+python audiobook.py --backend sglomni --model qwen3_tts_1_7b_voicedesign \
+  --instructions "A warm adult female narrator with a British accent"
+```