rewrite backend documentation

1 files changed, 42 insertions, 229 deletions

diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md
index 7af6a72..1d5dcda 100644
--- a/app/docs/backend-sglomni.md
+++ b/app/docs/backend-sglomni.md
@@ -1,257 +1,70 @@
-# SGLang-Omni Backend
+# SGLang-Omni (`--backend sglomni`)
 
-`--backend sglomni` talks to an [SGLang-Omni](https://github.com/sgl-project/sglang-omni)
-server (`sgl-omni serve`), which hosts **one TTS model per server process** behind the
-OpenAI-compatible `POST /v1/audio/speech` endpoint. The backend is installed as a pip
-package into its own managed venv (`app/envs/sglomni`) and its models are pre-downloaded
-into the standard HuggingFace cache — the same flow as the `qwen` and `faster` backends.
+Runs an [SGLang-Omni](https://github.com/sgl-project/sglang-omni) server (`sgl-omni serve`), one TTS model per server process. Installs into its own managed venv, `app/envs/sglomni`.
 
-## Requirements
-
-- **Linux with an NVIDIA GPU** (recent driver). SGLang-Omni's serving stack is
-  CUDA-only: there are no Windows builds, and Apple Silicon is limited to ASR.
-  The setup wizard warns loudly when `nvidia-smi` does not answer.
-- **Compute capability matters per model.** ZONOS2's default pipeline
-  quantizes its MoE experts to FP8 at load time, and the FP8 Triton kernels
-  only compile on compute capability 8.9+ (RTX 4090/5090, Hopper). On older
-  GPUs (Ampere: RTX 30xx, A100) the managed spec automatically launches the
-  vendored bf16 config (`app/backends/sglomni/configs/zonos2_bf16.yaml`)
-  instead, which runs ZONOS2 in bf16 at roughly twice the MoE VRAM — the
-  status line tags the model `zonos2 (bf16 fallback)` and the install/boot
-  output says so. That config also raises the engine's
-  `mem_fraction_static` to 0.70: the builder's 0.5 default leaves no room
-  for the KV cache once the bf16 weights (~11.5 GB) are resident on a
-  24 GB card, and the server aborts with "Loaded weights leave no GPU
-  memory for the KV cache". Practical floor: a ~16 GB-class card; all
-  stages stay colocated on GPU 0.
-- **Python 3.10, 3.11 or 3.12** for the backend venv. `sglang-omni` requires
-  `>=3.10,<3.13` while the app itself is version-agnostic, so the wizard
-  resolves a compatible interpreter automatically:
-  1. the app venv's interpreter when it is already 3.10-3.12 (no download);
-  2. a `python3.12`/`python3.11`/`python3.10` found on `PATH`;
-  3. otherwise it pip-installs `uv` into the app venv and provisions a managed
-     standalone CPython 3.12 into `app/envs/pythons` (checksum-verified, no root
-     required — this is the zero-prerequisites path on hosts like stock Arch).
-- **Disk space**: the venv alone is ~10 GB (torch, sglang, flash-attn, flashinfer,
-  CUDA-13 wheels); each model adds 1-10 GB of weights in `~/.cache/huggingface/hub`.
-- `ffmpeg` (already a project prerequisite).
-
-The venv pip-installs `sglang-omni` with `--pre` (its dependency stack includes
-prerelease components). The verified version at the time of writing is **0.1.4**.
+Run all commands from the project root (next to `audiobook.py`).
 
-## Model Catalog
+## Requirements
 
-Each catalog entry is one installable model. Both the setup wizard and the
-hub's **Configure Backends… → Configure SGLang-Omni** screen use the same
-checkbox tree (grouped by upstream org, like the audio.cpp one), with the
-already-installed models pre-checked: checking a model installs it — running
-its companion-package recipe, then pre-downloading its weights with the
-venv's `hf` CLI (resumable, cancelable) — and unchecking one removes its
-cached weights after a confirm (a managed server hosting that model is
-stopped first). The whole diff runs as one task-view pass, removals before
-downloads, and the tree re-opens afterwards reflecting the state on disk.
-Models can also be left uninstalled — the first server start for one fetches
-its weights implicitly, but a pre-download keeps the managed server's boot
-inside the start timeout.
+- Linux with an NVIDIA GPU
+- Python 3.10-3.12 (`sglang-omni` requires `>=3.10,<3.13`)
+- ~10 GB disk for the venv plus 1-10 GB per model of weights in `~/.cache/huggingface/hub`
 
-| Catalog key | Model | Voice | Notes |
-| --- | --- | --- | --- |
-| `qwen3_tts_0_6b_customvoice` | Qwen3-TTS 0.6B CustomVoice | built-in speakers | lightest model |
-| `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | |
-| `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality |
-| `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | |
-| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + a raised generation cap (requests carry `max_new_tokens=3000` — upstream pins the thinker engine's context length at 4096, so prompt + generation must fit 4095 tokens, ~40 s of speech — and sub-requests cap at 80 words, with a pre-run popup offering the CHUNK_SIZE clamp) |
-| `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | |
-| `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz |
-| `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` |
-| `dots_tts_mf` | dots.tts (MeanFlow) | clone (reference required) | |
-| `fish_s2_pro` | Fish Speech S2-Pro | default voice or clone | needs ~24 GB VRAM (known OOM on a single RTX 3090, upstream issue #359) |
-| `zonos2` | ZONOS2 | clone (reference required) | 44.1 kHz; FP8 pipeline falls back to bf16 on GPUs below compute capability 8.9; requests carry `max_new_tokens=12288` because the engine's 1024-frame default caps one request at ~12 s of speech |
+## Install
 
-In the hub's **Generate Audiobooks** form the Model picker reads as a table,
-like the audio.cpp one: each entry's label is padded to the widest one and its
-capabilities are rendered as fixed columns so every capability word lines up
-down its own column — `tts` in the first column (entries that voice plain
-text with a preset or built-in default voice), `clone` in the second (the
-entry clones a reference clip), `design` in the third (Qwen3-TTS VoiceDesign).
-Clone entries that also narrate without a reference (Higgs Audio v3, MOSS-TTS
-Local, Fish Speech S2-Pro) carry both `tts  clone`.
+1. Create the venv and install the server:
 
-### Companion packages
+```bash
+uv venv --seed --python 3.12 app/envs/sglomni   # or: python3.12 -m venv app/envs/sglomni
+app/envs/sglomni/bin/python -m pip install --pre sglang-omni
+```
 
-Some models need extra packages in the backend venv before their server
-starts; installing a model through this tool runs its recipe automatically.
+2. Install companion packages for the models you plan to host:
 
-| Model family | Companion packages (as upstream instructs) | System packages |
+| Models | Into `app/envs/sglomni` | System packages |
 | --- | --- | --- |
-| Qwen3-TTS (all four) | `sox`, `einops`, `qwen-tts==0.1.1` — all with `--no-deps`: the qwen-tts demo pins Transformers 4, which would replace sglang-omni's pinned 5.x stack (sglang-omni shims the API differences) | the `sox` **binary** (e.g. `sudo pacman -S sox`, `sudo apt install sox`) |
-| Fish Speech S2-Pro, ZONOS2 | `descript-audiotools==0.7.2`, `descript-audio-codec==1.0.0` (Descript DAC codec) | `ffmpeg` on the server's `PATH` |
-
-A missing system package never blocks a download — the wizard prints the
-remediation and the weights install anyway — but the server will fail to
-synthesize with that model until the package is present.
-
-## How voice cloning works
-
-Unlike `audio.cpp` (server-side voice presets) the reference clip travels
-**with each request** as `ref_audio` + `ref_text`:
-
-- The transcript comes from `--transcription`, or a local Whisper
-  transcription of the clip (the qwen backend's flow). Without any
-  transcript the request is sent without `ref_text`, which lowers quality
-  for models that use it.
-- On a locally-managed (or loopback `--api-url`) server the clip is sent as
-  its **local path** — the server reads the file directly.
-- Against a remote server the clip is **inlined as a base64 data URL**, so
-  `--api-url` runs need no server-side file setup.
-
-Reference clips live in the project's `voices/` directory (10-20 seconds of
-clean speech recommended).
-
-### Higgs Audio v3's request window
-
-The Higgs server pins its thinker engine's context length at 4096 tokens
-(not configurable), and the scheduler rejects any request whose prompt —
-including the reference-audio tokens — plus `max_new_tokens` exceeds that
-window ("Request requires more tokens than the thinker KV cache can hold",
-on every GPU: the KV pool side is never the binding constraint). One
-request can therefore narrate at most ~40 s of speech, so:
-
-- Requests carry `max_new_tokens=3000` (the vendored config raises the
-  engine's 2048-frame default to match), and sub-requests cap at 80 words.
-  Before a run with Higgs, a popup offers to clamp CHUNK_SIZE for the run
-  ("Set Chunk to 80"), keep the configured size (audio may cut off
-  mid-chunk), or cancel; CLI runs answer on the console (`[S/t/c]`, and a
-  closed stdin clamps so unattended runs never truncate silently).
-- When a request is still rejected (a long reference clip, for example),
-  the client refits `max_new_tokens` to the capacity the server reports
-  and resends once — the same request never retries as-is.
-- Voice cloning with a *long* reference clip eats into the same window:
-  keep references around 10-20 seconds (as recommended above) for the
-  most usable generation headroom.
-
-## Server lifecycle
-
-The hub and the CLI start and stop the managed instance around each run
-(like every backend): without `--api-url` the CLI boots the selected model's
-server, converts, and stops it again; a server already answering at the
-configured endpoint is used as-is and left running. Because one process
-hosts one model, a run whose selected model differs from the hosted one
-restarts a server this tool started — a foreign server hosting another
-model refuses the run with an actionable message instead.
+| All four Qwen3-TTS | `bin/python -m pip install --no-deps sox einops qwen-tts==0.1.1` | `sox` binary (e.g. `sudo apt install sox`) |
+| Fish Speech S2-Pro, ZONOS2 | `bin/python -m pip install descript-audiotools==0.7.2 descript-audio-codec==1.0.0 protobuf==6.33.6` | `ffmpeg` on PATH |
 
-The hub's **Start/Stop Backend Servers** menu asks which *downloaded* model
-to load when starting the server fresh (with none downloaded the entry says
-so and points at the Configure screen; a single model starts without
-asking). A start also auto-installs a model's missing companion packages
-first, so a boot never dies on their import even when the weights arrived
-via the shared HuggingFace cache or an earlier install's pip run failed.
-
-The server boots a multi-stage pipeline (preprocessing → TTS generation →
-vocoder) and may pull companion weights on first start, so its start
-timeout is larger than the other backends' (20 minutes). Pre-downloading
-models keeps cold boots well inside it.
-
-External servers: point the **Settings → SGLang-Omni remote URL** (or the
-CLI's `--api-url`) at an `sgl-omni` instance. The hub discovers it via
-`GET /health` (`{"status": "healthy", "stages": [...]}`) and lists its
-model from `GET /v1/models`; models uploaded to that server via
-`POST /v1/audio/voices` appear in its Voice menu.
-
-## Troubleshooting
-
-**The generate screen sits on "Status: starting", and the server log says
-`Port 8100 is already in use ... Using port 37183 instead`.** Something
-else already holds the configured port (often a stale `sgl-omni` from an
-earlier attempt). The upstream launcher does not fail — it silently moves
-the server to a random port, where no client ever looks for it. The app
-detects this in the boot log, aborts the boot, and names both ports.
-Find and stop the process holding the port
-(`ss -tlnp 'sport = :8100'`), or move this server to a free port
-(Settings → SGLang-Omni port), then start again.
-
-**The first request dies with `HTTP 500: CUDA out of memory. Tried to
-allocate ~100 MiB` (and every retry fails identically) on a 24 GB card,
-with the model far smaller than the card.** The model size is not the
-problem: upstream's Qwen3-TTS, MOSS-TTS and Voxtral pipelines colocate
-their AR engine and vocoder on GPU 0 with the engine's sglang
-`mem_fraction_static` unset, so the static pool (weights + KV cache) is
-auto-sized to nearly all free VRAM at boot. By the time a real request
-runs, the engine's CUDA graphs (~1.5 GB) and the colocated vocoder are
-resident too and only tens of MiB are left — the transient allocations
-generation needs do not fit. The vendored configs for those models pin
-the pool exactly for this — 0.70 for the small ones (~7 GB of headroom
-on a 24 GB card); Higgs (`gpu_memory_fraction: 0.80`) and ZONOS2's
-bf16 fallback (`0.70`) carry the same treatment. MOSS-TTS is the
-exception: its ~17.1 GB of v1.5 bf16 weights alone exceed the 0.70
-budget on the ~20.7 GB the engine profiler sees at that point (its
-colocated audio-tokenizer stage is already resident), so a 0.70 pin
-aborts the boot outright with "Loaded weights leave no GPU memory for
-the KV cache" (the floor is ~0.828 on a clean 24 GB card) instead of
-fixing the request-time OOM — it runs `mem_fraction_static: 0.87`,
-keeping a ~0.8 GB KV pool with ~2.7 GB of slack for the CUDA graphs
-and the colocated vocoder. If you hit the failure
-on a server started by hand or an older checkout, pin the budget on the
-command line and let this tool use the running server as-is:
+3. Download model weights:
 
 ```bash
-app/envs/sglomni/bin/sgl-omni serve \
-  --model-path Qwen/Qwen3-TTS-12Hz-0.6B-Base \
-  --config app/backends/sglomni/configs/qwen3_tts_0_6b.yaml \
-  --port 8100 --mem-fraction-static 0.70
+app/envs/sglomni/bin/hf download <hf-repo>      # repos in the table below
 ```
 
-(MOSS-TTS Local needs none of this: its upstream config already budgets
-its colocated stages explicitly. Fish Speech S2-Pro's OOM is a different
-problem — the model itself needs ~24 GB, upstream issue #359.)
+## Models
 
-## Manual setup
+| Key (TAG `--model`) | HF repo (`--model-path`) | Config |
+| --- | --- | --- |
+| `qwen3_tts_0_6b_customvoice` | `Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice` | `qwen3_tts_0_6b_customvoice.yaml` |
+| `qwen3_tts_0_6b_base` | `Qwen/Qwen3-TTS-12Hz-0.6B-Base` | `qwen3_tts_0_6b.yaml` |
+| `qwen3_tts_1_7b_base` | `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | `qwen3_tts_1_7b.yaml` |
+| `qwen3_tts_1_7b_voicedesign` | `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | `qwen3_tts_1_7b_voicedesign.yaml` |
+| `higgs_audio_v3_tts` | `bosonai/higgs-audio-v3-tts-4b` | `higgs_audio_v3_tts.yaml` |
+| `moss_tts` | `OpenMOSS-Team/MOSS-TTS-v1.5` | `moss_tts.yaml` |
+| `moss_tts_local` | `OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5` | `moss_tts_local.yaml` |
+| `voxtral_tts` | `mistralai/Voxtral-4B-TTS-2603` | `voxtral_tts.yaml` |
+| `dots_tts_mf` | `dots-studio/dots.tts-mf` | `dots_tts.yaml` |
+| `fish_s2_pro` | `fishaudio/s2-pro` | `s2pro_tts.yaml` |
+| `zonos2` | `Zyphra/zonos2` | none |
+
+Config files live in `app/backends/sglomni/configs/`. ZONOS2 runs without a config; on GPUs below compute capability 8.9 (RTX 30xx, A100) pass `--config app/backends/sglomni/configs/zonos2_bf16.yaml` instead.
+
+## Run the server
 
 ```bash
-# 1. A compatible venv (any of: system python3.10-3.12, or uv-managed)
-uv venv --seed --python 3.12 app/envs/sglomni
-
-# 2. The package (into that venv)
-app/envs/sglomni/bin/python -m pip install --pre sglang-omni
-
-# 3. A model's companion packages (example: Qwen3-TTS)
-app/envs/sglomni/bin/python -m pip install --no-deps sox einops
-app/envs/sglomni/bin/python -m pip install --no-deps qwen-tts==0.1.1
-
-# 4. Model weights (example)
-app/envs/sglomni/bin/hf download bosonai/higgs-audio-v3-tts-4b
-
-# 5. Start a server manually (the managed flow does this for you)
+# Example: Higgs Audio v3
 app/envs/sglomni/bin/sgl-omni serve \
   --model-path bosonai/higgs-audio-v3-tts-4b \
   --config app/backends/sglomni/configs/higgs_audio_v3_tts.yaml \
   --port 8100
 
-# 6. Convert
-python audiobook.py --backend sglomni --model higgs_audio_v3_tts \
-  --api-url http://127.0.0.1:8100
+# ZONOS2 (no config)
+app/envs/sglomni/bin/sgl-omni serve --model-path Zyphra/zonos2 --port 8100
 ```
 
-Models that take a vendored config file (all but ZONOS2) add
-`--config app/backends/sglomni/configs/<key>.yaml` to the serve command —
-the managed spec builds this from the catalog automatically. Higgs's config
-also exists for a reason: the upstream pipeline budgets 98% of the card
-across its colocated stages, which OOMs on a 24 GB GPU once anything else
-uses VRAM, and its engine caps requests at ~27 s of speech.
-
-## CLI examples
+## Use it
 
 ```bash
-# Zero-shot narration with Higgs Audio v3
-python audiobook.py --backend sglomni --model higgs_audio_v3_tts
-
-# Voice cloning from a reference clip (transcript transcribed locally)
-python audiobook.py --backend sglomni --model moss_tts_local --clone voices/narrator.wav
-
-# Built-in speakers (Qwen3-TTS CustomVoice)
-python audiobook.py --backend sglomni --model qwen3_tts_0_6b_customvoice --voice Vivian
-
-# Voice design
-python audiobook.py --backend sglomni --model qwen3_tts_1_7b_voicedesign \
-  --instructions "A warm adult female narrator with a British accent"
+python audiobook.py --backend sglomni --api-url http://127.0.0.1:8100 --model <key>
 ```