aboutsummaryrefslogtreecommitdiff
path: root/app/docs
diff options
context:
space:
mode:
authorhistoria <historiavg@proton.me>2026-09-04 12:41:24 -0400
committerhistoria <historiavg@proton.me>2026-09-04 12:41:24 -0400
commit1d59b3a412caaf0fdb982f2b6481c38b8a8a25d3 (patch)
tree8c83550d07ac25ef197714c81dd098bb89476c27 /app/docs
parenteadae12c3a8f31d16586a6241bfc3a90461522cc (diff)
downloadtts-audiobook-generator-1d59b3a412caaf0fdb982f2b6481c38b8a8a25d3.tar.gz
rewrite backend documentation
Diffstat (limited to 'app/docs')
-rw-r--r--app/docs/backend-audiocpp.md122
-rw-r--r--app/docs/backend-faster.md45
-rw-r--r--app/docs/backend-qwen.md85
-rw-r--r--app/docs/backend-sglomni.md271
4 files changed, 115 insertions, 408 deletions
diff --git a/app/docs/backend-audiocpp.md b/app/docs/backend-audiocpp.md
index aff8b12..b0367e1 100644
--- a/app/docs/backend-audiocpp.md
+++ b/app/docs/backend-audiocpp.md
@@ -1,32 +1,33 @@
-# Backend Option 1: audio.cpp
+# audio.cpp (`--backend audiocpp`)
-`--backend audiocpp` talks to `audiocpp_server` from [audio.cpp](https://github.com/0xShug0/audio.cpp), which hosts numerous TTS model families.
+Runs [audio.cpp](https://github.com/0xShug0/audio.cpp)'s `audiocpp_server`, a C++ TTS server hosting many model families. Installs into the `app/audio.cpp` checkout.
-The easiest way is the TUI: run `python audiobook.py`, choose **Configure Backends… → Install Backend → audio.cpp**, and it clones `audio.cpp` into `app/audio.cpp` (or reuses an existing checkout), installs `audiocpp_server` — by default downloading the prebuilt release binary on macOS and Windows (no compiler, no Xcode needed), or building from source elsewhere — lets you pick model families/packages from an expandable checkbox tree (reading the checkout's `model_specs/`), transcribes `.wav` voices with `whisper`, writes `server.json` into the checkout, syncs `app/converter/config.py`, and prints the launch command (the hub can also start the server for you via the **Start/Stop Backend Servers** menu or automatically when converting). The setup asks exactly two screens: first the model tree, then one combined options form (like **Generate Audiobooks**) for everything else — the inference backend and how to get the server binary (prebuilt download, source build, or skip), the voice-clone `.wav` directory and how to transcribe it, automatic model download, the default-model sync, and deleting models dropped on a re-run; rows that do not apply to your selection are hidden. The server always binds `127.0.0.1` on the port configured in `AUDIOCPP_API_URL` (edit it in **Settings**), so neither is ever asked. The clone, install, transcription and model downloads all run inside the TUI — each shows a status (and, where the tool can measure it, a progress bar), and can be cancelled — instead of dropping to console output. Run it directly with `python -m backends.audiocpp` from `app/` — flags like `--wavs`, `--families`, `--build-backend`, `--prebuilt auto|yes|no`, `--clone` skip the corresponding parts for scripting. The TUI runs in the managed `app/envs/tts` venv, which installs faster-whisper when wheels exist for your platform (it is tagged optional in `requirements.txt`: on platforms without compatible builds the setup skips it and voice-clone transcription degrades to manual transcripts). For a manual setup, make sure `whisper` or `faster_whisper` is installed in the environment you run the wizard from. The Qwen3-TTS model tree also offers hosting the VoiceDesign package as a `vdes` entry.
+Run all commands from the project root (next to `audiobook.py`).
-The hub's backend status table distinguishes how far audio.cpp is set up: `unavailable` (nothing present), `downloaded (not built)` (checkout cloned, `audiocpp_server` not installed), `built (not configured)` (binary installed, no `server.json`), `installed` (ready; or `installed (models missing)` when the config references undownloaded models), and `running` once its server answers. Whenever the checkout exists but `audiocpp_server` is missing, **Configure Backends… → Build audio.cpp Server** installs it from the TUI — asking whether to download the prebuilt release (recommended, macOS and Windows) or build from source (the wizard offers the same choice during setup), so a backend whose install you skipped is never stuck as "unavailable". On a fresh install the setup is one continuous flow: clone → configure → and then the install and the model downloads run **simultaneously** in a split view (half downloading/building, half downloading models). The setup steps are therefore ordered install > configure > download, and **Build audio.cpp Server** and **Download Missing Models (audio.cpp)** are never offered at the same time; **Build audio.cpp Server** downloads any missing models alongside the install, and **Download Missing Models (audio.cpp)** remains only as a fallback for when a download fails or is interrupted.
+## Requirements
-Prebuilt installs are tracked with a `prebuilt.json` marker inside the build directory. **Update Backends** then skips the git pull/rebuild flow for those and instead re-downloads when upstream publishes a newer release (and checks the checkout out at the release's tag, keeping its model catalog and tooling in sync with the binary). When that re-download fails (usually GitHub's API rate limit), **Update Backends** falls back to the same source-build route a source-built checkout uses — the installed binary is only replaced once a new one is in place, so it keeps working meanwhile — and a successful fallback build removes the now-stale marker so later updates take the git + rebuild route. A source-built checkout keeps updating by git pull + rebuild.
+- git
+- A C++ toolchain (gcc/clang + cmake on Linux, full Xcode on macOS, VS Build Tools with the C++ workload on Windows)
+- CUDA / Vulkan / HIP for GPU builds (or CPU)
-If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up.
+## Install
-### Download or build audiocpp_server
+1. Clone the repo:
-The wizard's default on macOS and Windows is to download a prebuilt `audiocpp_server` from [audio.cpp's releases](https://github.com/0xShug0/audio.cpp/releases) (checksum-verified, ~20–25 MB, no compiler or Xcode required — on Intel Macs the release runs on CPU only, and Windows CUDA builds come in 12.4/13.3 variants picked to match your NVIDIA driver). The newest version is resolved without GitHub's API, so the check rarely hits the API's rate limit; if the API *is* rate-limited when a download starts, the install proceeds without checksum verification (with a loud warning). If the download still fails, the TUI does not leave you half-installed: the same Build lane automatically falls back to building from source (a cancelled download, or an explicit `--prebuilt yes` — which exists to forbid source builds — do not fall back). **Update Backends** keeps a prebuilt install current automatically, and falls back to a source build the same way when its re-download fails. The manual build steps, if you prefer building from source, are below — build for your platform and backend `(cuda, vulkan, hip, cpu)`. Check [audio.cpp's readme](https://github.com/0xShug0/audio.cpp) for details. audio.cpp ships one helper script per platform, and the hub runs the one matching your OS:
+```bash
+git clone https://github.com/0xShug0/audio.cpp app/audio.cpp
+cd app/audio.cpp
+```
+
+2. Build `audiocpp_server` (macOS and Windows can instead download a prebuilt binary from [releases](https://github.com/0xShug0/audio.cpp/releases)):
```bash
-# Linux
-git clone https://github.com/0xShug0/audio.cpp
-cd audio.cpp
+# Linux (--backend: cuda, vulkan, hip, cpu)
scripts/build_linux.sh --backend cuda --target audiocpp_server --deployment-build
```
```bash
-# macOS (Metal is the only buildable backend there; the hub records it as "cpu").
-# Needs full Xcode (build_metal.sh requires its offline Metal compiler). Without
-# Xcode, the hub builds with cmake directly — Apple's Command Line Tools
-# (clang) plus cmake suffice, because the Metal shaders are then compiled by
-# macOS itself at runtime on first use:
+# macOS (Metal)
cmake -S . -B build/macos-metal-release -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DENGINE_ENABLE_METAL=ON -DENGINE_ENABLE_CUDA=OFF -DENGINE_ENABLE_VULKAN=OFF \
-DENGINE_ENABLE_OPENMP=OFF -DGGML_METAL_EMBED_LIBRARY=ON \
@@ -35,29 +36,19 @@ cmake --build build/macos-metal-release --parallel $(sysctl -n hw.logicalcpu) --
```
```powershell
-# Windows (needs VS Build Tools with the C++ workload; CUDA preset also needs
-# the CUDA Toolkit, Vulkan preset the Vulkan SDK — the script names what's missing)
+# Windows (presets: windows-cpu-release, windows-vulkan-release, windows-cuda-release)
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build_windows.ps1 -Preset windows-cuda-release -Target audiocpp_server -DeploymentBuild
-# presets: windows-cpu-release, windows-vulkan-release, windows-cuda-release (HIP: scripts\build_windows_hip.ps1)
```
-`--deployment-build` compiles the `model_specs/` catalog into the binary so every family resolves its model contract even when the server is started outside the checkout (GGUF packages whose embedded spec is legacy, such as today's Qwen3-TTS ones, otherwise need `model_specs/<family>.json` found relative to the working directory). The macOS script takes the same flag; the Windows PowerShell scripts take `-DeploymentBuild`.
-
-### Install models
-
-Download model packages with the python model manager script from the audio.cpp checkout. Each installs to `./models`. Here are two examples, Higgs Audio and Qwen3-TTS:
+3. Download models:
```bash
-python tools/model_manager_v2.py install higgs_audio_tts_4b_q8_0
+python tools/model_manager_v2.py list # see available models
python tools/model_manager_v2.py install qwen3_tts_1_7b_base_q8_0
python tools/model_manager_v2.py install qwen3_tts_1_7b_customvoice_q8_0
```
-You can run `python tools/model_manager_v2.py list` to see all available models. Inside the tool's setup the models are downloaded automatically from the TUI (models already on disk are reported and skipped); if you need to download them manually instead, decline the automatic download and it prints these commands for only the models that are still missing.
-
-### Create server.json
-
-Create a `server.json` config file. One server can host multiple models and multiple cloned voices. The `id:` fields are the model names you will set for `tts-audiobook-generator` with `--model` (the setup wizard names each entry after its model package directory).
+4. Create `server.json` in the checkout:
```json
{
@@ -68,13 +59,6 @@ Create a `server.json` config file. One server can host multiple models and mult
"voice_dir": "/path/to/clone/wavs",
"models": [
{
- "id": "Higgs-Audio-v3-TTS-4B-GGUF",
- "family": "higgs_audio_tts",
- "path": "models/Higgs-Audio-v3-TTS-4B-GGUF",
- "task": "tts",
- "mode": "offline"
- },
- {
"id": "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF",
"family": "qwen3_tts",
"path": "models/Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF",
@@ -99,68 +83,20 @@ Create a `server.json` config file. One server can host multiple models and mult
}
```
-The `task` field decides which session type the server creates for the entry, and model families are picky about it: clone-only families — **Chatterbox**, **Confucius4-TTS** and **Echo-TTS** — reject plain-TTS sessions outright (every speech request fails with an HTTP 500 like `Chatterbox supports VoiceCloning and VoiceConversion`), so they must be hosted with `"task": "clon"`. Voice-design packages are hosted with `"task": "vdes"`; every other family keeps `"task": "tts"`, which serves plain TTS and (where the family supports it) voice cloning through the request's `voice` field alike. The setup wizard writes the right task automatically, and opening **Generate Audiobooks** re-hosts stale `tts`-tasked clone-only entries in an existing `server.json` (restarting the managed server to load the fix).
+- `id` is the value passed as `--model` to TAG.
+- `task`: `tts` for plain TTS/clone families, `clon` for clone-only families (Chatterbox, Confucius4-TTS, Echo-TTS), `vdes` for voice-design packages.
+- Voices referenced in `voice_dir` need a transcript file `prompt_text` alongside the wavs, one `<voice>|<transcript>` line per wav.
-### Run audio.cpp and the audiobook script
-
-Run the server with this config file. The `audiocpp_server` path will be slightly different depending on your platform and build options:
+## Run the server
```bash
./build/linux-cuda-release/bin/audiocpp_server --config server.json
```
-In a different terminal, run `audiobook.py`. Pick the TTS `--model` and `--voice` from server.json:
-
-```bash
-# Higgs Audio voice cloning (the family also does plain TTS: omit --voice)
-python audiobook.py --backend audiocpp --model Higgs-Audio-v3-TTS-4B-GGUF --voice narrator
+(The binary path differs by platform/build preset.)
-# Chatterbox (clone-only: a reference voice is required)
-python audiobook.py --backend audiocpp --model Chatterbox-GGUF --voice narrator
+## Use it
-# Supertonic (pure TTS: no voice needed, none can be cloned)
-python audiobook.py --backend audiocpp --model Supertonic-GGUF
-
-# Qwen3-TTS built-in speaker (pick one with --voice)
-python audiobook.py --backend audiocpp --model Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF --voice Vivian
-
-# Qwen3-TTS voice cloning
-python audiobook.py --backend audiocpp --model Qwen3-TTS-12Hz-1.7B-Base-GGUF --voice narrator
-
-# Qwen-TTS voice design
-python audiobook.py --backend audiocpp --model Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF \
- --instructions "A warm adult female narrator with a British accent"
-
-# Per-model options passed through to the family (--option KEY=VALUE, repeatable)
-python audiobook.py --backend audiocpp --model <id> --voice narrator \
- --option emotion=neutral --option speed=1.1
+```bash
+python audiobook.py --backend audiocpp --api-url http://127.0.0.1:8080 --model <id> --voice <name>
```
-
-### Voice cloning transcripts (prompt_text)
-
-Some clone families require the **transcript of the reference audio** with
-every request (Qwen3-TTS's Base model clones in ICL mode and rejects each
-request without it: "voice clone ICL mode requires reference text"). The
-server supplies the transcript itself, from either of two places:
-
-- `prompt_text` in the configured `voice_dir`: one `<voice>|<transcript>` line
- per wav (the setup wizard writes it from whisper transcriptions), or
-- `reference_text` on a `voice_presets` entry in `server.json`.
-
-An **empty or missing transcript makes every clone request for that voice
-fail**, so `audiobook.py` aborts on the first chunk with instructions instead
-of retrying. The setup warns loudly about any voice whose transcript came
-back empty — fill those in before converting (edit `prompt_text` by hand and
-restart nothing: the server re-reads it per request), or rerun the audio.cpp
-setup to re-transcribe. A re-transcription that fails keeps any existing
-non-empty transcript rather than overwriting it with a blank.
-
-If accurate transcripts are not available, cloning without one is possible
-per run with `--option x_vector_only_mode=true` (speaker-embedding-only
-cloning — no transcript needed, noticeably lower speaker similarity).
-
-In the hub's **Generate Audiobooks** form the Model picker reads as a table: each entry's id is padded to the widest one and its capabilities are rendered as fixed columns — `tts` in the first column (families that need no voice at all, or take a built-in speaker / preset voice), `clone` (the entry clones a reference voice) in the second, `design` (the entry can design a voice from an Instructions description) in the third — so every capability word lines up down its own column. The `design` column is filled for `vdes` design-model entries and for families whose audio.cpp spec advertises design (e.g. OmniVoice, VoxCPM2); Qwen3-TTS designs only through its separate VoiceDesign entry, so its Base/CustomVoice rows stay without it. The Voice field is labelled **Built-in voice** on CustomVoice entries (listing the model's speakers) and **Voice to clone** on clone-capable entries (listing the server's preset/voice_dir entries) — it is hidden entirely on pure-TTS families, and on mixed tts+clone families it leads with a **&lt;built-in&gt; (no clone)** pick that means plain TTS with the model's own default voice (no reference cloned; the default). Clone-only families keep the voice required. Instructions are shown for every entry: required for `vdes` design models, an optional style/delivery instruction elsewhere — and on families without built-in speakers that read instructions, a description alone can define the voice, so leaving Voice empty is fine there. A Request options field accepts the same `KEY=VALUE` items as `--option`, and appears only for model families whose audio.cpp checkout spec declares request options (e.g. Neutts, Outetts, F5-TTS — not Qwen3-TTS or Higgs Audio). Editing Instructions or Request options shows a short dim hint with an example (for Instructions: `"Speak in a calm, soothing, and happy tone."`). Language is a static picker over the languages of audio.cpp's WebUI menus (it shows the same "Check model documentation for supported languages." hint while editing) and overrides the global setting for this run only.
-
-The hub also works with an audio.cpp server that runs somewhere else (another checkout, another machine): set `AUDIOCPP_REMOTE_URL` in `app/converter/config.py` (or the TUI **Settings** → "audio.cpp remote URL") to its `host:port`. The hub probes that URL and, when it answers, offers an `audio.cpp [remote]` entry in **Generate Audiobooks…** whose models and voices are queried live (`GET /v1/models` and `GET /v1/audio/voices`) — alongside the managed `audio.cpp` entry, which keeps reading the local `server.json`. The remote URL defaults to `127.0.0.1:8080`, so a server started outside this tool on the local port is found automatically. On the CLI, pass `--api-url http://host:port` (and `--model`/`--voice` matching that server's config).
-
-Before converting, `audiobook.py` asks the server to unload all currently loaded models (`POST /v1/tasks/unload_all_models`) so models left resident by earlier runs free their memory (e.g. VRAM on GPU backends) and only the selected entry loads. A server without that endpoint, or one busy unloading, only produces a warning. This behavior is controlled by the **Settings** → "Unload models" option (or `AUDIOCPP_UNLOAD_MODELS` in `app/converter/config.py`), which defaults to **Yes**; set it to **No** to keep other models resident across runs.
diff --git a/app/docs/backend-faster.md b/app/docs/backend-faster.md
index 5de337d..e7bcc8f 100644
--- a/app/docs/backend-faster.md
+++ b/app/docs/backend-faster.md
@@ -1,32 +1,32 @@
-# Backend Option 3: faster-qwen-tts
+# faster-qwen3-tts (`--backend faster`)
-`--backend faster` talks to the OpenAI-compatible server from [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts), which uses CUDA graph capture for roughly 5-10x faster inference with the same models. **It requires an NVIDIA GPU**.
+Runs the OpenAI-compatible demo server from [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts). The package installs into its own managed venv (`app/envs/faster`), the repo is cloned to `app/faster-qwen3-tts`.
-The easiest way is to run `python audiobook.py` → **Configure Backends… → Install Backend → faster-qwen3-tts** (or `python app/backends/faster.py path/to/clone/wavs`): the TUI pip-installs `faster-qwen3-tts[demo]` into its own managed venv (`app/envs/faster`, separate from the app's venv and from the qwen backend's — both TTS stacks ship conflicting versions of a shared `qwen_tts` module; the faster wheel pulls its own `qwen-tts-hf` build of it automatically), clones the repo, transcribes the `.wav` files with whisper (faster-whisper, installed when wheels exist for your platform — otherwise you type the transcripts), and writes `voices.json` for you — all on one options screen (voices directory, language, whisper model, and what to re-transcribe on a modify run). The server port is not asked: it lives in `FASTER_API_URL` (edit it in **Settings**). You can also start the server from the hub's **Start/Stop Backend Servers** menu, or let a conversion start it automatically.
+Run all commands from the project root (next to `audiobook.py`).
-If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up. To use a server on another machine, set `FASTER_REMOTE_URL` in `app/converter/config.py` to its `host:port` (default `127.0.0.1:8000`) — the hub probes it and offers a `faster-qwen3-tts [remote]` entry — or pass `--api-url` on the CLI.
+## Requirements
-Install into your environment (this backend does **not** need the `qwen-tts`
-pip package — the wheel pulls the compatible `qwen-tts-hf` build of the
-`qwen_tts` library automatically, so keep it out of any venv that also has
-upstream `qwen-tts` installed):
+- NVIDIA GPU
+- git
+
+## Install
+
+1. Venv and package:
```bash
-python -m venv audiobook-faster
-source audiobook-faster/bin/activate
-pip install "faster-qwen3-tts[demo]"
+python3 -m venv app/envs/faster
+app/envs/faster/bin/pip install "faster-qwen3-tts[demo]"
```
-**This backend always uses voice cloning**. The reference voice and language are configured on the **server**, not through the converter. The server does not transcribe reference audio itself, so do it manually or use the `backends.faster` setup wizard (see below).
+Do not install upstream `qwen-tts` into this venv — the two ship conflicting `qwen_tts` modules.
-The pip package does not include the server script, so clone the repository (the `backends.faster` wizard does this for you into `./faster-qwen3-tts`):
+2. Clone the repo (the pip package does not include the server script):
```bash
-git clone https://github.com/andimarafioti/faster-qwen3-tts
-cd faster-qwen3-tts
+git clone https://github.com/andimarafioti/faster-qwen3-tts app/faster-qwen3-tts
```
-Create a `voices.json` mapping names to reference configurations (.wav to clone, transcript, language). The TUI setup writes this for you; manually it looks like:
+3. Create a `voices.json` mapping voice names to reference clips:
```json
{
@@ -35,17 +35,18 @@ Create a `voices.json` mapping names to reference configurations (.wav to clone,
}
```
-Run the server
+This backend always clones: every voice is a key in this file, and `ref_text` is the transcript of the clip.
+
+## Run the server
```bash
-python examples/openai_server.py --voices voices.json --port 8000
+app/envs/faster/bin/python app/faster-qwen3-tts/examples/openai_server.py --voices voices.json --port 8000
```
-Then from another terminal, run audiobook.py with `--backend faster`
+## Use it
```bash
-python audiobook.py --backend faster --voice NAME
+python audiobook.py --backend faster --api-url http://127.0.0.1:8000 --voice default
```
-`--voice` is required: NAME must be a key in the server's `voices.json`
-(the server silently falls back to its first configured voice if it is not).
+`--voice` must be a key in the server's `voices.json`.
diff --git a/app/docs/backend-qwen.md b/app/docs/backend-qwen.md
index 31af47c..b1e975e 100644
--- a/app/docs/backend-qwen.md
+++ b/app/docs/backend-qwen.md
@@ -1,85 +1,42 @@
-# Backend Option 2: Qwen3-TTS
+# qwen-tts (`--backend qwen`)
-The easiest way is to run `python audiobook.py` → **Configure Backends… → Install Backend → qwen-tts** (or `python app/backends/qwen.py`): the TUI pip-installs `qwen-tts` into its own managed venv (`app/envs/qwen`, separate from the app's venv and from the faster backend's — the two TTS stacks ship conflicting versions of a shared `qwen_tts` module) — that's all there is to it, the install asks no questions. The demo port lives in `app/converter/config.py` (edit it in the hub's **Settings** screen). The qwen backend runs **one model at a time** on that single port: pick Base, CustomVoice or VoiceDesign per run on the **Generate Audiobooks** screen (switching models while a managed server is up restarts it with the newly-selected model; an autostart boots exactly the model the run picked). You can also start the server from the hub's **Start/Stop Backend Servers** menu (a fresh start asks which of CustomVoice, Base or VoiceDesign to load; stopping never asks), or let a conversion start it automatically.
+Runs the [qwen-tts](https://pypi.org/project/qwen-tts/) `qwen-tts-demo` server, one model at a time. Installs into its own managed venv, `app/envs/qwen`.
-If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port (its `GET /info` names which of the three demos answers), so a manually-installed backend works once its server is up. To use a demo server on another machine, set `QWEN_REMOTE_URL` in `app/converter/config.py` to its `host:port` (default `127.0.0.1:7860`) — the hub probes it and offers the matching `qwen-tts [remote]` mode limited to the model that server hosts — or pass `--api-url` on the CLI.
+Run all commands from the project root (next to `audiobook.py`).
-Model weights download automatically from HuggingFace into the standard cache (`~/.cache/huggingface/hub`) the first time a server for each model starts — there is nothing else to install per model. To pre-fetch or remove a single model's weights without starting its server, open **Configure Backends… → Configure qwen-tts**: each of Base / CustomVoice / VoiceDesign gets an Install (a streamed, resumable download — canceling one just means it resumes later) or Uninstall action, with a server hosting that model stopped first. Uninstalling the whole backend deletes all three of those directories along with the pip package; only they are ever touched — anything else in your HuggingFace cache is left alone.
+## Requirements
-Install qwen-tts with pip into your environment:
+- Python 3.12+ (as for the app itself)
-```bash
-python -m venv audiobook
-source audiobook/bin/activate
-pip install -U qwen-tts
-```
-
-Run the backend with `qwen-tts-demo <model>`. Add `--no-flash-attn` if FlashAttention isn't installed (see below).
-
-## Voice design
+## Install
```bash
-qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign --ip 127.0.0.1 --port 7860 [--no-flash-attn]
+python3 -m venv app/envs/qwen
+app/envs/qwen/bin/pip install -U qwen-tts
```
-Then:
-
-```bash
-python audiobook.py --backend qwen \
- --instructions "A warm adult female narrator with a British accent"
-```
+Model weights download from HuggingFace automatically on first server start.
-The narrator is tagged "designed" in output file names.
+## Run the server
-## Voice clone
+Start the demo matching what you want to do:
```bash
-qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7860 [--no-flash-attn]
-```
-
-Then in another terminal:
-
-```bash
-python audiobook.py --backend qwen --clone reference.wav
-```
-
-The reference `.wav` should be ~10-15 seconds (3 second minimum, 60 second maximum; ~15 seconds is ideal). Longer is **not** better.
+# Built-in speakers (pick a speaker with --voice in TAG)
+app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860
-Whisper (`faster_whisper` or `whisper`) is used automatically to transcribe the reference audio. Without a Whisper backend it falls back to x-vector-only cloning. Override with `--transcription "What the .wav says"` or skip transcription with `--no-transcription`.
+# Voice cloning (needs a reference clip)
+app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7860
-## Custom voice (i.e. built-in voice)
-
-```bash
-source audiobook/bin/activate
-qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860 [--no-flash-attn]
+# Voice design (style described with --instructions in TAG)
+app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign --ip 127.0.0.1 --port 7860
```
-```bash
-python audiobook.py --backend qwen
-```
-
-Change the voice settings in `app/converter/config.py`.
-
-## Optional: FlashAttention for qwen-tts-demo server
-
-FlashAttention provides a *small* speed boost on the `qwen` backend. It is **not** relevant with other backends, and switching to either of those will provide a bigger speed boost.
+- Add `--no-flash-attn` if FlashAttention is not installed.
+- For cloning, reference clips of 10-15 seconds of clean speech work best (pass with `--clone <wav>`).
-`qwen-tts-demo` server tries to use FlashAttention 2 by default and requires `--no-flash-attn` without it. You have two options to install FlashAttention in your python environment:
-
-1. Build from source (takes absolutely forever). If you run out of memory, lower MAX_JOBS until you don't.
-
-```bash
-source audiobook/bin/activate
-pip install ninja packaging psutil
-MAX_JOBS=4 pip install --no-build-isolation flash-attn
-```
-
-2. pip install a prebuilt wheel matching your torch / CUDA / Python / CXX11-ABI combination:
+## Use it
```bash
-source audiobook/bin/activate
-python -c "import torch; print(torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI)"
+python audiobook.py --backend qwen --api-url http://127.0.0.1:7860
```
-
-- [Official wheels](https://github.com/Dao-AILab/flash-attention/releases) - Pick `cp312` + matching `cuX` + `torchX.Y` + `cxx11abiTRUE/FALSE`
-- [Third-party wheels](https://mjunya.com/flash-attention-prebuild-wheels/)
diff --git a/app/docs/backend-sglomni.md b/app/docs/backend-sglomni.md
index 7af6a72..1d5dcda 100644
--- a/app/docs/backend-sglomni.md
+++ b/app/docs/backend-sglomni.md
@@ -1,257 +1,70 @@
-# SGLang-Omni Backend
+# SGLang-Omni (`--backend sglomni`)
-`--backend sglomni` talks to an [SGLang-Omni](https://github.com/sgl-project/sglang-omni)
-server (`sgl-omni serve`), which hosts **one TTS model per server process** behind the
-OpenAI-compatible `POST /v1/audio/speech` endpoint. The backend is installed as a pip
-package into its own managed venv (`app/envs/sglomni`) and its models are pre-downloaded
-into the standard HuggingFace cache — the same flow as the `qwen` and `faster` backends.
+Runs an [SGLang-Omni](https://github.com/sgl-project/sglang-omni) server (`sgl-omni serve`), one TTS model per server process. Installs into its own managed venv, `app/envs/sglomni`.
-## Requirements
-
-- **Linux with an NVIDIA GPU** (recent driver). SGLang-Omni's serving stack is
- CUDA-only: there are no Windows builds, and Apple Silicon is limited to ASR.
- The setup wizard warns loudly when `nvidia-smi` does not answer.
-- **Compute capability matters per model.** ZONOS2's default pipeline
- quantizes its MoE experts to FP8 at load time, and the FP8 Triton kernels
- only compile on compute capability 8.9+ (RTX 4090/5090, Hopper). On older
- GPUs (Ampere: RTX 30xx, A100) the managed spec automatically launches the
- vendored bf16 config (`app/backends/sglomni/configs/zonos2_bf16.yaml`)
- instead, which runs ZONOS2 in bf16 at roughly twice the MoE VRAM — the
- status line tags the model `zonos2 (bf16 fallback)` and the install/boot
- output says so. That config also raises the engine's
- `mem_fraction_static` to 0.70: the builder's 0.5 default leaves no room
- for the KV cache once the bf16 weights (~11.5 GB) are resident on a
- 24 GB card, and the server aborts with "Loaded weights leave no GPU
- memory for the KV cache". Practical floor: a ~16 GB-class card; all
- stages stay colocated on GPU 0.
-- **Python 3.10, 3.11 or 3.12** for the backend venv. `sglang-omni` requires
- `>=3.10,<3.13` while the app itself is version-agnostic, so the wizard
- resolves a compatible interpreter automatically:
- 1. the app venv's interpreter when it is already 3.10-3.12 (no download);
- 2. a `python3.12`/`python3.11`/`python3.10` found on `PATH`;
- 3. otherwise it pip-installs `uv` into the app venv and provisions a managed
- standalone CPython 3.12 into `app/envs/pythons` (checksum-verified, no root
- required — this is the zero-prerequisites path on hosts like stock Arch).
-- **Disk space**: the venv alone is ~10 GB (torch, sglang, flash-attn, flashinfer,
- CUDA-13 wheels); each model adds 1-10 GB of weights in `~/.cache/huggingface/hub`.
-- `ffmpeg` (already a project prerequisite).
-
-The venv pip-installs `sglang-omni` with `--pre` (its dependency stack includes
-prerelease components). The verified version at the time of writing is **0.1.4**.
+Run all commands from the project root (next to `audiobook.py`).
-## Model Catalog
+## Requirements
-Each catalog entry is one installable model. Both the setup wizard and the
-hub's **Configure Backends… → Configure SGLang-Omni** screen use the same
-checkbox tree (grouped by upstream org, like the audio.cpp one), with the
-already-installed models pre-checked: checking a model installs it — running
-its companion-package recipe, then pre-downloading its weights with the
-venv's `hf` CLI (resumable, cancelable) — and unchecking one removes its
-cached weights after a confirm (a managed server hosting that model is
-stopped first). The whole diff runs as one task-view pass, removals before
-downloads, and the tree re-opens afterwards reflecting the state on disk.
-Models can also be left uninstalled — the first server start for one fetches
-its weights implicitly, but a pre-download keeps the managed server's boot
-inside the start timeout.
+- Linux with an NVIDIA GPU
+- Python 3.10-3.12 (`sglang-omni` requires `>=3.10,<3.13`)
+- ~10 GB disk for the venv plus 1-10 GB per model of weights in `~/.cache/huggingface/hub`
-| Catalog key | Model | Voice | Notes |
-| --- | --- | --- | --- |
-| `qwen3_tts_0_6b_customvoice` | Qwen3-TTS 0.6B CustomVoice | built-in speakers | lightest model |
-| `qwen3_tts_0_6b_base` | Qwen3-TTS 0.6B Base | clone (reference required) | |
-| `qwen3_tts_1_7b_base` | Qwen3-TTS 1.7B Base | clone (reference required) | higher quality |
-| `qwen3_tts_1_7b_voicedesign` | Qwen3-TTS 1.7B VoiceDesign | `--instructions` | |
-| `higgs_audio_v3_tts` | Higgs Audio v3 TTS | default voice or clone | launches with a vendored config: VRAM headroom for 24 GB cards + a raised generation cap (requests carry `max_new_tokens=3000` — upstream pins the thinker engine's context length at 4096, so prompt + generation must fit 4095 tokens, ~40 s of speech — and sub-requests cap at 80 words, with a pre-run popup offering the CHUNK_SIZE clamp) |
-| `moss_tts` | MOSS-TTS v1.5 | clone (reference required) | |
-| `moss_tts_local` | MOSS-TTS Local v1.5 | default voice or clone | 48 kHz |
-| `voxtral_tts` | Voxtral TTS 4B | preset named voices | e.g. `default`, `casual_male` |
-| `dots_tts_mf` | dots.tts (MeanFlow) | clone (reference required) | |
-| `fish_s2_pro` | Fish Speech S2-Pro | default voice or clone | needs ~24 GB VRAM (known OOM on a single RTX 3090, upstream issue #359) |
-| `zonos2` | ZONOS2 | clone (reference required) | 44.1 kHz; FP8 pipeline falls back to bf16 on GPUs below compute capability 8.9; requests carry `max_new_tokens=12288` because the engine's 1024-frame default caps one request at ~12 s of speech |
+## Install
-In the hub's **Generate Audiobooks** form the Model picker reads as a table,
-like the audio.cpp one: each entry's label is padded to the widest one and its
-capabilities are rendered as fixed columns so every capability word lines up
-down its own column — `tts` in the first column (entries that voice plain
-text with a preset or built-in default voice), `clone` in the second (the
-entry clones a reference clip), `design` in the third (Qwen3-TTS VoiceDesign).
-Clone entries that also narrate without a reference (Higgs Audio v3, MOSS-TTS
-Local, Fish Speech S2-Pro) carry both `tts clone`.
+1. Create the venv and install the server:
-### Companion packages
+```bash
+uv venv --seed --python 3.12 app/envs/sglomni # or: python3.12 -m venv app/envs/sglomni
+app/envs/sglomni/bin/python -m pip install --pre sglang-omni
+```
-Some models need extra packages in the backend venv before their server
-starts; installing a model through this tool runs its recipe automatically.
+2. Install companion packages for the models you plan to host:
-| Model family | Companion packages (as upstream instructs) | System packages |
+| Models | Into `app/envs/sglomni` | System packages |
| --- | --- | --- |
-| Qwen3-TTS (all four) | `sox`, `einops`, `qwen-tts==0.1.1` — all with `--no-deps`: the qwen-tts demo pins Transformers 4, which would replace sglang-omni's pinned 5.x stack (sglang-omni shims the API differences) | the `sox` **binary** (e.g. `sudo pacman -S sox`, `sudo apt install sox`) |
-| Fish Speech S2-Pro, ZONOS2 | `descript-audiotools==0.7.2`, `descript-audio-codec==1.0.0` (Descript DAC codec) | `ffmpeg` on the server's `PATH` |
-
-A missing system package never blocks a download — the wizard prints the
-remediation and the weights install anyway — but the server will fail to
-synthesize with that model until the package is present.
-
-## How voice cloning works
-
-Unlike `audio.cpp` (server-side voice presets) the reference clip travels
-**with each request** as `ref_audio` + `ref_text`:
-
-- The transcript comes from `--transcription`, or a local Whisper
- transcription of the clip (the qwen backend's flow). Without any
- transcript the request is sent without `ref_text`, which lowers quality
- for models that use it.
-- On a locally-managed (or loopback `--api-url`) server the clip is sent as
- its **local path** — the server reads the file directly.
-- Against a remote server the clip is **inlined as a base64 data URL**, so
- `--api-url` runs need no server-side file setup.
-
-Reference clips live in the project's `voices/` directory (10-20 seconds of
-clean speech recommended).
-
-### Higgs Audio v3's request window
-
-The Higgs server pins its thinker engine's context length at 4096 tokens
-(not configurable), and the scheduler rejects any request whose prompt —
-including the reference-audio tokens — plus `max_new_tokens` exceeds that
-window ("Request requires more tokens than the thinker KV cache can hold",
-on every GPU: the KV pool side is never the binding constraint). One
-request can therefore narrate at most ~40 s of speech, so:
-
-- Requests carry `max_new_tokens=3000` (the vendored config raises the
- engine's 2048-frame default to match), and sub-requests cap at 80 words.
- Before a run with Higgs, a popup offers to clamp CHUNK_SIZE for the run
- ("Set Chunk to 80"), keep the configured size (audio may cut off
- mid-chunk), or cancel; CLI runs answer on the console (`[S/t/c]`, and a
- closed stdin clamps so unattended runs never truncate silently).
-- When a request is still rejected (a long reference clip, for example),
- the client refits `max_new_tokens` to the capacity the server reports
- and resends once — the same request never retries as-is.
-- Voice cloning with a *long* reference clip eats into the same window:
- keep references around 10-20 seconds (as recommended above) for the
- most usable generation headroom.
-
-## Server lifecycle
-
-The hub and the CLI start and stop the managed instance around each run
-(like every backend): without `--api-url` the CLI boots the selected model's
-server, converts, and stops it again; a server already answering at the
-configured endpoint is used as-is and left running. Because one process
-hosts one model, a run whose selected model differs from the hosted one
-restarts a server this tool started — a foreign server hosting another
-model refuses the run with an actionable message instead.
+| All four Qwen3-TTS | `bin/python -m pip install --no-deps sox einops qwen-tts==0.1.1` | `sox` binary (e.g. `sudo apt install sox`) |
+| Fish Speech S2-Pro, ZONOS2 | `bin/python -m pip install descript-audiotools==0.7.2 descript-audio-codec==1.0.0 protobuf==6.33.6` | `ffmpeg` on PATH |
-The hub's **Start/Stop Backend Servers** menu asks which *downloaded* model
-to load when starting the server fresh (with none downloaded the entry says
-so and points at the Configure screen; a single model starts without
-asking). A start also auto-installs a model's missing companion packages
-first, so a boot never dies on their import even when the weights arrived
-via the shared HuggingFace cache or an earlier install's pip run failed.
-
-The server boots a multi-stage pipeline (preprocessing → TTS generation →
-vocoder) and may pull companion weights on first start, so its start
-timeout is larger than the other backends' (20 minutes). Pre-downloading
-models keeps cold boots well inside it.
-
-External servers: point the **Settings → SGLang-Omni remote URL** (or the
-CLI's `--api-url`) at an `sgl-omni` instance. The hub discovers it via
-`GET /health` (`{"status": "healthy", "stages": [...]}`) and lists its
-model from `GET /v1/models`; models uploaded to that server via
-`POST /v1/audio/voices` appear in its Voice menu.
-
-## Troubleshooting
-
-**The generate screen sits on "Status: starting", and the server log says
-`Port 8100 is already in use ... Using port 37183 instead`.** Something
-else already holds the configured port (often a stale `sgl-omni` from an
-earlier attempt). The upstream launcher does not fail — it silently moves
-the server to a random port, where no client ever looks for it. The app
-detects this in the boot log, aborts the boot, and names both ports.
-Find and stop the process holding the port
-(`ss -tlnp 'sport = :8100'`), or move this server to a free port
-(Settings → SGLang-Omni port), then start again.
-
-**The first request dies with `HTTP 500: CUDA out of memory. Tried to
-allocate ~100 MiB` (and every retry fails identically) on a 24 GB card,
-with the model far smaller than the card.** The model size is not the
-problem: upstream's Qwen3-TTS, MOSS-TTS and Voxtral pipelines colocate
-their AR engine and vocoder on GPU 0 with the engine's sglang
-`mem_fraction_static` unset, so the static pool (weights + KV cache) is
-auto-sized to nearly all free VRAM at boot. By the time a real request
-runs, the engine's CUDA graphs (~1.5 GB) and the colocated vocoder are
-resident too and only tens of MiB are left — the transient allocations
-generation needs do not fit. The vendored configs for those models pin
-the pool exactly for this — 0.70 for the small ones (~7 GB of headroom
-on a 24 GB card); Higgs (`gpu_memory_fraction: 0.80`) and ZONOS2's
-bf16 fallback (`0.70`) carry the same treatment. MOSS-TTS is the
-exception: its ~17.1 GB of v1.5 bf16 weights alone exceed the 0.70
-budget on the ~20.7 GB the engine profiler sees at that point (its
-colocated audio-tokenizer stage is already resident), so a 0.70 pin
-aborts the boot outright with "Loaded weights leave no GPU memory for
-the KV cache" (the floor is ~0.828 on a clean 24 GB card) instead of
-fixing the request-time OOM — it runs `mem_fraction_static: 0.87`,
-keeping a ~0.8 GB KV pool with ~2.7 GB of slack for the CUDA graphs
-and the colocated vocoder. If you hit the failure
-on a server started by hand or an older checkout, pin the budget on the
-command line and let this tool use the running server as-is:
+3. Download model weights:
```bash
-app/envs/sglomni/bin/sgl-omni serve \
- --model-path Qwen/Qwen3-TTS-12Hz-0.6B-Base \
- --config app/backends/sglomni/configs/qwen3_tts_0_6b.yaml \
- --port 8100 --mem-fraction-static 0.70
+app/envs/sglomni/bin/hf download <hf-repo> # repos in the table below
```
-(MOSS-TTS Local needs none of this: its upstream config already budgets
-its colocated stages explicitly. Fish Speech S2-Pro's OOM is a different
-problem — the model itself needs ~24 GB, upstream issue #359.)
+## Models
-## Manual setup
+| Key (TAG `--model`) | HF repo (`--model-path`) | Config |
+| --- | --- | --- |
+| `qwen3_tts_0_6b_customvoice` | `Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice` | `qwen3_tts_0_6b_customvoice.yaml` |
+| `qwen3_tts_0_6b_base` | `Qwen/Qwen3-TTS-12Hz-0.6B-Base` | `qwen3_tts_0_6b.yaml` |
+| `qwen3_tts_1_7b_base` | `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | `qwen3_tts_1_7b.yaml` |
+| `qwen3_tts_1_7b_voicedesign` | `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | `qwen3_tts_1_7b_voicedesign.yaml` |
+| `higgs_audio_v3_tts` | `bosonai/higgs-audio-v3-tts-4b` | `higgs_audio_v3_tts.yaml` |
+| `moss_tts` | `OpenMOSS-Team/MOSS-TTS-v1.5` | `moss_tts.yaml` |
+| `moss_tts_local` | `OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5` | `moss_tts_local.yaml` |
+| `voxtral_tts` | `mistralai/Voxtral-4B-TTS-2603` | `voxtral_tts.yaml` |
+| `dots_tts_mf` | `dots-studio/dots.tts-mf` | `dots_tts.yaml` |
+| `fish_s2_pro` | `fishaudio/s2-pro` | `s2pro_tts.yaml` |
+| `zonos2` | `Zyphra/zonos2` | none |
+
+Config files live in `app/backends/sglomni/configs/`. ZONOS2 runs without a config; on GPUs below compute capability 8.9 (RTX 30xx, A100) pass `--config app/backends/sglomni/configs/zonos2_bf16.yaml` instead.
+
+## Run the server
```bash
-# 1. A compatible venv (any of: system python3.10-3.12, or uv-managed)
-uv venv --seed --python 3.12 app/envs/sglomni
-
-# 2. The package (into that venv)
-app/envs/sglomni/bin/python -m pip install --pre sglang-omni
-
-# 3. A model's companion packages (example: Qwen3-TTS)
-app/envs/sglomni/bin/python -m pip install --no-deps sox einops
-app/envs/sglomni/bin/python -m pip install --no-deps qwen-tts==0.1.1
-
-# 4. Model weights (example)
-app/envs/sglomni/bin/hf download bosonai/higgs-audio-v3-tts-4b
-
-# 5. Start a server manually (the managed flow does this for you)
+# Example: Higgs Audio v3
app/envs/sglomni/bin/sgl-omni serve \
--model-path bosonai/higgs-audio-v3-tts-4b \
--config app/backends/sglomni/configs/higgs_audio_v3_tts.yaml \
--port 8100
-# 6. Convert
-python audiobook.py --backend sglomni --model higgs_audio_v3_tts \
- --api-url http://127.0.0.1:8100
+# ZONOS2 (no config)
+app/envs/sglomni/bin/sgl-omni serve --model-path Zyphra/zonos2 --port 8100
```
-Models that take a vendored config file (all but ZONOS2) add
-`--config app/backends/sglomni/configs/<key>.yaml` to the serve command —
-the managed spec builds this from the catalog automatically. Higgs's config
-also exists for a reason: the upstream pipeline budgets 98% of the card
-across its colocated stages, which OOMs on a 24 GB GPU once anything else
-uses VRAM, and its engine caps requests at ~27 s of speech.
-
-## CLI examples
+## Use it
```bash
-# Zero-shot narration with Higgs Audio v3
-python audiobook.py --backend sglomni --model higgs_audio_v3_tts
-
-# Voice cloning from a reference clip (transcript transcribed locally)
-python audiobook.py --backend sglomni --model moss_tts_local --clone voices/narrator.wav
-
-# Built-in speakers (Qwen3-TTS CustomVoice)
-python audiobook.py --backend sglomni --model qwen3_tts_0_6b_customvoice --voice Vivian
-
-# Voice design
-python audiobook.py --backend sglomni --model qwen3_tts_1_7b_voicedesign \
- --instructions "A warm adult female narrator with a British accent"
+python audiobook.py --backend sglomni --api-url http://127.0.0.1:8100 --model <key>
```