1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
# Backend Option 1: audio.cpp
`--backend audiocpp` talks to `audiocpp_server` from [audio.cpp](https://github.com/0xShug0/audio.cpp), which hosts numerous TTS model families.
The easiest way is the TUI: run `python audiobook.py`, choose **Configure backends… → Install Backend → audio.cpp**, and it clones `audio.cpp` into `app/audio.cpp` (or reuses an existing checkout), builds `audiocpp_server`, lets you pick model families/packages from an expandable checkbox tree (reading the checkout's `model_specs/`), transcribes `.wav` voices with `whisper`, writes `server.json` into the checkout, syncs `app/converter/config.py`, and prints the launch command (the hub can also start the server for you via the **Start/Stop Backend Servers** menu or automatically when converting). The clone, build, transcription and model downloads all run inside the TUI — each shows a status (and, where the tool can measure it, a progress bar), and can be cancelled — instead of dropping to console output. Run it directly with `python app/backends/audiocpp.py` (flags like `--wavs`, `--families`, `--build-backend`, `--clone` skip the corresponding screens for scripting). The TUI runs in the managed `app/envs/tts` venv, which includes `whisper` via `requirements.txt`; for a manual setup, make sure `whisper` (or `faster_whisper`) is installed in the environment you run the wizard from. The Qwen3-TTS model tree also offers hosting the VoiceDesign package as a `vdes` entry.
The hub's backend status table distinguishes how far audio.cpp is set up: `unavailable` (nothing present), `downloaded (not built)` (checkout cloned, `audiocpp_server` not built), `built (not configured)` (binary built, no `server.json`), `installed` (ready; or `installed (models missing)` when the config references undownloaded models), and `running` once its server answers. Whenever the checkout exists but `audiocpp_server` is missing, **Configure backends… → Build audio.cpp server** builds it from the TUI (the wizard offers the build during setup too), so a backend whose build you skipped is never stuck as "unavailable". The setup steps run in order — build, then configure, then download models — so **Build audio.cpp server** and **Download Missing Models (audio.cpp)** are never offered at the same time; the model download appears only once the server binary is built.
If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up.
### Download and build audiocpp_server
Download and build `audiocpp_server` for your platform and backend `(cuda, vulkan, hip, cpu)`. Check [audio.cpp's readme](https://github.com/0xShug0/audio.cpp) for details. I'm using one of the helper scripts:
```bash
git clone https://github.com/0xShug0/audio.cpp
cd audio.cpp
scripts/build_linux.sh --backend cuda --target audiocpp_server
```
### Install models
Download model packages with the python model manager script from the audio.cpp checkout. Each installs to `./models`. Here are two examples, Higgs Audio and Qwen3-TTS:
```bash
python tools/model_manager_v2.py install higgs_audio_tts_4b_q8_0
python tools/model_manager_v2.py install qwen3_tts_1_7b_base_q8_0
python tools/model_manager_v2.py install qwen3_tts_1_7b_customvoice_q8_0
```
You can run `python tools/model_manager_v2.py list` to see all available models.
### Create server.json
Create a `server.json` config file. One server can host multiple models and multiple cloned voices. The `id:` fields are the model names you will set for `tts-audiobook-generator` with `--model`.
```json
{
"host": "127.0.0.1",
"port": 8080,
"backend": "cuda",
"lazy_load": true,
"voice_dir": "/path/to/clone/wavs",
"models": [
{
"id": "higgs",
"family": "higgs_audio_tts",
"path": "models/Higgs-Audio-v3-TTS-4B-GGUF",
"task": "tts",
"mode": "offline"
},
{
"id": "qwen",
"family": "qwen3_tts",
"path": "models/Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF",
"task": "tts",
"mode": "offline"
},
{
"id": "qwen-clone",
"family": "qwen3_tts",
"path": "models/Qwen3-TTS-12Hz-1.7B-Base-GGUF",
"task": "tts",
"mode": "offline"
}
]
}
```
### Run audio.cpp and the audiobook script
Run the server with this config file. The `audiocpp_server` path will be slightly different depending on your platform and build options:
```bash
./build/linux-cuda-release/bin/audiocpp_server --config server.json
```
In a different terminal, run `audiobook.py`. Pick the TTS `--model` and `--voice` from server.json:
```bash
# Higgs Audio (clone-only)
python audiobook.py --backend audiocpp --model higgs --voice narrator
# Qwen3-TTS built-in speaker
python audiobook.py --backend audiocpp --model qwen
# Qwen3-TTS voice cloning
python audiobook.py --backend audiocpp --model qwen-clone --voice narrator
# Qwen-TTS voice design
python audiobook.py --backend audiocpp --model qwen-design \
--instructions "A warm adult female narrator with a British accent"
```
The hub also works with an audio.cpp server that runs somewhere else (another checkout, another machine): set `AUDIOCPP_REMOTE_URL` in `app/converter/config.py` (or the TUI **Settings** → "audio.cpp remote URL") to its `host:port`. The hub probes that URL and, when it answers, offers an `audio.cpp [remote]` entry in **Convert books…** whose models and voices are queried live (`GET /v1/models` and `GET /v1/audio/voices`) — alongside the managed `audio.cpp` entry, which keeps reading the local `server.json`. The remote URL defaults to `127.0.0.1:8080`, so a server started outside this tool on the local port is found automatically. On the CLI, pass `--api-url http://host:port` (and `--model`/`--voice` matching that server's config).
Before converting, `audiobook.py` asks the server to unload all currently loaded models (`POST /v1/tasks/unload_all_models`) so models left resident by earlier runs free their memory (e.g. VRAM on GPU backends) and only the selected entry loads. A server without that endpoint, or one busy unloading, only produces a warning. This behavior is controlled by the **Settings** → "Unload models" option (or `AUDIOCPP_UNLOAD_MODELS` in `app/converter/config.py`), which defaults to **Yes**; set it to **No** to keep other models resident across runs.
|
