# tts-audiobook-generator Generate audiobooks from books (epub, txt, pdf) using text-to-speech backends such as `audio.cpp` and `qwen-tts`. ## Overview The converter sends text extracted from your books to a locally running TTS server and assembles the returned audio into a single audiobook file. - Input: `.txt`, `.pdf`, or `.epub` - Output: `.m4b`, `.mp3`, `.ogg`, or `.flac` - Supports [audio.cpp](https://github.com/0xShug0/audio.cpp), [qwen-tts](https://pypi.org/project/qwen-tts/), and [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts) backend servers (audio.cpp can host any of its TTS model families, not just Qwen3-TTS) - Output a single file or one per chapter - Automatic metadata (title/artist/album tags, chapter track numbers) and a generated cover - Supports text-to-speech, voice cloning, voice design, and per-model controls like emotion/speed ## Prerequisites - Python 3.12 - ffmpeg ## Installation Create a python 3.12 environment, clone the repo, and install the requirements. ```bash conda create -n audiobook python=3.12 -y conda activate audiobook git clone https://git.historia.vg/git/tts-audiobook-generator cd tts-audiobook-generator pip install -r requirements.txt ``` Put your book files (epub, etc.) in the `input/` directory. The output goes to `output/`. ## Quick start (TUI) Run the generator with no arguments in a terminal: ```bash python audiobook.py ``` A full-screen TUI opens and detects which TTS backends are already set up. From the menu you can: - **Convert books…** — process the `input/` directory with a ready backend (it reads the backend's `server.json` / `voices.json` so you pick the model and voice from menus), or - **Set up a backend…** — clone, build, and configure a backend end-to-end (audio.cpp, qwen, faster), or - **Modify a backend…** — regenerate its config (a new `server.json`, rebuild `voices.json`, change ports/speaker). Everything the TUI does can also be scripted with flags: `python audiobook.py --backend audiocpp --model higgs --voice narrator`, or `python -m backends.audiocpp --families higgs_audio_tts --clone --build-backend cuda`. You need one of the following backends (the TUI sets them up for you; manual steps below): | Backend | Description | | -------------------------------------------------------------------- | ------------------------------------------------------ | | [audio.cpp](https://github.com/0xShug0/audio.cpp) | Newer C++ TTS backend that supports many recent models | | [Qwen-TTS](https://pypi.org/project/qwen-tts/) | Qwen demo server (qwen-tts-demo) | | [Faster-Qwen-TTS](https://github.com/andimarafioti/faster-qwen3-tts) | Qwen server with 2-8x faster inference for NVidia GPUs | ## Options | Flag | Description | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `--backend {audiocpp,qwen,faster}` | TTS server to use (default `audiocpp`). | | `--format {mp3,m4b,ogg,flac}` | Output format (default `m4b`). | | `--speed ` | Playback speed, pitch-preserving (`1.0` = normal). A normal-speed copy is also output. | | `--single-file` | Merge all chapters into a single file. `m4b` is always one file. | | `--language ` | Output language for the synthesized speech. Can add an accent even if the text is English. | | `--chunk` | Force client-side chunking into `CHUNK_SIZE`-word requests. Only matters for `--backend audiocpp`, which otherwise sends each chapter as one request and lets the server chunk long text itself (may double-chunk); the `qwen` and `faster` backends always chunk. | | `--model ` | `audiocpp`: Choose the model from `server.json` | | `--instructions "..."` | `audiocpp`: voice design or style instruction. Required for voice design models (`vdes`) | | `--option KEY=VALUE` | `audiocpp`: Some models support custom options (e.g. `emotion=netural`) that can be passed with this flag | | `--debug` | Dump each chunk's raw audio and sent text to `debug/` and log every request. | | `--voice ` | `audiocpp`, `faster`: Server-side voice to request | | `--clone ` | `qwen`: Reference audio (`wav`) for voice cloning. | | `--transcription "..."` | `qwen`: Override whisper auto-transcription with manual audio transcript. | | `--no-transcription` | `qwen`: Skip auto-transcription of the reference audio. | Other options including backend server URLs/ports are configured in `converter/config.py` ## Backend Option 1: audio.cpp `audiocpp` is an easy to use server that hosts numerous TTS model families. ### Download and build audiocpp_server Download and build `audiocpp_server` for your platform and backend `(cuda, vulkan, hip, cpu)`. Check [audio.cpp's readme](https://github.com/0xShug0/audio.cpp) for details. I'm using one of the helper scripts: ```bash git clone https://github.com/0xShug0/audio.cpp cd audio.cpp scripts/build_linux.sh --backend cuda --target audiocpp_server ``` ### Install models Download model packages with the python model manager script from the audio.cpp checkout. Each installs to `./models`. Here are two examples, Higgs Audio and Qwen3-TTS: ```bash python tools/model_manager_v2.py install higgs_audio_tts_4b_q8_0 python tools/model_manager_v2.py install qwen3_tts_1_7b_base_q8_0 python tools/model_manager_v2.py install qwen3_tts_1_7b_customvoice_q8_0 ``` You can run `python tools/model_manager_v2.py list` to see all available models. ### Create server.json Create a `server.json` config file. One server can host multiple models and multiple cloned voices. The `id:` fields are the model names you will set for `tts-audiobook-generator` with `--model`. The easiest way is the TUI: run `python audiobook.py`, choose **Set up a backend… → audio.cpp**, and it clones `audio.cpp` into `./audio.cpp` (or reuses an existing checkout), builds `audiocpp_server`, lets you pick model families/packages from an expandable checkbox tree (reading the checkout's `model_specs/`), transcribes `.wav` voices with `whisper`, writes `server.json` into the checkout, syncs `converter/config.py`, and prints the launch command. Run it directly with `python -m backends.audiocpp` (flags like `--wavs`, `--families`, `--build-backend`, `--clone` skip the corresponding screens for scripting). Make sure you're in a Python environment that has `whisper` (i.e. `conda activate audiobook` before running). The Qwen3-TTS model tree also offers hosting the VoiceDesign package as a `vdes` entry (see [Voice design](#voice-design) below). ```json { "host": "127.0.0.1", "port": 8080, "backend": "cuda", "lazy_load": true, "voice_dir": "/path/to/clone/wavs", "models": [ { "id": "higgs", "family": "higgs_audio_tts", "path": "models/Higgs-Audio-v3-TTS-4B-GGUF", "task": "tts", "mode": "offline" }, { "id": "qwen", "family": "qwen3_tts", "path": "models/Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF", "task": "tts", "mode": "offline" }, { "id": "qwen-clone", "family": "qwen3_tts", "path": "models/Qwen3-TTS-12Hz-1.7B-Base-GGUF", "task": "tts", "mode": "offline" } ] } ``` ### Run audio.cpp and the audiobook script Run the server with this config file. The `audiocpp_server` path will be slightly different depending on your platform and build options: ```bash ./build/linux-cuda-release/bin/audiocpp_server --config server.json ``` In a different terminal, run `audiobook.py`. Pick the TTS `--model` and `--voice` from server.json: ```bash # Higgs Audio (clone-only) python audiobook.py --backend audiocpp --model higgs --voice narrator # Qwen3-TTS built-in speaker python audiobook.py --backend audiocpp --model qwen # Qwen3-TTS voice cloning python audiobook.py --backend audiocpp --model qwen-clone --voice narrator # Qwen-TTS voice design python audiobook.py --backend audiocpp --model qwen-design \ --instructions "A warm adult female narrator with a British accent" ``` ## Other TTS Backends Installation and usage documentation for other supported TTS backends is in the `docs/` directory: - [qwen-tts instructions](docs/backend-qwen.md) - [faster-qwen-tts instructions](docs/backend-faster.md) ## Tips Transcription affects the output a lot. Whisper does not always give perfect transcription. Manual transcription is better. If you're cloning one language and outputting another language, `--no-transcription` will remove the accent. Alternatively, setting the "wrong" output `--language` can add an accent. Even tiny amounts of pause between phrases in the sample audio can have a big impact. Try increasing or decreasing them or find a sample with different cadence. ## License MIT ## Credits - [Qwen3-Audiobook-Converter](https://github.com/WhiskeyCoder/Qwen3-Audiobook-Converter) by WhiskeyCoder. This project was originally built upon this. - [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) voice model. - [audio.cpp](https://github.com/0xShug0/audio.cpp) inference engine for the `audiocpp` backend.