diff options
| author | historia <historiavg@proton.me> | 2026-08-24 02:59:26 -0400 |
|---|---|---|
| committer | historia <historiavg@proton.me> | 2026-08-24 02:59:26 -0400 |
| commit | f00249db9d1ea051d29aa1bcca869fc4b88e83eb (patch) | |
| tree | a75f076fac1b63e0b4bf2eb8f54affbcc681a891 /docs | |
| parent | 9dd4f9595be3b1d76a3a07dc3eca90cfaf8a3f97 (diff) | |
| download | tts-audiobook-generator-f00249db9d1ea051d29aa1bcca869fc4b88e83eb.tar.gz | |
refactor: add app directory, dir structure change
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/backend-audiocpp.md | 91 | ||||
| -rw-r--r-- | docs/backend-faster.md | 45 | ||||
| -rw-r--r-- | docs/backend-qwen.md | 67 |
3 files changed, 0 insertions, 203 deletions
diff --git a/docs/backend-audiocpp.md b/docs/backend-audiocpp.md deleted file mode 100644 index ee511bb..0000000 --- a/docs/backend-audiocpp.md +++ /dev/null @@ -1,91 +0,0 @@ -# Backend Option 1: audio.cpp - -`--backend audiocpp` talks to `audiocpp_server` from [audio.cpp](https://github.com/0xShug0/audio.cpp), which hosts numerous TTS model families. - -The easiest way is the TUI: run `python audiobook.py`, choose **Set up a backend… → audio.cpp**, and it clones `audio.cpp` into `./audio.cpp` (or reuses an existing checkout), builds `audiocpp_server`, lets you pick model families/packages from an expandable checkbox tree (reading the checkout's `model_specs/`), transcribes `.wav` voices with `whisper`, writes `server.json` into the checkout, syncs `converter/config.py`, and prints the launch command (the hub can also start the server for you via the **Server** menu or automatically when converting). Run it directly with `python -m backends.audiocpp` (flags like `--wavs`, `--families`, `--build-backend`, `--clone` skip the corresponding screens for scripting). The TUI runs in the managed `envs/tts` venv, which includes `whisper` via `requirements.txt`; for a manual setup, make sure `whisper` (or `faster_whisper`) is installed in the environment you run the wizard from. The Qwen3-TTS model tree also offers hosting the VoiceDesign package as a `vdes` entry. - -If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up. - -### Download and build audiocpp_server - -Download and build `audiocpp_server` for your platform and backend `(cuda, vulkan, hip, cpu)`. Check [audio.cpp's readme](https://github.com/0xShug0/audio.cpp) for details. I'm using one of the helper scripts: - -```bash -git clone https://github.com/0xShug0/audio.cpp -cd audio.cpp -scripts/build_linux.sh --backend cuda --target audiocpp_server -``` - -### Install models - -Download model packages with the python model manager script from the audio.cpp checkout. Each installs to `./models`. Here are two examples, Higgs Audio and Qwen3-TTS: - -```bash -python tools/model_manager_v2.py install higgs_audio_tts_4b_q8_0 -python tools/model_manager_v2.py install qwen3_tts_1_7b_base_q8_0 -python tools/model_manager_v2.py install qwen3_tts_1_7b_customvoice_q8_0 -``` - -You can run `python tools/model_manager_v2.py list` to see all available models. - -### Create server.json - -Create a `server.json` config file. One server can host multiple models and multiple cloned voices. The `id:` fields are the model names you will set for `tts-audiobook-generator` with `--model`. - -```json -{ - "host": "127.0.0.1", - "port": 8080, - "backend": "cuda", - "lazy_load": true, - "voice_dir": "/path/to/clone/wavs", - "models": [ - { - "id": "higgs", - "family": "higgs_audio_tts", - "path": "models/Higgs-Audio-v3-TTS-4B-GGUF", - "task": "tts", - "mode": "offline" - }, - { - "id": "qwen", - "family": "qwen3_tts", - "path": "models/Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF", - "task": "tts", - "mode": "offline" - }, - { - "id": "qwen-clone", - "family": "qwen3_tts", - "path": "models/Qwen3-TTS-12Hz-1.7B-Base-GGUF", - "task": "tts", - "mode": "offline" - } - ] -} -``` - -### Run audio.cpp and the audiobook script - -Run the server with this config file. The `audiocpp_server` path will be slightly different depending on your platform and build options: - -```bash -./build/linux-cuda-release/bin/audiocpp_server --config server.json -``` - -In a different terminal, run `audiobook.py`. Pick the TTS `--model` and `--voice` from server.json: - -```bash -# Higgs Audio (clone-only) -python audiobook.py --backend audiocpp --model higgs --voice narrator - -# Qwen3-TTS built-in speaker -python audiobook.py --backend audiocpp --model qwen - -# Qwen3-TTS voice cloning -python audiobook.py --backend audiocpp --model qwen-clone --voice narrator - -# Qwen-TTS voice design -python audiobook.py --backend audiocpp --model qwen-design \ - --instructions "A warm adult female narrator with a British accent" -``` diff --git a/docs/backend-faster.md b/docs/backend-faster.md deleted file mode 100644 index c407a70..0000000 --- a/docs/backend-faster.md +++ /dev/null @@ -1,45 +0,0 @@ -# Backend Option 3: faster-qwen-tts - -`--backend faster` talks to the OpenAI-compatible server from [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts), which uses CUDA graph capture for roughly 5-10x faster inference with the same models. **It requires an NVIDIA GPU**. - -The easiest way is to run `python audiobook.py` → **Set up a backend… → faster-qwen3-tts** (or `python -m backends.faster path/to/clone/wavs`): the TUI pip-installs `faster-qwen3-tts[demo]` into its managed venv (`envs/tts`), clones the repo, transcribes the `.wav` files with `whisper`, and writes `voices.json` for you. You can also start the server from the hub's **Server** menu, or let a conversion start it automatically. - -If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up. - -Install into your environment (the same one used for qwen-tts is fine): - -```bash -conda activate audiobook -pip install -U qwen-tts -pip install "faster-qwen3-tts[demo]" -``` - -**This backend always uses voice cloning**. The reference voice and language are configured on the **server**, not through the converter. The server does not transcribe reference audio itself, so do it manually or use the `backends.faster` setup wizard (see below). - -The pip package does not include the server script, so clone the repository (the `backends.faster` wizard does this for you into `./faster-qwen3-tts`): - -```bash -git clone https://github.com/andimarafioti/faster-qwen3-tts -cd faster-qwen3-tts -``` - -Create a `voices.json` mapping names to reference configurations (.wav to clone, transcript, language). The TUI setup writes this for you; manually it looks like: - -```json -{ - "default": {"ref_audio": "voice1.wav", "ref_text": "Transcript of voice 1.", "language": "English"}, - "obama": {"ref_audio": "voice2.wav", "ref_text": "Transcript of voice 2.", "language": "English"} -} -``` - -Run the server - -```bash -python examples/openai_server.py --voices voices.json --port 8000 -``` - -Then from another terminal, run audiobook.py with `--backend faster` - -```bash -python audiobook.py --backend faster [--voice NAME] -``` diff --git a/docs/backend-qwen.md b/docs/backend-qwen.md deleted file mode 100644 index 028d6f3..0000000 --- a/docs/backend-qwen.md +++ /dev/null @@ -1,67 +0,0 @@ -# Backend Option 2: Qwen3-TTS - -The easiest way is to run `python audiobook.py` → **Set up a backend… → qwen-tts** (or `python -m backends.qwen`): the TUI pip-installs `qwen-tts` into its managed venv (`envs/tts`), configures the two ports and the built-in speaker in `converter/config.py`, and prints the launch commands. You can also start the server from the hub's **Server** menu, or let a conversion start it automatically. - -If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port, so a manually-installed backend works once its server is up. - -Install qwen-tts with pip into your environment: - -```bash -conda activate audiobook -pip install -U qwen-tts -``` - -Run the backend with `qwen-tts-demo`. Add `--no-flash-attn` if FlashAttention isn't installed (see below). Note that the Base model and CustomVoice model run on different ports. - -## Voice clone - -```bash -qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7861 [--no-flash-attn] -``` - -Then in another terminal: - -```bash -python audiobook.py --backend qwen --clone reference.wav -``` - -The reference `.wav` should be ~10-15 seconds (3 second minimum, 60 second maximum; ~15 seconds is ideal). Longer is **not** better. - -Whisper (`faster_whisper` or `whisper`) is used automatically to transcribe the reference audio. Without a Whisper backend it falls back to x-vector-only cloning. Override with `--transcription "What the .wav says"` or skip transcription with `--no-transcription`. - -## Custom voice (i.e. built-in voice) - -```bash -conda activate audiobook -qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860 [--no-flash-attn] -``` - -```bash -python audiobook.py --backend qwen -``` - -Change the voice settings in `converter/config.py`. - -## Optional: FlashAttention for qwen-tts-demo server - -FlashAttention provides a *small* speed boost on the `qwen` backend. It is **not** relevant with other backends, and switching to either of those will provide a bigger speed boost. - -`qwen-tts-demo` server tries to use FlashAttention 2 by default and requires `--no-flash-attn` without it. You have two options to install FlashAttention in your python environment: - -1. Build from source (takes absolutely forever). If you run out of memory, lower MAX_JOBS until you don't. - -```bash -conda activate audiobook -pip install ninja packaging psutil -MAX_JOBS=4 pip install --no-build-isolation flash-attn -``` - -2. pip install a prebuilt wheel matching your torch / CUDA / Python / CXX11-ABI combination: - -```bash -conda activate audiobook -python -c "import torch; print(torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI)" -``` - -- [Official wheels](https://github.com/Dao-AILab/flash-attention/releases) - Pick `cp312` + matching `cuX` + `torchX.Y` + `cxx11abiTRUE/FALSE` -- [Third-party wheels](https://mjunya.com/flash-attention-prebuild-wheels/) |
