diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/backend-faster.md | 41 | ||||
| -rw-r--r-- | docs/backend-qwen.md | 63 |
2 files changed, 104 insertions, 0 deletions
diff --git a/docs/backend-faster.md b/docs/backend-faster.md new file mode 100644 index 0000000..83614c7 --- /dev/null +++ b/docs/backend-faster.md @@ -0,0 +1,41 @@ +# Backend Option 3: faster-qwen-tts + +`--backend faster` talks to the OpenAI-compatible server from [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts), which uses CUDA graph capture for roughly 5-10x faster inference with the same models. **It requires an NVIDIA GPU**. + +Install into the **same `audiobook` conda environment** used for qwen-tts. + +```bash +conda activate audiobook +pip install -U qwen-tts +pip install "faster-qwen3-tts[demo]" +``` + +**This backend always uses voice cloning**. The reference voice and language are configured on the **server**, not through the converter. The server does not transcribe reference audio itself, so do it manually or use the `tools/make_faster_voices_json.py` helper (see below). + +The pip package does not include the server script, so clone the repository: + +```bash +git clone https://github.com/andimarafioti/faster-qwen3-tts +cd faster-qwen3-tts +``` + +Create a `voices.json` mapping names to reference configurations (.wav to clone, transcript, language). Optionally run `python ./tools/make_faster_voices_json.py path/to/clone/wavs` to automatically create a `voices.json` using whisper to automatically transcribe the test audio. + +```json +{ + "default": {"ref_audio": "voice1.wav", "ref_text": "Transcript of voice 1.", "language": "English"}, + "obama": {"ref_audio": "voice2.wav", "ref_text": "Transcript of voice 2.", "language": "English"} +} +``` + +Run the server + +```bash +python examples/openai_server.py --voices voices.json --port 8000 +``` + +Then from another terminal, run audiobook.py with `--backend faster` + +```bash +python audiobook.py --backend faster [--voice NAME] +``` diff --git a/docs/backend-qwen.md b/docs/backend-qwen.md new file mode 100644 index 0000000..34078e3 --- /dev/null +++ b/docs/backend-qwen.md @@ -0,0 +1,63 @@ +# Backend Option 2: Qwen3-TTS + +Install qwen-tts with pip: + +```bash +conda activate audiobook +pip install -U qwen-tts +``` + +Run the backend with `qwen-tts-demo`. Add `--no-flash-attn` if FlashAttention isn't installed (see below). Note that the Base model and CustomVoice model run on different ports. + +## Voice clone + +```bash +qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7861 [--no-flash-attn] +``` + +Then in another terminal: + +```bash +python audiobook.py --backend qwen --clone reference.wav +``` + +The reference `.wav` should be ~10-15 seconds (3 second minimum, 60 second maximum; ~15 seconds is ideal). Longer is **not** better. + +Whisper (`faster_whisper` or `whisper`) is used automatically to transcribe the reference audio. Without a Whisper backend it falls back to x-vector-only cloning. Override with `--transcription "What the .wav says"` or skip transcription with `--no-transcription`. + +## Custom voice (i.e. built-in voice) + +```bash +conda activate audiobook +qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860 [--no-flash-attn] +``` + +```bash +python audiobook.py --backend qwen +``` + +Change the voice settings in `converter/config.py`. + +## Optional: FlashAttention for qwen-tts-demo server + +FlashAttention provides a *small* speed boost on the `qwen` backend. It is **not** relevant with other backends, and switching to either of those will provide a bigger speed boost. + +`qwen-tts-demo` server tries to use FlashAttention 2 by default and requires `--no-flash-attn` without it. You have two options to install FlashAttention in your python environment: + +1. Build from source (takes absolutely forever). If you run out of memory, lower MAX_JOBS until you don't. + +```bash +conda activate audiobook +pip install ninja packaging psutil +MAX_JOBS=4 pip install --no-build-isolation flash-attn +``` + +2. pip install a prebuilt wheel matching your torch / CUDA / Python / CXX11-ABI combination: + +```bash +conda activate audiobook +python -c "import torch; print(torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI)" +``` + +- [Official wheels](https://github.com/Dao-AILab/flash-attention/releases) - Pick `cp312` + matching `cuX` + `torchX.Y` + `cxx11abiTRUE/FALSE` +- [Third-party wheels](https://mjunya.com/flash-attention-prebuild-wheels/) |