diff options
Diffstat (limited to 'app/docs/backend-qwen.md')
| -rw-r--r-- | app/docs/backend-qwen.md | 85 |
1 files changed, 21 insertions, 64 deletions
diff --git a/app/docs/backend-qwen.md b/app/docs/backend-qwen.md index 31af47c..b1e975e 100644 --- a/app/docs/backend-qwen.md +++ b/app/docs/backend-qwen.md @@ -1,85 +1,42 @@ -# Backend Option 2: Qwen3-TTS +# qwen-tts (`--backend qwen`) -The easiest way is to run `python audiobook.py` → **Configure Backends… → Install Backend → qwen-tts** (or `python app/backends/qwen.py`): the TUI pip-installs `qwen-tts` into its own managed venv (`app/envs/qwen`, separate from the app's venv and from the faster backend's — the two TTS stacks ship conflicting versions of a shared `qwen_tts` module) — that's all there is to it, the install asks no questions. The demo port lives in `app/converter/config.py` (edit it in the hub's **Settings** screen). The qwen backend runs **one model at a time** on that single port: pick Base, CustomVoice or VoiceDesign per run on the **Generate Audiobooks** screen (switching models while a managed server is up restarts it with the newly-selected model; an autostart boots exactly the model the run picked). You can also start the server from the hub's **Start/Stop Backend Servers** menu (a fresh start asks which of CustomVoice, Base or VoiceDesign to load; stopping never asks), or let a conversion start it automatically. +Runs the [qwen-tts](https://pypi.org/project/qwen-tts/) `qwen-tts-demo` server, one model at a time. Installs into its own managed venv, `app/envs/qwen`. -If you prefer to install the backend yourself (in your own environment, not the managed venv), the manual steps are below. Either way the hub detects a running server by its port (its `GET /info` names which of the three demos answers), so a manually-installed backend works once its server is up. To use a demo server on another machine, set `QWEN_REMOTE_URL` in `app/converter/config.py` to its `host:port` (default `127.0.0.1:7860`) — the hub probes it and offers the matching `qwen-tts [remote]` mode limited to the model that server hosts — or pass `--api-url` on the CLI. +Run all commands from the project root (next to `audiobook.py`). -Model weights download automatically from HuggingFace into the standard cache (`~/.cache/huggingface/hub`) the first time a server for each model starts — there is nothing else to install per model. To pre-fetch or remove a single model's weights without starting its server, open **Configure Backends… → Configure qwen-tts**: each of Base / CustomVoice / VoiceDesign gets an Install (a streamed, resumable download — canceling one just means it resumes later) or Uninstall action, with a server hosting that model stopped first. Uninstalling the whole backend deletes all three of those directories along with the pip package; only they are ever touched — anything else in your HuggingFace cache is left alone. +## Requirements -Install qwen-tts with pip into your environment: +- Python 3.12+ (as for the app itself) -```bash -python -m venv audiobook -source audiobook/bin/activate -pip install -U qwen-tts -``` - -Run the backend with `qwen-tts-demo <model>`. Add `--no-flash-attn` if FlashAttention isn't installed (see below). - -## Voice design +## Install ```bash -qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign --ip 127.0.0.1 --port 7860 [--no-flash-attn] +python3 -m venv app/envs/qwen +app/envs/qwen/bin/pip install -U qwen-tts ``` -Then: - -```bash -python audiobook.py --backend qwen \ - --instructions "A warm adult female narrator with a British accent" -``` +Model weights download from HuggingFace automatically on first server start. -The narrator is tagged "designed" in output file names. +## Run the server -## Voice clone +Start the demo matching what you want to do: ```bash -qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7860 [--no-flash-attn] -``` - -Then in another terminal: - -```bash -python audiobook.py --backend qwen --clone reference.wav -``` - -The reference `.wav` should be ~10-15 seconds (3 second minimum, 60 second maximum; ~15 seconds is ideal). Longer is **not** better. +# Built-in speakers (pick a speaker with --voice in TAG) +app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860 -Whisper (`faster_whisper` or `whisper`) is used automatically to transcribe the reference audio. Without a Whisper backend it falls back to x-vector-only cloning. Override with `--transcription "What the .wav says"` or skip transcription with `--no-transcription`. +# Voice cloning (needs a reference clip) +app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --ip 127.0.0.1 --port 7860 -## Custom voice (i.e. built-in voice) - -```bash -source audiobook/bin/activate -qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --ip 127.0.0.1 --port 7860 [--no-flash-attn] +# Voice design (style described with --instructions in TAG) +app/envs/qwen/bin/qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign --ip 127.0.0.1 --port 7860 ``` -```bash -python audiobook.py --backend qwen -``` - -Change the voice settings in `app/converter/config.py`. - -## Optional: FlashAttention for qwen-tts-demo server - -FlashAttention provides a *small* speed boost on the `qwen` backend. It is **not** relevant with other backends, and switching to either of those will provide a bigger speed boost. +- Add `--no-flash-attn` if FlashAttention is not installed. +- For cloning, reference clips of 10-15 seconds of clean speech work best (pass with `--clone <wav>`). -`qwen-tts-demo` server tries to use FlashAttention 2 by default and requires `--no-flash-attn` without it. You have two options to install FlashAttention in your python environment: - -1. Build from source (takes absolutely forever). If you run out of memory, lower MAX_JOBS until you don't. - -```bash -source audiobook/bin/activate -pip install ninja packaging psutil -MAX_JOBS=4 pip install --no-build-isolation flash-attn -``` - -2. pip install a prebuilt wheel matching your torch / CUDA / Python / CXX11-ABI combination: +## Use it ```bash -source audiobook/bin/activate -python -c "import torch; print(torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI)" +python audiobook.py --backend qwen --api-url http://127.0.0.1:7860 ``` - -- [Official wheels](https://github.com/Dao-AILab/flash-attention/releases) - Pick `cp312` + matching `cuX` + `torchX.Y` + `cxx11abiTRUE/FALSE` -- [Third-party wheels](https://mjunya.com/flash-attention-prebuild-wheels/) |