diff options
| author | historia <historiavg@proton.me> | 2026-08-20 22:58:52 -0400 |
|---|---|---|
| committer | historia <historiavg@proton.me> | 2026-08-20 22:58:52 -0400 |
| commit | 5c3df0a434059bd0d541bda35a51e49e3c44dd55 (patch) | |
| tree | 1d18e5f41ed9fc1184275a2a2b1a6555dffa4dc4 | |
| parent | 0c197324f5444b448c285d2a57bd0a5834c2fc84 (diff) | |
| download | tts-audiobook-generator-5c3df0a434059bd0d541bda35a51e49e3c44dd55.tar.gz | |
feat: experimental support for non-qwen models
| -rw-r--r-- | README.md | 99 | ||||
| -rwxr-xr-x | audiobook.py | 20 | ||||
| -rw-r--r-- | converter/config.py | 10 | ||||
| -rw-r--r-- | converter/converter.py | 1 | ||||
| -rw-r--r-- | converter/tts.py | 296 | ||||
| -rw-r--r-- | tests/test_make_audiocpp_server_json.py | 222 | ||||
| -rw-r--r-- | tests/test_tts.py | 151 | ||||
| -rwxr-xr-x | tools/make_audiocpp_server_json.py | 415 |
8 files changed, 1007 insertions, 207 deletions
@@ -2,7 +2,7 @@ Convert TXT, PDF, and EPUB files into audiobooks using the Qwen3-TTS voice model. -This builds upon [WhiskeyCoder/Qwen3-Audiobook-Converter](https://github.com/WhiskeyCoder/Qwen3-Audiobook-Converter) adding more output files, support for faster backends, metadata, generated cover art, transcription/speed/language options, better text cleanup, and clearer instructions. The converter supports three TTS backends, selected with `--backend`: the original Qwen3-TTS Gradio demos (`gradio`, default), [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts) (`faster`), and [audio.cpp](https://github.com/0xShug0/audio.cpp) (`audiocpp`) — all serving the same Qwen3-TTS 1.7B model. +This builds upon [WhiskeyCoder/Qwen3-Audiobook-Converter](https://github.com/WhiskeyCoder/Qwen3-Audiobook-Converter) adding more output files, support for faster backends, metadata, generated cover art, transcription/speed/language options, better text cleanup, and clearer instructions. The converter supports three TTS backends, selected with `--backend`: the original Qwen3-TTS Gradio demos (`gradio`, default), [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts) (`faster`), and [audio.cpp](https://github.com/0xShug0/audio.cpp) (`audiocpp`) — the last of which can serve either the same Qwen3-TTS 1.7B model (Option 3) or any of audio.cpp's larger non-Qwen TTS families like Higgs Audio v3 4B, VoxCPM2, and IndexTTS-2/2.5 (Option 4). ## Overview @@ -10,7 +10,7 @@ The converter sends text extracted from your books to a locally running Qwen3-TT - Input: `.txt`, `.pdf`, or `.epub` - Output: `.m4b`, `.mp3`, `.ogg`, or `.flac` -- Supports [qwen-tts](https://pypi.org/project/qwen-tts/), [faster-qwen-tts](https://github.com/andimarafioti/faster-qwen3-tts), and [audio.cpp](https://github.com/0xShug0/audio.cpp) backend servers +- Supports [qwen-tts](https://pypi.org/project/qwen-tts/), [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts), and [audio.cpp](https://github.com/0xShug0/audio.cpp) backend servers (audio.cpp can host any of its TTS model families, not just Qwen3-TTS) - Output a single file or one per chapter - Automatic metadata (title/artist/album tags, chapter track numbers) and a generated cover - Clone voices from .wav reference files or use the built-in speaker in the CustomVoice model. @@ -39,7 +39,8 @@ You will also need to install one of the following backends (see below for insta | -------------------------------------------------------------------- | ------------------------------------------------- | | [Qwen-TTS](https://pypi.org/project/qwen-tts/) | Gradio server released by Qwen | | [Faster-Qwen-TTS](https://github.com/andimarafioti/faster-qwen3-tts) | Server with 2-8x faster inference for NVidia GPUs | -| [audio.cpp](https://github.com/0xShug0/audio.cpp) | Newer C++ TTS backend that supports Qwen-TTS | +| [audio.cpp](https://github.com/0xShug0/audio.cpp) (Qwen) | Newer C++ TTS backend that supports Qwen-TTS | +| [audio.cpp](https://github.com/0xShug0/audio.cpp) (other families) | Same backend hosting larger/higher-quality models | ## Options @@ -52,8 +53,8 @@ You will also need to install one of the following backends (see below for insta | `--speed <n>` | Playback speed, pitch-preserving (`1.0` = normal). A normal-speed copy is also output. | | `--single-file` | Merge all chapters into a single file (default: one file per chapter). `m4b` is always one file. | | `--language <lang>` | Output language for the synthesized speech. Can add an accent even if the text is English. | -| `--backend {gradio,faster,audiocpp}` | TTS server to use (default `gradio`). `faster` and `audiocpp` require their server running first — see the backend sections above. | -| `--voice <name>` | Voice to request from a server-side voice configuration (`--backend faster` or `audiocpp` only). | +| `--backend {gradio,faster,audiocpp}` | TTS server to use (default `gradio`). `faster` and `audiocpp` require their server running first — see the backend sections above. With `audiocpp` the server may host any audio.cpp TTS model family (see Option 4). | +| `--voice <name>` | Voice to request from a server-side voice configuration (`--backend faster` or `audiocpp` only). Required for audio.cpp families without built-in speakers (everything except Qwen3-TTS). | | `--chunk` | Force client-side chunking into `CHUNK_SIZE`-word requests. Only matters for `--backend audiocpp`, which otherwise sends each chapter as one request and lets the server chunk long text itself (may double-chunk); the `gradio` and `faster` backends always chunk. | | `--debug` | Troubleshooting: dump each chunk's raw audio and sent text to `debug/` and log every request. | @@ -146,7 +147,7 @@ Then from another terminal, run audiobook.py with `--backend faster` python audiobook.py --backend faster [--voice NAME] ``` -## Backend Option 3: audio.cpp +## Backend Option 3: audio.cpp with Qwen3-TTS Build `audiocpp_server` for your platform and backend `(cuda, vulkan, hip, cpu)`. Check [audio.cpp's readme](https://github.com/0xShug0/audio.cpp) for details. I'm using one of the helper scripts: @@ -216,6 +217,92 @@ python audiobook.py --backend audiocpp python audiobook.py --backend audiocpp --voice narrator ``` +## Backend Option 4: audio.cpp with non-Qwen models + +The same `audiocpp_server` can host most of audio.cpp's other TTS model families, including models that are larger or higher quality than Qwen3-TTS 1.7B. The converter detects the model family from the server at startup and adapts its requests automatically (language codes, style instructions, etc.), so no other converter settings change: point `AUDIOCPP_MODEL_ID` at the entry you want, start the server, and convert with `--backend audiocpp --voice <name>`. + +One difference from Qwen3-TTS: **all of these families are clone-only** — they have no built-in speakers, so a reference voice must be configured on the server and selected with `--voice`. Running without `--voice` fails fast with a hint instead of synthesizing a random voice. + +Supported families (see [audio.cpp's model list](https://github.com/0xShug0/audio.cpp#supported-models) for the full catalog): + +| Family | Model | Languages | Notes | +| ------------------------------------------- | --------------------- | ------------------------ | ------------------------------------------------ | +| `higgs_audio_tts` | Higgs Audio v3 TTS 4B | 100+ | Largest TTS in audio.cpp; expressive, inline emotion/style control | +| `voxcpm2` | VoxCPM2-2B | 29 listed | 48 kHz output (others are 24 kHz); cloning + "ultimate clone" (audio + transcript) | +| `index_tts2` | IndexTTS-2 | zh, en | Top-tier cloning quality | +| `index_tts2` (package `index_tts2_5_*`) | IndexTTS-2.5 | zh, en, ja, es, ar | Multilingual IndexTTS variant | + +The converter also works with families not in this table (Fish Audio, Chatterbox, DotTTS, OmniVoice, ...) through its generic profile: clone-only, voice from `--voice`, language detected by the model itself. Anything you can host in `audiocpp_server` with `"task": "tts"` should work. + +### Install and run + +Build `audiocpp_server` exactly as in Option 3 (same binary), then download a model package with audio.cpp's model manager from the audio.cpp checkout: + +```bash +python3 tools/model_manager_v2.py install higgs_audio_tts_4b_q8_0 +# or: python3 tools/model_manager_v2.py install voxcpm2_q8_0 +# or: python3 tools/model_manager_v2.py install index_tts2_q8_0 +# or: python3 tools/model_manager_v2.py install index_tts2_5_q8_0 +``` + +Create a `server.json` hosting the model plus your cloning voices as `voice_presets`. Note that `id` must match `AUDIOCPP_MODEL_ID` in `converter/config.py` (set `AUDIOCPP_CLONE_MODEL_ID` to the same id — single-model servers use one entry for both): + +```json +{ + "host": "127.0.0.1", + "port": 8080, + "backend": "cuda", + "lazy_load": false, + "models": [ + { + "id": "higgs", + "family": "higgs_audio_tts", + "path": "models/Higgs-Audio-v3-TTS-4B-GGUF", + "task": "tts", + "mode": "offline", + "voice_presets": { + "narrator": { + "voice_ref": "/path/to/reference.wav", + "reference_text": "Transcript of the reference audio." + } + } + } + ] +} +``` + +Start the server and convert: + +```bash +./build/linux-cuda-release/bin/audiocpp_server --config server.json + +# In another terminal +python audiobook.py --backend audiocpp --voice narrator +``` + +VRAM note: the 4B Higgs Audio Q8_0 package needs roughly 2.5x the memory of the 1.7B Qwen3-TTS packages; VoxCPM2-2B and IndexTTS-2 sit in between. BF16/F16 packages roughly double the footprint again. + +### Generating server.json with make_audiocpp_server_json + +`tools/make_audiocpp_server_json.py` supports the families above directly, including automatic whisper transcription of your reference wavs and updating `converter/config.py` to point at the generated entry: + +```bash +# Interactive: pick the family from a menu +python tools/make_audiocpp_server_json.py path/to/clone/wavs + +# Fully specified: Higgs Audio with wavs transcribed into voice presets +python tools/make_audiocpp_server_json.py path/to/clone/wavs \ + --family higgs_audio_tts --model-id higgs \ + --model-path models/Higgs-Audio-v3-TTS-4B-GGUF \ + --backend cuda --output server.json --force +``` + +The tool offers to rewrite `AUDIOCPP_MODEL_ID`/`AUDIOCPP_CLONE_MODEL_ID` in `converter/config.py` to the new entry id so `audiobook.py` talks to it without manual editing (answer "y" at the prompt). As with the Qwen flow, transcripts matter a lot for cloning quality — fill in any empty `reference_text` fields by hand before starting the server. + +### Language handling + +`--language` works with these families too, adapted per family: IndexTTS sends a language code (`English` → `en`), while Higgs Audio and VoxCPM2 detect the language from the text themselves and omit the field. `--language Auto` never sends a language. + ## Optional: FlashAttention for qwen-tts-demo server FlashAttention provides a small speed boost on the `qwen-tts-demo` backend. It is **not** relevant with the `faster` or `audiocpp` backends, and switching to either of those will provide a bigger speed boost. diff --git a/audiobook.py b/audiobook.py index 0b3734c..a984a0a 100755 --- a/audiobook.py +++ b/audiobook.py @@ -4,8 +4,6 @@ Qwen-Based Audiobook Converter Converts TXT, PDF and EPUB files into audiobooks using a local Qwen3-TTS server. Edit converter/config.py to change voice and processing settings. - -License: MIT """ import argparse @@ -89,7 +87,10 @@ Examples: metavar="LANG", help=("Output language for the synthesized speech, e.g. English, Japanese, " "or Auto (language names and short codes like en/ja are accepted). " - "Defaults to the LANGUAGE setting in converter/config.py (English).") + "With --backend audiocpp the language is adapted to the model " + "family: sent as a code (e.g. 'en') for families that take one, or " + "omitted when the model detects the language itself. Defaults to " + "the LANGUAGE setting in converter/config.py (English).") ) parser.add_argument( @@ -120,8 +121,10 @@ Examples: default=config.BACKEND, help=("TTS server to talk to: the Qwen3-TTS Gradio demos (gradio), the " "faster-qwen3-tts OpenAI-compatible server (faster), or an " - "audio.cpp audiocpp_server (audiocpp). Defaults to the BACKEND " - "setting in converter/config.py (gradio).") + "audio.cpp audiocpp_server (audiocpp) hosting any of its TTS " + "model families — Qwen3-TTS, Higgs Audio, VoxCPM2, IndexTTS2, " + "and more. Defaults to the BACKEND setting in " + "converter/config.py (gradio).") ) parser.add_argument( @@ -132,9 +135,10 @@ Examples: help=("Voice to request from a server-side voice configuration. faster: " "a key in the server's voices.json ('default' when it was started " "with --ref-audio). audiocpp: a voice_preset or voice_dir entry " - "(cloning); without this flag the audiocpp backend uses a built-in " - "CustomVoice speaker instead. Not used by the gradio backend " - "(use converter/config.py SPEAKER or --clone there).") + "(cloning); required for audio.cpp families without built-in " + "speakers (everything except Qwen3-TTS CustomVoice). Not used by " + "the gradio backend (use converter/config.py SPEAKER or --clone " + "there).") ) parser.add_argument( diff --git a/converter/config.py b/converter/config.py index 83778ab..5cf4a8f 100644 --- a/converter/config.py +++ b/converter/config.py @@ -54,6 +54,14 @@ FASTER_VOICE = "default" ############################################################################### AUDIOCPP_API_URL = "http://127.0.0.1:8080" # audio.cpp audiocpp_server -# Model ids in the audio.cpp server.json config. +# Model ids in the audio.cpp server.json config. AUDIOCPP_MODEL_ID may point +# at any TTS model entry the server hosts (qwen3_tts, higgs_audio_tts, +# voxcpm2, index_tts2, ...); the family is detected from the server at +# startup and adapts the request automatically. Only qwen3_tts has built-in +# speakers (speaker mode); every other family needs --voice with a +# server-side voice preset. For single-model servers, set +# AUDIOCPP_CLONE_MODEL_ID to the same id as AUDIOCPP_MODEL_ID (or leave it +# empty); for Qwen3-TTS it typically names a second entry with the Base +# (cloning) model. AUDIOCPP_MODEL_ID = "qwen" AUDIOCPP_CLONE_MODEL_ID = "qwen-clone" diff --git a/converter/converter.py b/converter/converter.py index 84ac766..de302ad 100644 --- a/converter/converter.py +++ b/converter/converter.py @@ -562,6 +562,7 @@ class AudiobookConverter: elif self.backend == BACKEND_AUDIOCPP: print(f"audio.cpp endpoint: {config.AUDIOCPP_API_URL}") print(f"Model id: {self.tts.model_id}") + print(f"Model family: {getattr(self.tts, 'family', 'unknown')}") if self.voice: print("Backend: audio.cpp (voice cloning, reference configured on server)") print(f"Voice: {self.voice}") diff --git a/converter/tts.py b/converter/tts.py index 142cf6d..1a41643 100644 --- a/converter/tts.py +++ b/converter/tts.py @@ -5,8 +5,10 @@ FasterTTSClient talks to the OpenAI-compatible server from the faster-qwen3-tts repository (voice cloning only; the reference voice is configured server-side — see the "Faster backend" section of the README). AudioCppTTSClient talks to the audiocpp_server from the audio.cpp -repository, which serves the same Qwen3-TTS models through an -OpenAI-style API (see the "audio.cpp backend" section of the README). +repository, which can host any TTS model family audio.cpp supports +(Qwen3-TTS, Higgs Audio, VoxCPM2, IndexTTS2, ...) through one OpenAI-style +API; the family is detected from the server at startup (see the +"audio.cpp backend" sections of the README). """ import contextlib @@ -80,6 +82,74 @@ TTS_LANGUAGE_ALIASES = { "en-gb": "English", } +# Qwen display names -> ISO 639-1 codes, for audio.cpp families whose +# language request option takes a code instead of a display name. "Auto" +# has no code and maps to None so the field is omitted and the server +# applies its own default. +LANGUAGE_ISO_CODES = { + "Chinese": "zh", + "English": "en", + "German": "de", + "Italian": "it", + "Portuguese": "pt", + "Spanish": "es", + "Japanese": "ja", + "Korean": "ko", + "French": "fr", + "Russian": "ru", +} + +# --- audio.cpp model families --------------------------------------------- +# +# audiocpp_server exposes the same OpenAI-style API for every TTS family it +# hosts; families only differ in a few request conventions, captured here as +# profiles. Families that are not listed use the default profile below. + +# How the "language" request field is expressed by a family. +AUDIOCPP_LANG_DISPLAY = "display" # Qwen display names, e.g. "English" +AUDIOCPP_LANG_ISO = "iso" # ISO 639-1 codes, e.g. "en" +AUDIOCPP_LANG_OMIT = "omit" # no language field; the model detects it + +# The only family with a built-in speaker mode (CustomVoice speaker names +# plus the INSTRUCT style prompt). Every other family is clone-only: the +# voice comes from a server-side preset requested with --voice. +AUDIOCPP_FAMILY_QWEN3_TTS = "qwen3_tts" + + +class AudioCppFamilyProfile: + """Request conventions of one audio.cpp model family.""" + + def __init__(self, language_style: str = AUDIOCPP_LANG_OMIT, + sends_instructions: bool = False, + builtin_speakers: bool = False): + self.language_style = language_style + self.sends_instructions = sends_instructions + self.builtin_speakers = builtin_speakers + + +# Generic profile for families not listed in AUDIOCPP_FAMILY_PROFILES: +# clone-only, no style instructions, and no language field (the model +# detects the language itself). Describes higgs_audio_tts, voxcpm2, +# fish_audio, dots_tts, dramabox, omnivoice, outetts, glm_tts, miotts, +# moss_tts_*, pocket_tts, vibevoice, ... as well as families added to +# audio.cpp after this table was written. +AUDIOCPP_DEFAULT_FAMILY_PROFILE = AudioCppFamilyProfile() + +AUDIOCPP_FAMILY_PROFILES = { + AUDIOCPP_FAMILY_QWEN3_TTS: AudioCppFamilyProfile( + language_style=AUDIOCPP_LANG_DISPLAY, + sends_instructions=True, + builtin_speakers=True, + ), + # Families whose language option takes a code (e.g. "en") instead of + # a Qwen display name; otherwise clone-only like the default profile. + "chatterbox": AudioCppFamilyProfile(language_style=AUDIOCPP_LANG_ISO), + "confucius4_tts": AudioCppFamilyProfile(language_style=AUDIOCPP_LANG_ISO), + "index_tts2": AudioCppFamilyProfile(language_style=AUDIOCPP_LANG_ISO), + "magpie_tts": AudioCppFamilyProfile(language_style=AUDIOCPP_LANG_ISO), + "supertonic": AudioCppFamilyProfile(language_style=AUDIOCPP_LANG_ISO), +} + # Canonical speaker names -> display names used by the qwen-tts demo. SPEAKER_DISPLAY_NAMES = { "ryan": "Ryan", @@ -723,31 +793,37 @@ class FasterTTSClient(_BaseTTSClient): class AudioCppTTSClient(_BaseTTSClient): """Generates audio chunks through an audio.cpp audiocpp_server. - Talks to the OpenAI-style HTTP API of audiocpp_server, which serves - the same Qwen3-TTS models as the Gradio demos through a native - ggml runtime (GGUF weights, no Python serving stack). Two voice - modes, both resolved server-side from the request's "voice" field: - - - Speaker mode (no ``voice``): a built-in CustomVoice speaker name - (e.g. "Vivian") is passed through, plus the INSTRUCT style prompt. - The server must be configured with the CustomVoice model for this. + Talks to the OpenAI-style HTTP API of audiocpp_server, which hosts TTS + model families through a native ggml runtime (GGUF weights, no Python + serving stack). The server API is family-agnostic; the family of the + configured model entry is read from GET /v1/models at startup and + adapts the request payload (language field style, style instructions) + through AUDIOCPP_FAMILY_PROFILES. Two voice modes, both resolved + server-side from the request's "voice" field: + + - Speaker mode (no ``voice``): Qwen3-TTS only. A built-in CustomVoice + speaker name (e.g. "Vivian") is passed through, plus the INSTRUCT + style prompt. The server must be configured with the CustomVoice + model for this. Families without built-in speakers reject this mode + with a hint to pass --voice. - Preset mode (``voice=NAME``): a voice configured on the server (``voice_presets`` or ``voice_dir`` in its config, e.g. a cloning reference). The name is validated against GET /v1/audio/voices at startup because an unresolvable name would silently fall back to - plain TTS on the Base model instead of failing. When - AUDIOCPP_CLONE_MODEL_ID names a second server entry (typically the - Base model), preset requests are routed to it. Only the entry - actually used needs to exist on the server: a clone-only (Base) - server works for --voice runs, while speaker mode on such a server - fails with a hint to pass --voice. - - Chunking: the server does its own long-form text chunking (its - ``text_chunk_size`` option, 8192 chars by default for qwen3_tts), so by - default each chapter is sent as a single request and the audio comes - back already stitched. With ``chunk_text=True`` (the --chunk CLI flag), - text is instead split client-side into CHUNK_SIZE-word sub-requests, - which may needlessly double-chunk — the warning is printed by the CLI. + plain TTS on a clone-based model instead of failing. When + AUDIOCPP_CLONE_MODEL_ID names a second server entry of the same + family (typically the Qwen Base model), preset requests are routed + to it. Only the entry actually used needs to exist on the server: a + clone-only (Base) server works for --voice runs, while speaker mode + on such a server fails with a hint to pass --voice. + + Chunking: the server does its own long-form text chunking for every + family (its ``text_chunk_size`` option, with a per-family default), so + by default each chapter is sent as a single request and the audio + comes back already stitched. With ``chunk_text=True`` (the --chunk CLI + flag), text is instead split client-side into CHUNK_SIZE-word + sub-requests, which may needlessly double-chunk — the warning is + printed by the CLI. Each response is a complete WAV file, so sub-request audio is concatenated with the same lossless path used for the Gradio client. @@ -771,27 +847,49 @@ class AudioCppTTSClient(_BaseTTSClient): # server does its own long-form chunking (text_chunk_size); when True, # text is split client-side into CHUNK_SIZE-word sub-requests first. self.chunk_text = bool(chunk_text) + # Family of the selected model entry and its request profile; both + # are resolved from GET /v1/models during _connect. + self.family = "" + self.profile = AUDIOCPP_DEFAULT_FAMILY_PROFILE + self._connect() + + # ------------------------------------------------------------------ + # Connection + # ------------------------------------------------------------------ + + def _connect(self) -> None: + """Health-check the server and resolve the model, family, and voice. + + Speaker mode is only offered to families with built-in speakers + (Qwen3-TTS); every other family must select a server-side voice + with --voice, so it fails fast with a hint instead of silently + synthesizing with a random default voice. + """ self._check_health() - model_ids = self._list_model_ids() + models = self._list_models() + if self.preset_mode: + self._select_model(models) + self._require_model_id(models) + self._resolve_family(models) if self.preset_mode: - # Resolve the model before validating so the check covers the - # id actually used; a clone-only server works for --voice runs. - self._select_model(model_ids) - self._require_model_id(model_ids) self._check_voice() print(f"[OK] Connected to audio.cpp server at {self.api_url} " - f"(model '{self.model_id}', voice '{self.voice}')") - else: - self._require_model_id(model_ids) + f"(model '{self.model_id}', family '{self.family}', " + f"voice '{self.voice}')") + elif self.profile.builtin_speakers: print(f"[OK] Connected to audio.cpp server at {self.api_url} " - f"(model '{self.model_id}', speaker '{self.voice}')") + f"(model '{self.model_id}', family '{self.family}', " + f"speaker '{self.voice}')") print("[INFO] Speaker mode expects the server to be configured with the " "CustomVoice model; with the Base model the speaker name is ignored " "and a random default voice is used (see README).") - - # ------------------------------------------------------------------ - # Connection - # ------------------------------------------------------------------ + else: + raise RuntimeError( + f"The audio.cpp model '{self.model_id}' (family " + f"'{self.family}') has no built-in speakers, so its voice " + "must come from the server: rerun with --voice NAME " + "matching a voice_preset or voice_dir entry in the server " + "config (see README).") def _get_json(self, path: str, timeout: int = 10) -> Dict[str, Any]: """GET a JSON document from the server.""" @@ -825,8 +923,8 @@ class AudioCppTTSClient(_BaseTTSClient): f"The audio.cpp server at {self.api_url} reports status " f"{payload.get('status')!r} instead of 'ok'") - def _list_model_ids(self) -> List[str]: - """Fetch the model ids reported by the server.""" + def _list_models(self) -> List[Dict[str, str]]: + """Fetch the (id, family) pairs reported by the server.""" try: payload = self._get_json("/v1/models") except Exception as exc: @@ -834,16 +932,23 @@ class AudioCppTTSClient(_BaseTTSClient): f"The audio.cpp server at {self.api_url} did not answer " f"/v1/models: {exc}") from exc entries = payload.get("data") or [] - model_ids = [entry.get("id") for entry in entries if isinstance(entry, dict)] - return [mid for mid in model_ids if mid] - - def _require_model_id(self, model_ids: List[str]) -> None: + models: List[Dict[str, str]] = [] + for entry in entries: + if isinstance(entry, dict) and entry.get("id"): + models.append({ + "id": entry["id"], + "family": entry.get("family") or "", + }) + return models + + def _require_model_id(self, models: List[Dict[str, str]]) -> None: """Verify the model id chosen for this run exists on the server. Speaker mode needs AUDIOCPP_MODEL_ID (the CustomVoice entry). Preset mode validates whichever id _select_model resolved, so a - server hosting only the Base (cloning) model works for --voice. + server hosting only a cloning model works for --voice. """ + model_ids = [model["id"] for model in models] if self.model_id in model_ids: return configured = ", ".join(model_ids) or "none" @@ -852,39 +957,90 @@ class AudioCppTTSClient(_BaseTTSClient): f"The audio.cpp server at {self.api_url} has no model id " f"'{self.model_id}' or clone model id " f"'{config.AUDIOCPP_CLONE_MODEL_ID}' (configured: {configured}). " - "Add a qwen3_tts model entry to the server config and match " - "AUDIOCPP_MODEL_ID / AUDIOCPP_CLONE_MODEL_ID in " - "converter/config.py to its id (see README)." + "Add a TTS model entry for the family you want to the server " + "config and match AUDIOCPP_MODEL_ID / AUDIOCPP_CLONE_MODEL_ID " + "in converter/config.py to its id (see README)." ) raise RuntimeError( f"The audio.cpp server at {self.api_url} has no model id " f"'{self.model_id}' (configured: {configured}). Speaker mode needs " - "the CustomVoice model: add a qwen3_tts model entry to the server " - "config and match AUDIOCPP_MODEL_ID in converter/config.py to its " - "id, or rerun with --voice to use a cloning preset on the Base " + "the Qwen3-TTS CustomVoice model: add a qwen3_tts model entry to " + "the server config and match AUDIOCPP_MODEL_ID in converter/config.py to its " + "id, or rerun with --voice to use a voice preset on any TTS " "model (see README)." ) - def _select_model(self, model_ids: List[str]) -> None: + def _select_model(self, models: List[Dict[str, str]]) -> None: """Pick the model for preset (cloning) requests. Defaults to the primary model id. When AUDIOCPP_CLONE_MODEL_ID is - configured (typically a Base-model entry, since only that variant - consumes reference audio) and present on the server, preset - requests are routed to it instead, so one server can host the - CustomVoice model for speaker mode and the Base model for - cloning. + configured and present on the server, preset requests are routed + to it instead, so one server can host the CustomVoice model for + speaker mode and the Base model for cloning (Qwen3-TTS setups). + A clone id that names a model of a different family is ignored + with a warning, since preset requests must synthesize with the + family the run is configured for. """ clone_model_id = config.AUDIOCPP_CLONE_MODEL_ID if not clone_model_id or clone_model_id == self.model_id: return - if clone_model_id in model_ids: - self.model_id = clone_model_id - else: + families = {model["id"]: model["family"] for model in models} + if clone_model_id not in families: + # A qwen3_tts primary without its clone entry silently degrades + # (presets are ignored on the CustomVoice model), so that case + # keeps the warning; single-model servers of other families are + # the normal configuration and only get a debug note. + primary_is_qwen = (families.get(self.model_id) + or AUDIOCPP_FAMILY_QWEN3_TTS) \ + == AUDIOCPP_FAMILY_QWEN3_TTS + if primary_is_qwen: + logger.warning( + "AUDIOCPP_CLONE_MODEL_ID %r is not configured on the audio.cpp " + "server; preset requests use '%s' instead", + clone_model_id, self.model_id) + else: + logger.debug( + "AUDIOCPP_CLONE_MODEL_ID %r is not configured on the audio.cpp " + "server; preset requests use '%s' instead", + clone_model_id, self.model_id) + return + primary_family = families.get(self.model_id) + clone_family = families[clone_model_id] + if primary_family and clone_family and primary_family != clone_family: logger.warning( - "AUDIOCPP_CLONE_MODEL_ID %r is not configured on the audio.cpp " - "server; preset requests use '%s' instead", - clone_model_id, self.model_id) + "AUDIOCPP_CLONE_MODEL_ID %r hosts family %r, but " + "AUDIOCPP_MODEL_ID %r hosts %r; preset requests stay on " + "'%s'. Point both ids at the same model entry in " + "converter/config.py (single-model servers use the same id " + "for both)", + clone_model_id, clone_family, self.model_id, primary_family, + self.model_id) + return + self.model_id = clone_model_id + + def _resolve_family(self, models: List[Dict[str, str]]) -> None: + """Resolve the selected model's family and its request profile. + + The family comes from GET /v1/models. Servers that predate the + family field served Qwen3-TTS only, so a missing family is treated + as qwen3_tts, which also preserves this client's legacy behavior + against those versions. + """ + entry = next( + (model for model in models if model["id"] == self.model_id), None) + family = (entry["family"] if entry is not None else "") or "" + if not family: + family = AUDIOCPP_FAMILY_QWEN3_TTS + logger.debug("Model '%s' reported no family; assuming qwen3_tts", + self.model_id) + self.family = family + self.profile = AUDIOCPP_FAMILY_PROFILES.get( + family, AUDIOCPP_DEFAULT_FAMILY_PROFILE) + if family not in AUDIOCPP_FAMILY_PROFILES: + logger.info( + "audio.cpp family '%s' has no dedicated profile; using the " + "generic profile (voice cloning via --voice, model-detected " + "language)", family) def _check_voice(self) -> None: """Verify the requested voice is available on the server. @@ -929,15 +1085,25 @@ class AudioCppTTSClient(_BaseTTSClient): "model": self.model_id, "input": text, "voice": self.voice, - "language": self.language, } + if self.profile.language_style == AUDIOCPP_LANG_DISPLAY: + payload["language"] = self.language + elif self.profile.language_style == AUDIOCPP_LANG_ISO: + iso_code = LANGUAGE_ISO_CODES.get(self.language) + if iso_code: + payload["language"] = iso_code + else: + # "Auto": no code to send, so let the server pick its default. + logger.debug("%s: no language code for %r; omitted from request", + self.family, self.language) if self._seed >= 0: # audio.cpp has no negative "randomize" seed; a negative seed # means "let the server randomize", so the field is omitted. payload["seed"] = self._seed - if not self.preset_mode and config.INSTRUCT: - # Style instruction for the CustomVoice speakers; ignored by - # the Base (cloning) model. + if not self.preset_mode and config.INSTRUCT \ + and self.profile.sends_instructions: + # Style instruction for the Qwen3-TTS CustomVoice speakers; + # ignored by the Base (cloning) model and other families. payload["instructions"] = config.INSTRUCT request = urllib.request.Request( url, data=json.dumps(payload).encode("utf-8"), diff --git a/tests/test_make_audiocpp_server_json.py b/tests/test_make_audiocpp_server_json.py index bdc3604..f9fa794 100644 --- a/tests/test_make_audiocpp_server_json.py +++ b/tests/test_make_audiocpp_server_json.py @@ -20,6 +20,13 @@ FAKE_CONFIG = ( "CHUNK_SIZE = 250\n" ) +FAKE_CONFIG_WITH_MODEL_IDS = ( + 'AUDIOCPP_API_URL = "http://127.0.0.1:9999" # audio.cpp audiocpp_server\n' + "\n" + 'AUDIOCPP_MODEL_ID = "qwen" # server entry for speaker mode\n' + 'AUDIOCPP_CLONE_MODEL_ID = "qwen-clone"\n' +) + class FindWavFilesTests(unittest.TestCase): def setUp(self): @@ -117,6 +124,92 @@ class UpdateConfigPortTests(unittest.TestCase): 8080, config_path=Path(self._tmp.name) / "nope.py")) +class UpdateConfigModelIdsTests(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.config_path = Path(self._tmp.name) / "config.py" + self.config_path.write_text(FAKE_CONFIG_WITH_MODEL_IDS, + encoding="utf-8") + + def tearDown(self): + self._tmp.cleanup() + + def test_rewrites_both_ids_preserving_lines(self): + changed = make_server.update_config_model_ids( + "higgs", "higgs", config_path=self.config_path) + self.assertTrue(changed) + text = self.config_path.read_text(encoding="utf-8") + self.assertIn('AUDIOCPP_MODEL_ID = "higgs" # server entry for speaker mode', + text) + self.assertIn('AUDIOCPP_CLONE_MODEL_ID = "higgs"', text) + self.assertIn('AUDIOCPP_API_URL = "http://127.0.0.1:9999"', text) + + def test_clone_id_optional(self): + changed = make_server.update_config_model_ids( + "voxcpm2", config_path=self.config_path) + self.assertTrue(changed) + text = self.config_path.read_text(encoding="utf-8") + self.assertIn('AUDIOCPP_MODEL_ID = "voxcpm2"', text) + self.assertIn('AUDIOCPP_CLONE_MODEL_ID = "qwen-clone"', text) + + def test_returns_false_when_ids_unchanged(self): + changed = make_server.update_config_model_ids( + "qwen", "qwen-clone", config_path=self.config_path) + self.assertFalse(changed) + self.assertEqual(self.config_path.read_text(encoding="utf-8"), + FAKE_CONFIG_WITH_MODEL_IDS) + + def test_returns_false_when_lines_missing(self): + path = Path(self._tmp.name) / "other.py" + path.write_text('CHUNK_SIZE = 250\n', encoding="utf-8") + self.assertFalse(make_server.update_config_model_ids( + "higgs", "higgs", config_path=path)) + + def test_returns_false_when_file_missing(self): + self.assertFalse(make_server.update_config_model_ids( + "higgs", "higgs", + config_path=Path(self._tmp.name) / "nope.py")) + + +class BuildSingleFamilyServerConfigTests(unittest.TestCase): + def test_single_entry_with_presets(self): + presets = {"narrator": {"voice_ref": "/x.wav", + "reference_text": "hi"}} + server_config = make_server.build_single_family_server_config( + host="127.0.0.1", port=8080, backend="cuda", lazy_load=False, + family="higgs_audio_tts", model_id="higgs", + model_path="models/Higgs-Audio-v3-TTS-4B-GGUF", + voice_presets=presets) + self.assertEqual(server_config["host"], "127.0.0.1") + self.assertEqual(server_config["port"], 8080) + self.assertEqual(server_config["backend"], "cuda") + self.assertFalse(server_config["lazy_load"]) + self.assertEqual(len(server_config["models"]), 1) + entry = server_config["models"][0] + self.assertEqual(entry["id"], "higgs") + self.assertEqual(entry["family"], "higgs_audio_tts") + self.assertEqual(entry["path"], "models/Higgs-Audio-v3-TTS-4B-GGUF") + self.assertEqual(entry["task"], "tts") + self.assertEqual(entry["mode"], "offline") + self.assertEqual(entry["voice_presets"], presets) + + def test_no_presets_omits_key(self): + server_config = make_server.build_single_family_server_config( + host="127.0.0.1", port=8080, backend="cpu", lazy_load=True, + family="index_tts2", model_id="indextts2", + model_path="models/IndexTTS2-GGUF", voice_presets={}) + self.assertNotIn("voice_presets", server_config["models"][0]) + + def test_family_entries_reference_real_families(self): + for entry in make_server.FAMILY_ENTRIES: + if entry["key"] == make_server.FAMILY_QWEN3_TTS: + continue + self.assertIn("install", entry) + self.assertIn("default_id", entry) + self.assertIn("default_path", entry) + self.assertIn("family", entry) + + class BuildVoicePresetsTests(unittest.TestCase): def setUp(self): self._tmp = tempfile.TemporaryDirectory() @@ -279,7 +372,8 @@ class MainTests(unittest.TestCase): def _defaults(self, models="", host="", port="", backend="", lazy="", custom_path="", clone_path="", wav_dir="", confirm="y", prefix=()): - return list(prefix) + [models, host, port, backend, lazy, + # First input selects the model family (default: Qwen3-TTS). + return list(prefix) + ["", models, host, port, backend, lazy, custom_path, clone_path, wav_dir, confirm] def test_default_run_hosts_both_models(self): @@ -313,7 +407,7 @@ class MainTests(unittest.TestCase): def test_clone_only_with_positional_wav_dir(self): (self.folder / "narrator.wav").write_bytes(b"x") (self.folder / "alpha.wav").write_bytes(b"x") - inputs = ["3", "", "", "", "", "", "y"] + inputs = ["", "3", "", "", "", "", "", "y"] exit_code = self._run( [str(self.folder), "--output", str(self.output)], inputs=inputs, @@ -331,7 +425,7 @@ class MainTests(unittest.TestCase): "reference_text": "transcript of narrator.wav"}) def test_custom_only_single_model(self): - inputs = ["", "", "", "", "", "y"] + inputs = ["", "", "", "", "", "", "y"] exit_code = self._run( ["--output", str(self.output), "--models", "custom"], inputs=inputs) @@ -343,7 +437,7 @@ class MainTests(unittest.TestCase): def test_duplicate_ids_prompt_for_distinct_clone_id(self): with patch.object(config, "AUDIOCPP_MODEL_ID", "qwen"), \ patch.object(config, "AUDIOCPP_CLONE_MODEL_ID", "qwen"): - inputs = ["1", "qwen-clone-2", "", "", "", "", "", "", "", "y"] + inputs = ["", "1", "qwen-clone-2", "", "", "", "", "", "", "", "y"] exit_code = self._run(["--output", str(self.output)], inputs=inputs) self.assertEqual(exit_code, 0) @@ -362,7 +456,7 @@ class MainTests(unittest.TestCase): def test_port_sync_accepted_updates_config(self): with patch.object(config, "AUDIOCPP_API_URL", "http://127.0.0.1:9999"): - inputs = ["", "", "y", "", "", "", "", "", "y"] + inputs = ["", "", "", "y", "", "", "", "", "", "y"] exit_code = self._run(["--output", str(self.output), "--port", "8080"], inputs=inputs) @@ -375,7 +469,7 @@ class MainTests(unittest.TestCase): def test_port_sync_declined_keeps_config(self): with patch.object(config, "AUDIOCPP_API_URL", "http://127.0.0.1:9999"): - inputs = ["", "", "n", "", "", "", "", "", "y"] + inputs = ["", "", "", "n", "", "", "", "", "", "y"] exit_code = self._run(["--output", str(self.output), "--port", "8080"], inputs=inputs) @@ -394,7 +488,8 @@ class MainTests(unittest.TestCase): FAKE_CONFIG) def test_invalid_menu_choice_reprompts(self): - inputs = ["9", "", "", "", "", "", "", "", "", "y"] + # Family menu default, then an invalid models-menu choice retried. + inputs = ["", "9", "", "", "", "", "", "", "", "", "y"] exit_code = self._run(["--output", str(self.output)], inputs=inputs) self.assertEqual(exit_code, 0) @@ -435,11 +530,14 @@ class MainTests(unittest.TestCase): self.assertEqual(len(data["models"]), 2) def test_flags_skip_prompts(self): + # Family still asked (no --family flag); port 9000 differs from the + # config port so its sync prompt fires; custom/clone paths and the + # wav dir use their defaults. exit_code = self._run( ["--output", str(self.output), "--models", "both", "--host", "0.0.0.0", "--port", "9000", "--backend", "cpu", "--lazy-load"], - inputs=["y", "", "", "", "y"]) + inputs=["", "y", "", "", "", "y"]) self.assertEqual(exit_code, 0) self.assertIn('"http://127.0.0.1:9000"', self.fake_config.read_text(encoding="utf-8")) @@ -457,6 +555,105 @@ class MainTests(unittest.TestCase): self.assertEqual(ctx.exception.code, 2) +class NonQwenFamilyMainTests(unittest.TestCase): + """The --family flow for clone-only model families.""" + + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.folder = Path(self._tmp.name) + self.output = self.folder / "server.json" + self.fake_config = self.folder / "config.py" + self.fake_config.write_text(FAKE_CONFIG_WITH_MODEL_IDS, + encoding="utf-8") + patcher = patch.object(make_server, "CONFIG_PATH", self.fake_config) + patcher.start() + self.addCleanup(patcher.stop) + + def tearDown(self): + self._tmp.cleanup() + + def _run(self, argv, inputs=None, transcribe=None, whisper="faster_whisper"): + argv = ["make_audiocpp_server_json.py"] + argv + input_effect = inputs if inputs is not None else EOFError + transcribe_effect = transcribe if transcribe is not None else MagicMock() + with patch.object(sys, "argv", argv), \ + patch("builtins.input", side_effect=input_effect), \ + patch.object(make_server, "transcribe_reference_audio", + side_effect=transcribe_effect), \ + patch.object(make_server, "whisper_backend_available", + return_value=whisper): + return make_server.main() + + def test_higgs_family_run(self): + (self.folder / "narrator.wav").write_bytes(b"x") + # Inputs: model-id sync accepted, host, port, backend, lazy, confirm. + inputs = ["y", "", "", "", "", "y"] + exit_code = self._run( + [str(self.folder), "--output", str(self.output), + "--family", "higgs_audio_tts", "--model-id", "higgs", + "--model-path", "models/Higgs-Audio-v3-TTS-4B-GGUF"], + inputs=inputs, + transcribe=lambda path, model_name="base": "a transcript") + self.assertEqual(exit_code, 0) + data = json.loads(self.output.read_text(encoding="utf-8")) + self.assertEqual(len(data["models"]), 1) + entry = data["models"][0] + self.assertEqual(entry["id"], "higgs") + self.assertEqual(entry["family"], "higgs_audio_tts") + self.assertEqual(entry["path"], "models/Higgs-Audio-v3-TTS-4B-GGUF") + self.assertEqual(entry["task"], "tts") + self.assertEqual(entry["mode"], "offline") + self.assertEqual(entry["voice_presets"]["narrator"], + {"voice_ref": str((self.folder / "narrator.wav").resolve()), + "reference_text": "a transcript"}) + # Both converter model ids point at the single server entry. + self.assertIn('AUDIOCPP_MODEL_ID = "higgs"', + self.fake_config.read_text(encoding="utf-8")) + self.assertIn('AUDIOCPP_CLONE_MODEL_ID = "higgs"', + self.fake_config.read_text(encoding="utf-8")) + + def test_model_id_sync_declined_keeps_config(self): + # sync declined, host, port, backend, lazy, wav dir skipped, confirm + inputs = ["n", "", "", "", "", "", "y"] + exit_code = self._run( + ["--output", str(self.output), "--family", "voxcpm2", + "--model-id", "voxcpm2", "--model-path", "models/VoxCPM2-GGUF"], + inputs=inputs) + self.assertEqual(exit_code, 0) + text = self.fake_config.read_text(encoding="utf-8") + self.assertIn('AUDIOCPP_MODEL_ID = "qwen"', text) + self.assertIn('AUDIOCPP_CLONE_MODEL_ID = "qwen-clone"', text) + data = json.loads(self.output.read_text(encoding="utf-8")) + self.assertEqual(data["models"][0]["family"], "voxcpm2") + + def test_no_voice_presets_warns(self): + buf = io.StringIO() + # sync accepted, host, port, backend, lazy, wav dir skipped, confirm + with patch.object(sys, "argv", + ["make_audiocpp_server_json.py", + "--output", str(self.output), + "--family", "index_tts2", "--model-id", "indextts2", + "--model-path", "models/IndexTTS2-GGUF"]), \ + patch("builtins.input", side_effect=["y", "", "", "", "", "", "y"]), \ + patch.object(make_server, "transcribe_reference_audio"), \ + patch.object(make_server, "whisper_backend_available", + return_value="faster_whisper"), \ + redirect_stdout(buf): + code = make_server.main() + self.assertEqual(code, 0) + out = buf.getvalue() + self.assertIn("No voice presets were configured", out) + self.assertIn("model_manager_v2.py install index_tts2_q8_0", out) + data = json.loads(self.output.read_text(encoding="utf-8")) + self.assertNotIn("voice_presets", data["models"][0]) + + def test_models_flag_rejected_for_non_qwen_family(self): + with self.assertRaises(SystemExit) as ctx: + self._run(["--output", str(self.output), + "--family", "higgs_audio_tts", "--models", "both"]) + self.assertEqual(ctx.exception.code, 2) + + class TranscriptWarningTests(unittest.TestCase): """Empty transcripts and a missing Whisper backend produce loud warnings.""" @@ -489,10 +686,11 @@ class TranscriptWarningTests(unittest.TestCase): def test_empty_transcript_prints_loud_end_warning(self): (self.folder / "narrator.wav").write_bytes(b"x") (self.folder / "alpha.wav").write_bytes(b"x") - # Clone-only run (menu choice 3); transcribe returns None (empty). + # Qwen family default, clone-only run (menu choice 3); transcribe + # returns None (empty). code, out = self._run_capturing( [str(self.folder), "--output", str(self.output)], - inputs=["3", "", "", "", "", "", "y"], + inputs=["", "3", "", "", "", "", "", "y"], transcribe=lambda path, model_name="base": None, whisper="faster_whisper") self.assertEqual(code, 0) @@ -505,7 +703,7 @@ class TranscriptWarningTests(unittest.TestCase): (self.folder / "narrator.wav").write_bytes(b"x") code, out = self._run_capturing( [str(self.folder), "--output", str(self.output)], - inputs=["3", "", "", "", "", "", "y"], + inputs=["", "3", "", "", "", "", "", "y"], transcribe=lambda path, model_name="base": "a transcript", whisper=None) self.assertEqual(code, 0) @@ -516,7 +714,7 @@ class TranscriptWarningTests(unittest.TestCase): (self.folder / "narrator.wav").write_bytes(b"x") code, out = self._run_capturing( [str(self.folder), "--output", str(self.output)], - inputs=["3", "", "", "", "", "", "y"], + inputs=["", "3", "", "", "", "", "", "y"], transcribe=lambda path, model_name="base": "a real transcript", whisper="faster_whisper") self.assertEqual(code, 0) diff --git a/tests/test_tts.py b/tests/test_tts.py index ec357c1..e2fe921 100644 --- a/tests/test_tts.py +++ b/tests/test_tts.py @@ -687,6 +687,103 @@ class AudioCppTTSClientHealthTests(unittest.TestCase): self.assertIn("pocket-tts", message) +class AudioCppFamilyDetectionTests(unittest.TestCase): + """Family auto-detection and per-family adaptations.""" + + @staticmethod + def _json_response(payload): + response = MagicMock() + response.__enter__.return_value = response + response.read.return_value = json.dumps(payload).encode("utf-8") + return response + + def _client(self, voice="narrator", models=None): + def _dispatch(request, **_kwargs): + url = request if isinstance(request, str) else request.full_url + if url.endswith("/health"): + return self._json_response({"status": "ok"}) + if url.endswith("/v1/models"): + return self._json_response(models) + if "/v1/audio/voices" in url: + return self._json_response({"voices": [voice] if voice else []}) + raise AssertionError(f"unexpected URL: {url}") + + with patch("converter.tts.urllib.request.urlopen", + side_effect=_dispatch): + return AudioCppTTSClient(voice=voice) + + def test_family_detected_from_models_endpoint(self): + client = self._client(models={"data": [ + {"id": config.AUDIOCPP_MODEL_ID, "family": "higgs_audio_tts"}]}) + self.assertEqual(client.family, "higgs_audio_tts") + self.assertIs(client.profile, tts.AUDIOCPP_DEFAULT_FAMILY_PROFILE) + + def test_missing_family_falls_back_to_qwen3_tts(self): + client = self._client(models={"data": [ + {"id": config.AUDIOCPP_MODEL_ID}]}) + self.assertEqual(client.family, "qwen3_tts") + self.assertTrue(client.profile.builtin_speakers) + + def test_unknown_family_uses_generic_profile(self): + client = self._client(models={"data": [ + {"id": config.AUDIOCPP_MODEL_ID, "family": "future_tts"}]}) + self.assertEqual(client.family, "future_tts") + self.assertIs(client.profile, tts.AUDIOCPP_DEFAULT_FAMILY_PROFILE) + self.assertFalse(client.profile.builtin_speakers) + self.assertEqual(client.profile.language_style, tts.AUDIOCPP_LANG_OMIT) + + def test_speaker_mode_rejected_for_clone_only_family(self): + client = None + try: + client = self._client(voice=None, models={"data": [ + {"id": config.AUDIOCPP_MODEL_ID, "family": "voxcpm2"}]}) + except RuntimeError as exc: + message = str(exc) + self.assertIn("voxcpm2", message) + self.assertIn("--voice", message) + self.assertIn("no built-in speakers", message) + self.assertIsNone(client) + + def test_speaker_mode_allowed_for_qwen_family(self): + client = self._client(voice=None, models={"data": [ + {"id": config.AUDIOCPP_MODEL_ID, "family": "qwen3_tts"}]}) + self.assertEqual(client.family, "qwen3_tts") + + def test_clone_model_id_of_different_family_is_ignored(self): + with patch.object(config, "AUDIOCPP_MODEL_ID", "higgs"), \ + patch.object(config, "AUDIOCPP_CLONE_MODEL_ID", "qwen-clone"), \ + self.assertLogs("converter.tts", level="WARNING") as logs: + client = self._client(models={"data": [ + {"id": "higgs", "family": "higgs_audio_tts"}, + {"id": "qwen-clone", "family": "qwen3_tts"}]}) + self.assertEqual(client.model_id, "higgs") + self.assertTrue(any("different family" in line.lower() or + "hosts family" in line.lower() + for line in logs.output)) + + def test_clone_model_id_missing_on_non_qwen_server_is_debug_only(self): + with patch.object(config, "AUDIOCPP_MODEL_ID", "higgs"), \ + patch.object(config, "AUDIOCPP_CLONE_MODEL_ID", "qwen-clone"), \ + self.assertNoLogs("converter.tts", level="WARNING"): + client = self._client(models={"data": [ + {"id": "higgs", "family": "higgs_audio_tts"}]}) + self.assertEqual(client.model_id, "higgs") + + def test_clone_model_id_missing_on_qwen_server_still_warns(self): + with patch.object(config, "AUDIOCPP_MODEL_ID", "qwen3-tts"), \ + patch.object(config, "AUDIOCPP_CLONE_MODEL_ID", "qwen3-tts-clone"), \ + self.assertLogs("converter.tts", level="WARNING") as logs: + client = self._client(models={"data": [ + {"id": "qwen3-tts", "family": "qwen3_tts"}, + {"id": "pocket-tts", "family": "pocket_tts"}]}) + self.assertEqual(client.model_id, "qwen3-tts") + self.assertTrue(any("qwen3-tts-clone" in line for line in logs.output)) + + def test_iso_language_code_helper(self): + self.assertEqual(tts.LANGUAGE_ISO_CODES["English"], "en") + self.assertIsNone(tts.LANGUAGE_ISO_CODES.get("Auto")) + + class AudioCppTTSClientRequestTests(unittest.TestCase): """The /v1/audio/speech payload and response validation.""" @@ -704,7 +801,7 @@ class AudioCppTTSClientRequestTests(unittest.TestCase): @staticmethod def _make_client(preset_mode=False, voice="Vivian", language="English", seed=-1, - chunk_text=True): + chunk_text=True, family="qwen3_tts"): client = AudioCppTTSClient.__new__(AudioCppTTSClient) client.api_url = "http://127.0.0.1:8080" client.model_id = config.AUDIOCPP_MODEL_ID @@ -713,6 +810,9 @@ class AudioCppTTSClientRequestTests(unittest.TestCase): client.language = language client._seed = seed client.chunk_text = chunk_text + client.family = family + client.profile = tts.AUDIOCPP_FAMILY_PROFILES.get( + family, tts.AUDIOCPP_DEFAULT_FAMILY_PROFILE) return client @staticmethod @@ -795,6 +895,44 @@ class AudioCppTTSClientRequestTests(unittest.TestCase): payload = json.loads(mock_urlopen.call_args[0][0].data.decode("utf-8")) self.assertEqual(payload["instructions"], config.INSTRUCT) + def test_generic_family_omits_language_and_instructions(self): + # Clone-only families (higgs_audio_tts, voxcpm2, ...) detect the + # language themselves and take no style instruction. + client = self._make_client(preset_mode=False, family="higgs_audio_tts") + with patch("converter.tts.urllib.request.urlopen", + return_value=self._post_response(self._wav_bytes())) as mock_urlopen: + client._request_wav("Hello.") + payload = json.loads(mock_urlopen.call_args[0][0].data.decode("utf-8")) + self.assertNotIn("language", payload) + self.assertNotIn("instructions", payload) + + def test_iso_family_sends_language_code(self): + client = self._make_client(preset_mode=True, voice="narrator", + language="Japanese", family="index_tts2") + with patch("converter.tts.urllib.request.urlopen", + return_value=self._post_response(self._wav_bytes())) as mock_urlopen: + client._request_wav("Hello.") + payload = json.loads(mock_urlopen.call_args[0][0].data.decode("utf-8")) + self.assertEqual(payload["language"], "ja") + + def test_iso_family_auto_omits_language(self): + client = self._make_client(preset_mode=True, voice="narrator", + language="Auto", family="index_tts2") + with patch("converter.tts.urllib.request.urlopen", + return_value=self._post_response(self._wav_bytes())) as mock_urlopen: + client._request_wav("Hello.") + payload = json.loads(mock_urlopen.call_args[0][0].data.decode("utf-8")) + self.assertNotIn("language", payload) + + def test_qwen_language_display_name_still_sent(self): + client = self._make_client(preset_mode=True, voice="narrator", + language="Japanese", family="qwen3_tts") + with patch("converter.tts.urllib.request.urlopen", + return_value=self._post_response(self._wav_bytes())) as mock_urlopen: + client._request_wav("Hello.") + payload = json.loads(mock_urlopen.call_args[0][0].data.decode("utf-8")) + self.assertEqual(payload["language"], "Japanese") + def test_non_wav_response_rejected(self): client = self._make_client() for body in (b"", b"RIFFxxxx", b"MP3DATA-MP3DATA", b"RIFF\x00\x00\x00\x00mpeg"): @@ -899,6 +1037,8 @@ class AudioCppTTSClientTruncationTests(unittest.TestCase): client.language = "English" client._seed = -1 client.chunk_text = True + client.family = "qwen3_tts" + client.profile = tts.AUDIOCPP_FAMILY_PROFILES["qwen3_tts"] return client @staticmethod @@ -1073,6 +1213,15 @@ class BackendWiringTests(unittest.TestCase): converter = self._audiocpp_converter() converter._print_banner() + def test_audiocpp_banner_prints_model_family(self): + from contextlib import redirect_stdout + converter = self._audiocpp_converter(voice="narrator") + converter.tts.family = "higgs_audio_tts" + buffer = io.StringIO() + with redirect_stdout(buffer): + converter._print_banner() + self.assertIn("higgs_audio_tts", buffer.getvalue()) + def test_non_faster_narrator_tag_unchanged(self): with tempfile.TemporaryDirectory() as tmp: ref = Path(tmp) / "ref.wav" diff --git a/tools/make_audiocpp_server_json.py b/tools/make_audiocpp_server_json.py index c999f49..1273e43 100755 --- a/tools/make_audiocpp_server_json.py +++ b/tools/make_audiocpp_server_json.py @@ -1,26 +1,34 @@ #!/usr/bin/env python3 """Interactively generate a server.json for the audio.cpp audiocpp_server. -Asks which Qwen3-TTS models to host, pulls the model ids expected by this +Asks which TTS model family to host, pulls the model ids expected by this converter (AUDIOCPP_MODEL_ID / AUDIOCPP_CLONE_MODEL_ID) from converter/config.py, and writes a server.json that can be passed to audiocpp_server: audiocpp_server --config server.json +Hostable families: Qwen3-TTS (built-in CustomVoice speakers plus voice +cloning through the Base model) and the clone-only families Higgs Audio +v3 TTS 4B, VoxCPM2-2B, and IndexTTS-2 / 2.5 (see the "Option 4" section +of the README). The converter works with other audio.cpp TTS families +too; host them by writing server.json by hand. + Reference .wav files for voice cloning (a directory argument or an interactive prompt) are transcribed with a local Whisper backend -(faster_whisper or whisper) and added as voice_presets on the Base-model -entry. +(faster_whisper or whisper) and added as voice_presets on the cloning +model entry. Every value can also be supplied as a command-line flag; anything missing is asked interactively with the default shown in brackets. Pressing Enter accepts the default, so running the tool with no arguments and pressing -Enter through produces a server.json hosting both models on +Enter through produces a server.json hosting both Qwen3-TTS models on 127.0.0.1:8080 with the cuda backend. Usage: python tools/make_audiocpp_server_json.py [WAV_DIR] [--output PATH] + [--family {qwen3_tts,higgs_audio_tts,voxcpm2,index_tts2,index_tts2_5}] + [--model-id ID] [--model-path PATH] [--host HOST] [--port PORT] [--models {both,custom,clone}] [--backend {cuda,vulkan,hip,cpu}] [--lazy-load] [--whisper-model NAME] [--force] @@ -32,7 +40,7 @@ import re import sys import urllib.parse from pathlib import Path -from typing import Dict, Optional +from typing import Dict, List, Optional, Tuple # Allow running from any working directory. sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) @@ -49,6 +57,56 @@ CONFIG_PATH = Path(__file__).resolve().parent.parent / "converter" / "config.py" MODEL_SELECTIONS = ("both", "custom", "clone") BACKENDS = ("cuda", "vulkan", "hip", "cpu") +FAMILY_QWEN3_TTS = "qwen3_tts" + +# Families this tool can host, in menu order. "family" is the audio.cpp +# family name written to server.json (IndexTTS-2.5 uses the index_tts2 +# family; its variant is selected by the downloaded model package); +# "install" is the model_manager_v2.py package that downloads the model; +# "default_id" is the suggested server entry id; "default_path" is where +# the package lands relative to the audio.cpp checkout. +FAMILY_ENTRIES = [ + { + "key": FAMILY_QWEN3_TTS, + "label": "Qwen3-TTS 1.7B - built-in speakers + voice cloning", + "family": "qwen3_tts", + }, + { + "key": "higgs_audio_tts", + "label": "Higgs Audio v3 TTS 4B - voice cloning, 100+ languages", + "family": "higgs_audio_tts", + "install": "higgs_audio_tts_4b_q8_0", + "default_id": "higgs", + "default_path": "models/Higgs-Audio-v3-TTS-4B-GGUF", + }, + { + "key": "voxcpm2", + "label": "VoxCPM2-2B - voice cloning, multilingual, 48 kHz audio", + "family": "voxcpm2", + "install": "voxcpm2_q8_0", + "default_id": "voxcpm2", + "default_path": "models/VoxCPM2-GGUF", + }, + { + "key": "index_tts2", + "label": "IndexTTS-2 - voice cloning, Chinese/English", + "family": "index_tts2", + "install": "index_tts2_q8_0", + "default_id": "indextts2", + "default_path": "models/IndexTTS2-GGUF", + }, + { + "key": "index_tts2_5", + "label": "IndexTTS-2.5 - voice cloning, zh/en/ja/es/ar", + "family": "index_tts2", + "install": "index_tts2_5_q8_0", + "default_id": "indextts25", + "default_path": "models/IndexTTS2.5-GGUF", + }, +] +FAMILY_KEYS = tuple(entry["key"] for entry in FAMILY_ENTRIES) +FAMILY_BY_KEY = {entry["key"]: entry for entry in FAMILY_ENTRIES} + def find_wav_files(input_dir: Path) -> list: """Return the .wav files in INPUT_DIR, sorted alphabetically by name.""" @@ -136,6 +194,13 @@ def ask_menu(title: str, options: list, default_index: int = 1) -> str: print(f"Please enter a number between 1 and {len(options)}.") +def ask_family() -> str: + """Ask which model family the server should host.""" + return ask_menu( + "Which model family should the server host?", + [(entry["label"], entry["key"]) for entry in FAMILY_ENTRIES]) + + def ask_models() -> str: return ask_menu( "Which models should the server host?", @@ -233,6 +298,37 @@ def update_config_api_url_port(port: int, config_path: Optional[Path] = None) -> return True +def update_config_model_ids(model_id: str, + clone_model_id: Optional[str] = None, + config_path: Optional[Path] = None) -> bool: + """Rewrite AUDIOCPP_MODEL_ID (and AUDIOCPP_CLONE_MODEL_ID when given). + + Only the quoted id literals are replaced; surrounding lines and + comments are preserved. Returns True when the file was changed. + """ + path = Path(config_path) if config_path is not None else CONFIG_PATH + try: + text = path.read_text(encoding="utf-8") + except OSError: + return False + updates: List[Tuple[str, str]] = [("AUDIOCPP_MODEL_ID", model_id)] + if clone_model_id is not None: + updates.append(("AUDIOCPP_CLONE_MODEL_ID", clone_model_id)) + changed = False + for name, value in updates: + match = re.search(r'(?m)^(\s*' + name + r'\s*=\s*")([^"]*)(")', text) + if match and match.group(2) != value: + text = text[:match.start(2)] + value + text[match.end(2):] + changed = True + if not changed: + return False + try: + path.write_text(text, encoding="utf-8") + except OSError: + return False + return True + + def build_voice_presets(wav_files: list, whisper_model: str) -> Dict[str, dict]: """Transcribe each wav file and build the voice_presets mapping.""" presets: Dict[str, dict] = {} @@ -258,7 +354,7 @@ def build_server_config(host: str, port: int, backend: str, lazy_load: bool, custom_voice_id: str, clone_model_id: str, custom_voice_path: str, base_path: str, voice_presets: Dict[str, dict]) -> dict: - """Assemble the server.json document.""" + """Assemble the Qwen3-TTS server.json document.""" models = [] if include_custom: models.append({ @@ -288,34 +384,31 @@ def build_server_config(host: str, port: int, backend: str, lazy_load: bool, } -def _print_next_steps(output_path: Path, include_custom: bool, - include_clone: bool, voice_presets: Dict[str, dict]) -> None: - print("\nNext steps:") - print(" 1. Start the server (build path varies by platform, e.g.") - print(" ./build/linux-cuda-release/bin/):") - print(f" audiocpp_server --config {output_path}") - print(" 2. Convert a book from this repository:") - if include_custom: - print(" python audiobook.py --backend audiocpp" - " # built-in speaker") - if include_clone: - names = ", ".join(voice_presets) or "none configured yet" - print(" python audiobook.py --backend audiocpp --voice NAME" - f" # cloned voice ({names})") - if include_clone and not include_custom: - print("[INFO] Only the Base model is hosted: --voice is required, " - "since speaker mode needs the CustomVoice model.") +def build_single_family_server_config(host: str, port: int, backend: str, + lazy_load: bool, family: str, + model_id: str, model_path: str, + voice_presets: Dict[str, dict]) -> dict: + """Assemble a server.json hosting one clone-only model family entry.""" + entry = { + "id": model_id, + "family": family, + "path": model_path, + "task": "tts", + "mode": "offline", + } + if voice_presets: + entry["voice_presets"] = voice_presets + return { + "host": host, + "port": port, + "backend": backend, + "lazy_load": lazy_load, + "models": [entry], + } def print_empty_transcript_warning(voice_presets: Dict[str, dict]) -> None: - """Print a loud, final warning for voices whose transcript is empty. - - A cloning preset with an empty ``reference_text`` will not produce a - usable voice (the server has nothing to match the reference audio - against for in-context cloning), so the user must edit server.json by - hand. This is printed last, after the next-steps, so it is the last - thing seen and hardest to miss. - """ + """Print a loud, final warning for voices whose transcript is empty.""" empty = sorted(name for name, preset in voice_presets.items() if not preset.get("reference_text")) if not empty: @@ -334,25 +427,117 @@ def print_empty_transcript_warning(voice_presets: Dict[str, dict]) -> None: print(bar) +def _ask_host_port_backend_lazy(args: argparse.Namespace + ) -> Tuple[str, int, str, bool]: + """Ask for (or take from flags) the shared server settings.""" + host = args.host if args.host else ask("Bind host", DEFAULT_HOST) + port = args.port if args.port is not None else ask_port(config_port()) + if port != config_port(): + if ask_bool(f"Update AUDIOCPP_API_URL in converter/config.py to port " + f"{port} so audiobook.py talks to this server", True): + if update_config_api_url_port(port): + print(f"[OK] Updated AUDIOCPP_API_URL in {CONFIG_PATH}") + else: + print(f"[WARNING] Could not update {CONFIG_PATH}; edit " + "AUDIOCPP_API_URL by hand so audiobook.py uses the " + "new port") + else: + print("[WARNING] Left AUDIOCPP_API_URL unchanged; audiobook.py " + f"will still use port {config_port()}") + backend = args.backend if args.backend else ask_backend() + lazy_load = args.lazy_load or ask_bool( + "Load models lazily (on first use instead of at startup)", False) + return host, port, backend, lazy_load + + +def _collect_voice_presets(args: argparse.Namespace, + include_clone: bool) -> Dict[str, dict]: + """Resolve the clone-reference wav directory and transcribe it. + + Returns the voice_presets mapping (empty when no wavs were given or + found). Cloning entries only: a run without any cloning model ignores + the wav directory entirely. + """ + wav_dir: Optional[Path] = None + if args.input_dir is not None: + if include_clone: + wav_dir = args.input_dir + else: + print(f"[WARNING] Ignoring {args.input_dir}: no cloning model " + "selected, so voice presets are not used") + elif include_clone: + wav_dir = ask_wav_dir() + if wav_dir is None: + return {} + + wav_files = find_wav_files(wav_dir) + if not wav_files: + print(f"[WARNING] No .wav files found in {wav_dir}; writing the " + "config without voice presets") + return {} + if whisper_backend_available() is None: + print("[WARNING] Neither faster_whisper nor whisper was found, so " + "reference .wav files cannot be transcribed automatically and " + "every reference_text will be empty.") + print(' Did you remember to "conda activate qwen3-tts"? ' + "Transcripts must be added by hand (see the warning at the end).") + return build_voice_presets(wav_files, args.whisper_model) + + +def _offer_config_model_id_sync(model_id: str) -> None: + """Offer to point converter/config.py at a non-Qwen model entry. + + The converter requests the model id configured in AUDIOCPP_MODEL_ID, + and single-model servers use the same id for the clone entry, so both + ids are rewritten together. + """ + if config.AUDIOCPP_MODEL_ID == model_id \ + and config.AUDIOCPP_CLONE_MODEL_ID == model_id: + return + if ask_bool("Update AUDIOCPP_MODEL_ID and AUDIOCPP_CLONE_MODEL_ID in " + f"converter/config.py to '{model_id}' so audiobook.py uses " + "this model", True): + if update_config_model_ids(model_id, model_id): + print(f"[OK] Updated the model ids in {CONFIG_PATH}") + else: + print(f"[WARNING] Could not update {CONFIG_PATH}; edit " + "AUDIOCPP_MODEL_ID and AUDIOCPP_CLONE_MODEL_ID by hand so " + "audiobook.py uses this model") + else: + print("[WARNING] Left the model ids unchanged; audiobook.py will " + f"still request model '{config.AUDIOCPP_MODEL_ID}'") + + def main() -> int: parser = argparse.ArgumentParser( description="Generate a server.json for the audio.cpp audiocpp_server " - "hosting the Qwen3-TTS models used by this converter.") + "hosting a TTS model used by this converter.") parser.add_argument("input_dir", type=Path, nargs="?", default=None, help="Optional directory with .wav reference files " "to add as voice cloning presets") parser.add_argument("--output", type=Path, default=Path("server.json"), help="Output path for server.json (default: " "server.json in the current directory)") + parser.add_argument("--family", choices=FAMILY_KEYS, default=None, + help="Model family to host (default: Qwen3-TTS). " + "Non-Qwen families are clone-only and host a " + "single model entry") + parser.add_argument("--model-id", type=str, default=None, + help="Server model id for a non-Qwen family entry " + "(default: a family-based name such as 'higgs')") + parser.add_argument("--model-path", type=str, default=None, + help="Path to a non-Qwen family model package " + "(default: the model manager install location)") parser.add_argument("--host", type=str, default=None, help="Bind host for the server (default: 127.0.0.1)") parser.add_argument("--port", type=int, default=None, help="Port for the server (default: the port in " "AUDIOCPP_API_URL from converter/config.py)") parser.add_argument("--models", choices=MODEL_SELECTIONS, default=None, - help="Which models to host: both (default), custom " - "(CustomVoice speakers only), or clone " - "(Base voice cloning only)") + help="Which Qwen3-TTS models to host: both (default), " + "custom (CustomVoice speakers only), or clone " + "(Base voice cloning only). Only valid with " + "--family qwen3_tts") parser.add_argument("--backend", choices=BACKENDS, default=None, help="Inference backend audiocpp_server was built " "for (default: cuda)") @@ -374,87 +559,77 @@ def main() -> int: print("[INFO] Aborted; existing server.json kept") return 1 + family_key = args.family if args.family is not None else ask_family() + is_qwen = family_key == FAMILY_QWEN3_TTS + if not is_qwen and args.models is not None: + parser.error("--models only applies to --family qwen3_tts") + print("[INFO] Model ids from converter/config.py:") print(f" built-in speakers (CustomVoice): '{config.AUDIOCPP_MODEL_ID}'") print(f" voice cloning (Base): '{config.AUDIOCPP_CLONE_MODEL_ID}'") - selection = args.models if args.models is not None else ask_models() - include_custom = selection in ("both", "custom") - include_clone = selection in ("both", "clone") - - custom_voice_id = config.AUDIOCPP_MODEL_ID - clone_model_id = config.AUDIOCPP_CLONE_MODEL_ID - if include_custom and include_clone and custom_voice_id == clone_model_id: - print(f"[WARNING] AUDIOCPP_MODEL_ID and AUDIOCPP_CLONE_MODEL_ID are " - f"both '{custom_voice_id}' in converter/config.py, but server " - "model ids must be unique.") - clone_model_id = ask_distinct_clone_id(custom_voice_id) - - host = args.host if args.host else ask("Bind host", DEFAULT_HOST) - port = args.port if args.port is not None else ask_port(config_port()) - if port != config_port(): - if ask_bool(f"Update AUDIOCPP_API_URL in converter/config.py to port " - f"{port} so audiobook.py talks to this server", True): - if update_config_api_url_port(port): - print(f"[OK] Updated AUDIOCPP_API_URL in {CONFIG_PATH}") - else: - print(f"[WARNING] Could not update {CONFIG_PATH}; edit " - "AUDIOCPP_API_URL by hand so audiobook.py uses the " - "new port") - else: - print("[WARNING] Left AUDIOCPP_API_URL unchanged; audiobook.py " - f"will still use port {config_port()}") - - backend = args.backend if args.backend else ask_backend() - lazy_load = args.lazy_load or ask_bool( - "Load models lazily (on first use instead of at startup)", False) - - custom_voice_path = base_path = None - if include_custom: - custom_voice_path = ask("Path to the Qwen3-TTS CustomVoice GGUF package", - DEFAULT_CUSTOM_VOICE_PATH) - if include_clone: - base_path = ask("Path to the Qwen3-TTS Base GGUF package", - DEFAULT_BASE_PATH) - - wav_dir: Optional[Path] = None - if args.input_dir is not None: + if is_qwen: + selection = args.models if args.models is not None else ask_models() + include_custom = selection in ("both", "custom") + include_clone = selection in ("both", "clone") + + custom_voice_id = config.AUDIOCPP_MODEL_ID + clone_model_id = config.AUDIOCPP_CLONE_MODEL_ID + if include_custom and include_clone and custom_voice_id == clone_model_id: + print(f"[WARNING] AUDIOCPP_MODEL_ID and AUDIOCPP_CLONE_MODEL_ID are " + f"both '{custom_voice_id}' in converter/config.py, but server " + "model ids must be unique.") + clone_model_id = ask_distinct_clone_id(custom_voice_id) + else: + entry = FAMILY_BY_KEY[family_key] + include_custom = False + include_clone = True + model_id = args.model_id if args.model_id else ask( + f"Server model id for the {entry['label']} entry", + entry["default_id"]) + _offer_config_model_id_sync(model_id) + + host, port, backend, lazy_load = _ask_host_port_backend_lazy(args) + + if is_qwen: + custom_voice_path = base_path = None + if include_custom: + custom_voice_path = ask("Path to the Qwen3-TTS CustomVoice GGUF package", + DEFAULT_CUSTOM_VOICE_PATH) if include_clone: - wav_dir = args.input_dir - else: - print(f"[WARNING] Ignoring {args.input_dir}: no cloning (Base) " - "model selected, so voice presets are not used") - elif include_clone: - wav_dir = ask_wav_dir() - - voice_presets: Dict[str, dict] = {} - if wav_dir is not None: - wav_files = find_wav_files(wav_dir) - if wav_files: - if whisper_backend_available() is None: - print("[WARNING] Neither faster_whisper nor whisper was found, so " - "reference .wav files cannot be transcribed automatically and " - "every reference_text will be empty.") - print(' Did you remember to "conda activate qwen3-tts"? ' - "Transcripts must be added by hand (see the warning at the end).") - voice_presets = build_voice_presets(wav_files, args.whisper_model) - else: - print(f"[WARNING] No .wav files found in {wav_dir}; writing the " - "config without voice presets") - - server_config = build_server_config( - host=host, - port=port, - backend=backend, - lazy_load=lazy_load, - include_custom=include_custom, - include_clone=include_clone, - custom_voice_id=custom_voice_id, - clone_model_id=clone_model_id, - custom_voice_path=custom_voice_path, - base_path=base_path, - voice_presets=voice_presets, - ) + base_path = ask("Path to the Qwen3-TTS Base GGUF package", + DEFAULT_BASE_PATH) + else: + model_path = args.model_path if args.model_path else ask( + f"Path to the {entry['label']} package", entry["default_path"]) + + voice_presets = _collect_voice_presets(args, include_clone) + + if is_qwen: + server_config = build_server_config( + host=host, + port=port, + backend=backend, + lazy_load=lazy_load, + include_custom=include_custom, + include_clone=include_clone, + custom_voice_id=custom_voice_id, + clone_model_id=clone_model_id, + custom_voice_path=custom_voice_path, + base_path=base_path, + voice_presets=voice_presets, + ) + else: + server_config = build_single_family_server_config( + host=host, + port=port, + backend=backend, + lazy_load=lazy_load, + family=entry["family"], + model_id=model_id, + model_path=model_path, + voice_presets=voice_presets, + ) print("\nGenerated server.json:") print(json.dumps(server_config, indent=2, ensure_ascii=False)) @@ -466,9 +641,21 @@ def main() -> int: json.dump(server_config, handle, indent=2, ensure_ascii=False) handle.write("\n") - print(f"\n[OK] Wrote {args.output} with {len(server_config['models'])} " - f"model(s) and {len(voice_presets)} voice preset(s)") - _print_next_steps(args.output, include_custom, include_clone, voice_presets) + if is_qwen: + print(f"\n[OK] Wrote {args.output} with {len(server_config['models'])} " + f"model(s) and {len(voice_presets)} voice preset(s)") + else: + print(f"\n[OK] Wrote {args.output} hosting {entry['label']} " + f"(model id '{model_id}') with {len(voice_presets)} " + f"voice preset(s)") + print(f"[INFO] Install the model package from the audio.cpp checkout: " + f"python3 tools/model_manager_v2.py install {entry['install']}") + print("[INFO] Clone-only family: run audiobook.py with " + f"--backend audiocpp --voice <preset name>") + if not voice_presets: + print("[WARNING] No voice presets were configured; clone-only " + "families have no built-in speakers, so add voice_presets " + "(or a voice_dir) to server.json before converting") print_empty_transcript_warning(voice_presets) return 0 |
