⚠️ Notice: This project is largely AI-generated and is under active development. Expect rough edges.
Canto is a self-hosted AI audiobook studio. Describe the story you want, and Canto writes an original script with an LLM, designs a distinct voice for every character, and synthesizes a full multi-character audiobook with emotional speech, powered by Qwen3-TTS and IndexTTS2. Stories can be written in English or Chinese.
- AI-Original Scripts: Pick a genre, subgenre, protagonist, tone and core conflict. The LLM drafts a synopsis, then writes the script chapter by chapter, and you can keep extending it.
- English & Chinese Stories: Choose the story language per project. Prompts, script format, emotion tags, voice design and synthesis all follow it.
- Multi-character Audiobooks: Characters are extracted automatically and each one gets its own voice. Chapters are split into lines and synthesized in batches, with a live progress log. Export the whole book as a single WAV.
- NSFW Mode: A dedicated adult script generator with independent sliders for violence and eroticism and adult genres, backed by a separately configured Grok (or any OpenAI-compatible) model. Access is granted per user by an administrator and gated behind an 18+ disclaimer.
- Emotional Synthesis: The script tags every line with weighted, blendable emotions (e.g. happy 0.6 + surprised 0.3), so IndexTTS2 delivers a performance instead of flat narration.
- Voice Design: Each character's voice is designed from a natural-language description with Qwen3-TTS VoiceDesign, then cloned for consistent synthesis. Edit the description and regenerate the preview until it sounds right.
- Fine-grained Editing: Edit any line's text or emotion and regenerate just that segment.
- Multi-user: Username and password login. Admins manage users, NSFW permissions and usage statistics.
- Multi-language UI: English, 简体中文, 繁體中文, 日本語, 한국어, with light and dark themes.
| Minimum | Recommended | |
|---|---|---|
| OS | Linux x86_64 | Ubuntu 22.04+ / Debian 12+ |
| GPU | NVIDIA, 12 GB VRAM | NVIDIA, 16 GB VRAM |
| NVIDIA driver | 550 | latest stable |
| RAM | 16 GB | 32 GB |
| Disk | 35 GB free | 50 GB free |
| Software | Python 3.12, Node.js 20.19+ (build only), git, sox, libsndfile, ffmpeg |
Disk covers about 15 GB of models, a 9 GB Python environment and room for generated audio. macOS, Windows and non-NVIDIA GPUs are not supported.
You also need an OpenAI-compatible LLM API to write scripts (OpenAI, DeepSeek, OpenRouter, a local vLLM / Ollama server, ...).
The easiest way to install Canto is to let a coding agent (Claude Code, Codex, Cursor, ...) do it on your server. Open the agent on the target machine and send:
Deploy Canto on this machine by following https://github.com/bdim404/Canto/blob/main/DEPLOY.md step by step. Run every check, and ask me before installing system packages or whenever the guide says to ask.
The agent checks your hardware, installs dependencies, builds the frontend, downloads the models, starts Canto and optionally sets it up as a systemd service.
DEPLOY.md is a complete step-by-step guide. In short:
git clone https://github.com/bdim404/Canto.git
cd Canto
cd frontend && npm ci && npm run build && cd ..
cd backend
uv venv --python 3.12 .venv
uv pip install --python .venv torch==2.10.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install --python .venv -r requirements.txt
cp .env.example .env
.venv/bin/uvicorn main:app --host 0.0.0.0 --port 8000Open http://localhost:8000 and sign in as admin. If you didn't set ADMIN_PASSWORD in backend/.env, a random password is printed once in the server log on first start.
Then, as admin, open Settings and fill in LLM Config (base URL, model, API key). To use NSFW mode, also fill in Grok Config and grant NSFW access to users in User Management (admins always have it).
Models are downloaded from Hugging Face automatically the first time they're needed, so the first start and the first generation take a while. In mainland China, set HF_ENDPOINT=https://hf-mirror.com in backend/.env.
| Model | Size | Used for |
|---|---|---|
| IndexTTS-2 | ~4.4 GB | Audiobook synthesis with emotion |
| Qwen3-TTS-12Hz-1.7B-VoiceDesign | ~4.3 GB | Voice design |
| Qwen3-TTS-12Hz-1.7B-Base | ~4.3 GB | Cloning designed voices |
| w2v-bert-2.0, MaskGCT, CAMPPlus, BigVGAN | ~2.8 GB | IndexTTS2 dependencies |
See DEPLOY.md to download them in advance.
All settings live in backend/.env; backend/.env.example lists every option.
| Variable | Default | Description |
|---|---|---|
SECRET_KEY |
auto | Signs login tokens and encrypts stored API keys. Generated and saved to backend/secret.key if empty. Don't lose it, or saved LLM keys can't be decrypted. |
ADMIN_USERNAME / ADMIN_EMAIL / ADMIN_PASSWORD |
admin / admin@example.com / random |
First admin account, created only when the database is empty |
ACCESS_TOKEN_EXPIRE_MINUTES |
10080 |
Login session lifetime |
DATABASE_URL |
sqlite:///./canto.db |
Database location |
OUTPUT_DIR |
./outputs |
Generated audio |
MODEL_BASE_PATH |
./models |
Where models are stored |
MODEL_DEVICE |
cuda:0 |
Torch device for Qwen3-TTS |
MODEL_IDLE_TIMEOUT_MINUTES |
30 |
Unload models from VRAM after this much idle time |
AUDIOBOOK_PARSE_CONCURRENCY / AUDIOBOOK_GENERATE_CONCURRENCY |
3 / 2 |
Parallel LLM parsing and TTS generation |
LLM_TIMEOUT_SECONDS |
300 |
How long to wait for the LLM (and Grok) API, including the gap between streamed tokens. Raise it for slow or local models |
HF_ENDPOINT / HF_HUB_OFFLINE |
Hugging Face / unset | Model download mirror and offline switch |
./dev.shStarts the backend with auto-reload and DEV_MODE=true (authentication skipped, for local use only) on port 8000, and the Vite dev server on http://localhost:5173, which proxies /api to the backend.
The marketing site in landing/ is a separate Vite app: cd landing && npm ci && npm run dev.
Backend: FastAPI, SQLAlchemy (SQLite), PyTorch, Qwen3-TTS, IndexTTS2, APScheduler
Frontend: React 19, TypeScript, Vite, Tailwind CSS, shadcn/ui, i18next
NSFW mode generates fictional content with adult themes. It is disabled for regular users until an administrator grants access, and every user must confirm they are at least 18 and accept the disclaimer before using it. As the operator of your instance, you are responsible for complying with the laws of your jurisdiction and for the content you generate. Never use Canto to generate content involving minors or for any unlawful purpose.
- Qwen3-TTS by the Qwen Team, Alibaba Cloud.
- IndexTTS2 by bilibili.
backend/indextts/is derived from it.
Canto is released under the Apache-2.0 License.
backend/indextts/ is derived from IndexTTS2 and is subject to the bilibili Model Use License Agreement (中文). Read it and the DISCLAIMER before use. Any modifications made to the original model in this Derivative Work are not endorsed, warranted, or guaranteed by the original right-holder of the original model, and the original right-holder disclaims all liability related to this Derivative Work.