Storykeeper

Storykeeper is a local multi-voice audiobook workbench. It turns prose or script text into editable narration/dialogue segments, assigns voices to characters, generates per-segment audio, and merges finished segments into an MP3.

The web app is designed to run locally or on a trusted private host. It has no account system and stores one workspace under DATA_DIR.

Features

• Split stories into narrator and character dialogue segments.
• Import JSON produced by an external LLM prompt.
• Edit every segment's text, speaker, voice, emotion instruction, and speed.
• Upload custom reference voices.
• Optionally transcribe reference audio through an ASR service.
• Generate audio through a TTS service with progress updates over SSE.
• Preview individual segments and merge completed segments into one MP3.

Repository Boundaries

This repository should contain source code, tests, lightweight static assets, and placeholder voice metadata only.

Do not publish:

• Runtime workspace data from DATA_DIR.
• Generated audio, merged MP3 files, logs, cache folders, or temporary prompts.
• TTS/ASR/LLM model weights.
• Private reference voices or licensed audio.
• Local .env files, API keys, secrets, or machine-specific paths.

There is currently no LICENSE file. Add one before describing this as an open-source project under a specific license.

Quick Start

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.web.txt
uvicorn app.main:app --host 127.0.0.1 --port 7861

On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.web.txt
uvicorn app.main:app --host 127.0.0.1 --port 7861

Open the web app at http://127.0.0.1:7861.

Configuration

Copy .env.example to .env and edit it for your machine. Do not commit .env.

DATA_DIR=data

TTS_URL=http://localhost:9999

ASR_URL=http://localhost:9881
ASR_SECRET=change-me

LLM_URL=http://localhost:2345/v1
LLM_API_KEY=
LLM_MODEL=openai/gpt-oss-20b

RATE_LIMIT_ENABLED=false

Variable	Purpose
DATA_DIR	Runtime workspace storage. Generated files live here.
TTS_URL	Base URL for the text-to-speech service.
ASR_URL	Base URL for the optional speech-to-text service.
ASR_SECRET	Shared secret sent to ASR as X-Shared-Secret.
LLM_URL	OpenAI-compatible API base URL, usually ending in /v1.
LLM_API_KEY	Optional bearer token for hosted LLM providers.
LLM_MODEL	Model id sent in chat completion requests.
RATE_LIMIT_ENABLED	Enables request limits when set to true.

External Interfaces

Storykeeper is a web app that delegates model work to external services. Those services can run locally or remotely as long as they implement the contracts below.

LLM Service

The LLM service must expose an OpenAI-compatible Chat Completions API.

Required endpoint:

POST {LLM_URL}/chat/completions
Authorization: Bearer {LLM_API_KEY}   # only when LLM_API_KEY is set
Content-Type: application/json

Request body:

{
  "model": "openai/gpt-oss-20b",
  "messages": [{"role": "user", "content": "prompt text"}],
  "temperature": 0.2,
  "max_tokens": 8192,
  "reasoning_effort": "low"
}

Expected response shape:

{
  "choices": [
    {
      "message": {
        "content": "{\"characters\":[],\"segments\":[]}"
      }
    }
  ]
}

Optional health/model endpoint:

GET {LLM_URL}/models

The returned model names are not interpreted by Storykeeper except for the id you configure in LLM_MODEL.

TTS Service

The TTS service receives one segment at a time and returns WAV bytes. Storykeeper does not send an authentication header to TTS, so expose this service only on a trusted local or private network.

Required endpoint:

POST {TTS_URL}/tts
Content-Type: multipart/form-data

Form fields:

Field	Required	Description
text	yes	Segment text to synthesize.
voice_id	yes	Storykeeper sends custom when passing a reference voice file.
instruct	yes	Emotion/style instruction. May be empty.
lang	yes	zh or en, inferred from segment text.
speed	yes	Playback speed, such as 1.0.
ref_audio	when voice_id=custom	Reference WAV file.
ref_text	when voice_id=custom	Transcript of the reference audio.

Successful response:

200 OK
Content-Type: audio/wav

Optional endpoints used by helper scripts or diagnostics:

GET  {TTS_URL}/health
GET  {TTS_URL}/voices
POST {TTS_URL}/log

POST /log receives JSON event metadata and should return 204 No Content.

ASR Service

ASR is only needed when using the custom voice transcription helper.

Required endpoint:

POST {ASR_URL}/asr
X-Shared-Secret: {ASR_SECRET}
Content-Type: multipart/form-data

Form fields:

Field	Required	Description
audio	yes	Uploaded audio file bytes.

Expected response:

{
  "text": "transcribed reference text",
  "language": "en"
}

Optional health endpoint:

GET {ASR_URL}/health

Web Routes

Most app routes return HTML fragments for htmx rather than stable public API responses.

Route	Method	Purpose
/ and /app	GET	Main workbench.
/story/analyze	POST	Split/import story segments.
/voices	POST	Upload a custom voice.
/voices/transcribe	POST	Transcribe uploaded voice audio through ASR.
/voices/{voice_id}/preview	GET	Stream a voice preview WAV.
/segments	GET	Render the segment editor.
/segments/{idx}	POST	Update one segment.
/segments/{idx}/split	POST	Split one segment.
/segments/{idx}	DELETE	Delete one segment.
/generate	POST	Queue audio generation.
/events	GET	Server-sent generation events.
/segments-status	GET	Current generation status snapshot.
/merge	POST	Merge completed segment WAVs into MP3.
/audio/{subpath}	GET	Stream generated WAV/MP3 files from the workspace.

Voices

Preset voice metadata lives in voices/presets.json. Each preset points to a reference audio file and optional preview assets. Keep only assets you have the right to redistribute.

Custom voices uploaded in the app are stored under DATA_DIR and should not be committed.

Helper Servers

The included tts_server.py, tts_server_v3.py, and asr_server.py are reference adapters for local model runtimes. They require compatible model files that are not included in this repository.

start.sh can start the helper services and the web app in a Git Bash style environment. Configure interpreter paths with environment variables instead of editing the script:

TTS_PYTHON=/path/to/python ASR_PYTHON=/path/to/python bash start.sh

Dockerizing CosyVoice3 TTS

CosyVoice3 can be run as a separate GPU container with Dockerfile.tts-cv3. The image contains the HTTP adapter and Python runtime, but not model weights or private voice references.

Expected mounted files:

pretrained_models/
  Fun-CosyVoice3-0.5B/
voices/
  presets.json
  generated/*.wav
  generated_clean/*.wav

Build and start:

docker compose -f docker-compose.tts-cv3.yml up --build

Or with plain Docker:

docker build -f Dockerfile.tts-cv3 -t storykeeper-tts-cv3:local .
docker run --rm --gpus all -p 9999:9999 \
  -v ./pretrained_models:/models:ro \
  -v ./voices:/app/voices:ro \
  storykeeper-tts-cv3:local

Then point Storykeeper at:

TTS_URL=http://localhost:9999

The TTS container has no request-level shared secret. Keep it bound to localhost or a trusted private network.

Tests

pytest

UI tests live under tests/ui and require Playwright:

pip install pytest-playwright
playwright install chromium
pytest tests/ui -v