OpenCut for Claude — AI Video Editor controlled by Claude Code via MCP

AI video editing · AI video editor · AI montage · automated video editing with neural networks · LLM video editor · Claude video editing · MCP video editor · agentic video editing · text-to-edit · AI-driven post-production.

Languages: English · Русский · Español · Français

A fork of OpenCut turned into a video editing tool for Claude Code. The editor itself is still there — a Next.js + Rust core web video editor — but on top of it sits an MCP server through which Claude directly moves the timeline, splits clips, adds captions, applies effects, generates graphics via Hyperframes, and exports the final cut.

The idea: instead of a human dragging clips around manually, the edit is described in natural language — Claude reads the transcript, the analysis, and selective frames, then performs the editing operations on a real OpenCut project.

Why this exists

Editing a short video is 80% mechanics (cutting on word boundaries, aligning to silence, syncing captions, color grading with presets) and 20% creative work. The mechanics can be handed to a model if it has:

1. A textual surface for the video — word-level whisper transcript with timestamps, silence/scene/loudness analysis. Claude reads the video instead of watching it frame by frame.
2. Direct access to the timeline — not "generate an ffmpeg command", but literally timeline_split(t), timeline_add_text(...), effects_add(...) against the live project.
3. Visual drill-down on demand — preview_frame(t) only at decision points where the transcript is ambiguous. Not a 30,000-frame scan.

This is the same pattern browser-use applies to a browser (DOM instead of screenshots) and video-use applies to video analysis — here it's brought inside the editor itself. The naive "let the model watch frames" approach is 30,000 frames × 1,500 tokens ≈ 45M tokens of noise. This pipeline: 12 KB of text plus a handful of PNGs at decision points.

How it works

┌─────────────────┐     MCP      ┌──────────────────┐     WebSocket     ┌────────────────┐
│  Claude Code    │ ───────────▶ │  opencut MCP     │ ────────────────▶ │  OpenCut UI    │
│  (editor agent) │              │  server          │  bridge :3002     │  (Next.js)     │
│                 │ ◀─────────── │  apps/web/mcp/   │ ◀──────────────── │  localhost:3001│
└─────────────────┘  tool calls  └──────────────────┘  state + frames    └────────────────┘

• MCP server (apps/web/mcp/server.ts) exposes ~50 tools: timeline_*, effects_*, preview_*, analyze_*, transcript_*, media_*, playback_*, project_*. Catalog under apps/web/mcp/tools/.
• Bridge (apps/web/server/pty-server.ts, port 3002) relays calls between the MCP server and the OpenCut browser tab over WebSocket, and also serves a PTY for the in-editor terminal.
• The editor (apps/web/) is the regular OpenCut UI plus a subscribed bridge client that listens for incoming commands and applies them to the timeline state.
• The terminal inside the editor opens at the repo root — same directory where OpenCut lives. Convenient for running the tools/ utilities while the model works.

Connection config is .mcp.json at the repo root. After bun dev:desktop, Claude Code picks up the server automatically.

What's added on top of OpenCut

MCP tools

Layer	Tools
Analysis (text-first)	transcript_get, transcript_search, analyze_silence, analyze_loudness, analyze_scene_changes
Timeline	timeline_get_elements, timeline_split, timeline_trim, timeline_delete, timeline_update, timeline_move_element, timeline_add_text, timeline_add_track, timeline_insert_element, timeline_snap_to_word, timeline_find_gap, timeline_get_duration
Effects	effects_list_available, effects_add, effects_remove, effects_update_params
Preview (on demand)	preview_frame(t), preview_thumbnail_strip, preview_waveform
Media / project	media_import, media_list, project_get_info, project_update_settings, selection_*, scene_*, history_undo/redo, export_info

Full spec, limits, and formats — docs/ai-montage/mcp-knowledge.md.

AI-montage toolkit (tools/ai-montage/)

CLI utilities the model runs through the in-editor terminal:

• whisper_word_level.sh — word-level transcription.
• pack_transcripts.py — packs transcripts + analysis into compact markdown for context.
• timeline_view.py — text-mode timeline visualization.
• grade.py, easing.py, palettes.json, subtitle-presets.json — color grading, easing, palette, and caption presets.
• hyperframes-animation-brief.md, pil-word-render-brief.md — brief templates for sub-agents that render animations.

Hyperframes (tools/hyperframes/)

HTML+GSAP → video. Used to generate titles, kinetic captions, overlays, and transitions. Invoked via npx --prefix tools/hyperframes hyperframes render — each clip is an HTML scene with a GSAP timeline, rendered to MP4 with alpha and dropped back onto the timeline as a separate element.

Agent pipeline

Inventory → Transcribe → Pack → EDL → Apply → Self-eval. See docs/ai-montage/pipeline.md. Hard production-correctness rules (never split inside a word, strategy confirmation before edits, parallel sub-agents for render-heavy tasks, captions as separate timeline elements) — docs/ai-montage/hard-rules.md.

For specific styles (TikTok-vertical, kinetic typography, SLIK Forged reel) — docs/ai-montage/reel-playbook.md.

Project structure

• apps/web/ — OpenCut Next.js app.
  • apps/web/mcp/ — MCP server and tool handlers.
  • apps/web/src/mcp/ — browser-side bridge client and command dispatcher.
  • apps/web/server/pty-server.ts — PTY + bridge relay on port 3002.
• apps/desktop/ — native desktop wrapper (Tauri / GPUI, in progress).
• rust/ — platform-agnostic core: GPU compositor, effects, masks, WASM bindings.
• tools/ai-montage/ — CLI utilities for the pipeline.
• tools/hyperframes/ — HTML+GSAP→video renderer.
• docs/ai-montage/ — agent guides (pipeline, hard rules, MCP knowledge, reel playbook).
• .mcp.json — Claude Code MCP server config.
• CLAUDE.md — instructions Claude reads when entering the project.

Getting started

Prerequisites

• Bun
• Claude Code — without it the MCP layer is pointless.
• Docker and Docker Compose — optional, for local DB and Redis.
• ffmpeg / ffprobe — fallback for audio extraction and codec conversion.

Setup

1. Fork and clone the repository.

2. Copy the env file:

   cp apps/web/.env.example apps/web/.env.local

3. (Optional) Bring up DB and Redis:

   docker compose up -d db redis serverless-redis-http

4. Install dependencies and start the dev stack:

   bun install
   bun dev:desktop   # web (3001) + PTY/bridge (3002) + Tauri window

   Or individually:

   bun dev:web      # Next.js on 3001
   bun dev:pty      # PTY + Bridge on 3002

5. Open the project in Claude Code — it will pick up .mcp.json and connect the opencut server. Verify: tools prefixed mcp__opencut__* should appear in Claude Code.

6. Open http://localhost:3001, create a project, import a video — and ask Claude to edit something.

First prompt

Import /path/to/video.mp4, read the transcript, cut pauses longer than 600 ms,
and add kinetic word-level captions. Show me the strategy before cutting.

Claude will read CLAUDE.md, ask about the mode (text-only vs visual), pull the transcript via transcript_get, apply analyze_silence, agree on the plan — and only then start calling timeline_split.

Desktop / WASM / self-hosting

These sections inherit from upstream OpenCut and work as before — see apps/desktop/README.md and rust/wasm/. For self-hosted production:

docker compose up -d

The app will be available at http://localhost:3100. The MCP layer isn't needed in production — it's for local work with Claude Code.

What's unchanged from upstream

The editor itself, the render pipeline, project format, effects, export — all of that stays upstream OpenCut. This fork adds a control layer (MCP + bridge + ai-montage toolkit) on top without touching the core. I merge upstream regularly.

License

MIT — same as upstream.

Credits

• OpenCut — the base editor.
• browser-use and video-use — the text-first surface idea for agents.
• Hyperframes — HTML→video renderer.
• Anthropic Claude — the editor's brain.

Keywords

AI video editor, AI video editing, AI montage, automated video editing, video editing with AI, video editing with neural networks, LLM video editor, Claude Code video editor, Claude video editing, agentic video editing, MCP video editor, MCP server for video editing, AI-driven post-production, AI auto-cut, AI subtitle generator, kinetic typography AI, text-to-edit, prompt-driven video editor, OpenCut fork, AI CapCut alternative, AI Premiere alternative, AI DaVinci Resolve alternative, AI short-form video, AI Reels editor, AI TikTok editor, AI YouTube Shorts editor, AI podcast editor, whisper video editor, transcript-based video editor, EDL generator, generative video editing, autonomous video editing agent.