[wassgha]

Why

Exports still felt much slower than CapCut. I benchmarked the bundled ffmpeg on an M1 Pro to find where the time goes.

• 1080p H.264 sources: the existing pipeline is already close to the hardware's limit for a full re-encode.
• iPhone-style 4K HEVC 10-bit sources: this is where it's slow.
  • Software HEVC decoding alone takes 19.1 s per 30 s of 4K, so the export can't beat about realtime.
  • Rendering 4K when the export ends up on a 1080p screen costs about 3.7× more on top of that.

What

1. Hardware decode on macOS for HEVC, ProRes and AV1 sources (-hwaccel videotoolbox). This is decided by codec, from measurements:

   20–30 s clip	software	hardware
   4K HEVC 10-bit	19.1 s	3.2 s
   1080p HEVC	3.9 s	1.1 s
   4K ProRes	4.2 s	3.2 s
   4K H.264	3.9 s	7.0 s

   H.264 therefore stays in software. Decoded frames come back to system memory, so the CPU filtergraph is unchanged. VideoToolbox quietly falls back to software for codecs it can't decode (verified with VP9).

2. Hardware H.264 encode on macOS for MP4 (h264_videotoolbox -q:v 65).

   • Quality: -q:v 65 matched x264 -crf 22 on a 30 s 1080p sample: 31 MB vs 32 MB, SSIM 0.9818 vs 0.9823.
   • Speed: it matches x264 on an 8-core M1 Pro, and is 1.5× (1080p) to 2.2× (4K) faster when x264 gets 4 cores, which is closer to most users' Macs.
   • Availability check: a tiny test encode runs once per session. Intel Macs have no constant-quality VideoToolbox, so they keep libx264.
   • Fallback: if any hardware attempt fails, the same job re-runs in software into the same output (runAttempts).
     • The progress bar restarts from 0, and the user still gets their file.
     • The failure is reported as stage=native-hw-fallback, with the stderr tail.
     • Cancellation and a full disk are not retried.
   • Off switch: RESCRIPT_MEDIA_HW=0 turns hardware off. Windows and Linux stay on software until hardware encoding is checked on real GPUs.

3. Default export resolution is 1080p when the source is larger, and the source size otherwise. A user's explicit pick still sticks.

   • Short-side cap: resolutions now cap the frame's short side, so "1080p" of a vertical 2160×3840 video is 1080×1920. Before, it would have been 608×1080, which a 1080p default would have hit on every vertical video.
   • Scale expression: the new scale filter is plain arithmetic. I checked it gives the same results on ffmpeg 5.1.4 (the wasm core, run in Node), 6.0 (arm64) and 6.1.1 (x64).
   • Web build: it gets the new default too, which also lowers wasm memory use for 4K sources.

Results (in the app, 30 s 4K HEVC 10-bit clip with cuts, M1 Pro)

	before (software)	after (VideoToolbox)
1080p	21.0 s	6.2 s
original 4K	40.2 s	15.5 s
old default → new default	40.2 s (original)	6.2 s (1080p)

The 1080p H.264 source → MP4 job uses hardware encode only, because software decode is faster there.

Testing

• tsc (web + electron), eslint, next build, build:electron, test:i18n, test:timeline and test:ffmpeg all pass.
• export-args-test:
  • VideoToolbox changes only the encoder args; filtergraph, maps and audio are unchanged.
  • WebM stays on libvpx.
  • Short-side scale expressions.
  • defaultExportResolution cases: landscape, portrait, 1440p, 1080p, 720p, unknown.
• native-ffmpeg-test:
  • parseVideoStream on real input-dump lines: iPhone HDR colour info with commas, ProRes, VP9, cover art skipped, audio-only.
  • The hardware-decode rule.
  • runAttempts: a broken first attempt falls back and the output is the full edit; a cancelled run isn't retried.
  • A real vertical 1080×1920 → 720p render comes out 720×1280.
  • On Apple Silicon: a real VideoToolbox decode + encode of an HEVC 10-bit source produces 8-bit H.264 with the right duration and progress.
• In Electron (electron:dev with a throwaway --user-data-dir), driven over CDP:
  • The main process picked hardware decode + encode for HEVC→MP4, hardware decode only for HEVC→WebM, and hardware encode only for H.264→MP4, with no fallbacks.
  • A software A/B run with RESCRIPT_MEDIA_HW=0 gave the numbers above.
  • With a 4K source, the export dialog opens at 1080p, and the render took 7.0 s through the UI.

Not verified

• Intel Mac. The probe should send it to libx264; I couldn't run this, since the x64 binary under Rosetta still uses Apple Silicon hardware.
• The fallback firing in the shipped app. It's covered by runAttempts tests, but I couldn't make VideoToolbox fail on demand.

Follow-ups

• Windows NVENC / QSV / AMF encoding: the bundled gyan.dev build has these encoders. They need measuring on real GPUs (CI runners have none).
• Smart rendering (copy untouched GOPs, re-encode only around cuts) for lightly edited original-resolution exports.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
app.rescript	Ready	Preview	Oct 3, 2026 6:13pm UTC