iFrame

Low-latency Mac → iPad, Mac, Windows or Linux remote desktop for Apple silicon. It sends a video stream instead of pixel tiles.

Host requirements: a Mac with Apple silicon running macOS 14 or later (developed on an M4 Mac mini, macOS 15) and Xcode command-line tools.

Clients: an iPad on iPadOS 17 or later, a Mac on macOS 14 or later (Apple silicon or Intel), or a Windows 10/11 x64 PC. Building the iPad app requires Xcode and XcodeGen, plus rsvg-convert and ImageMagick if you regenerate icons (brew install xcodegen librsvg imagemagick). The Windows client uses Node.js to build and includes its runtime when packaged.

Mac (iframe-host)                                   iPad (iFrame app)
ScreenCaptureKit, IOSurface 4:2:0 ─┐               ┌─ VideoToolbox HW decode (sync)
VideoToolbox HEVC/H.264 encoder    │  TCP, no-delay│  AVSampleBufferVideoRenderer,
  low-latency rate control,        ├──────────────►│  display-immediately, fed off-main
  no B-frames, keyframes on demand │  WMM video AC │
ack-driven flow control,           │◄──────────────┤  acks, pings, mouse/keys/scroll
  adaptive bitrate, idle refine    ┘               └─ trackpad, touch, hardware keyboard
CGEvent input injection

What makes it fast:

• Local cursor. The Mac sends its cursor shape (arrow, I-beam, resize…) and leaves the pointer out of the video. Every client draws it at the local pointer position, so it moves with no network delay.
• Zero-copy pipeline. Capture hands IOSurfaces straight to the media engine. No color conversion and no CPU copies.
• Low-latency rate control. One frame in, one frame out, no reordering. Keyframes only on connect or after an error, so there are no periodic spikes.
• Ack-based flow control. At most 3 frames can be unacknowledged. When Wi-Fi backs up, frames are skipped before encoding instead of piling up in socket buffers, and the newest screen goes out as soon as the link frees up. Lag never builds up.
• Adaptive bitrate. It backs off quickly under congestion and recovers slowly.
• Idle refinement. When the screen stops changing, the last frame is re-encoded twice so text sharpens. A static screen then costs zero bandwidth.
• Static screens send nothing. ScreenCaptureKit only delivers frames that changed.
• Wi-Fi video priority. Traffic is tagged with the Wi-Fi video access category.
• Virtual display shaped like the iPad. The host creates a Retina display at the iPad's exact resolution and 120 Hz, and reshapes it when the iPad rotates. No letterboxing or scaling.

Host (Mac)

scripts/install-host.sh      # build, sign, install as a LaunchAgent (starts at login, restarts if it dies)
tail -f ~/Library/Logs/iframe-host.log
launchctl kickstart -k gui/$(id -u)/com.toddfaulls.iframe.host   # restart

It installs ~/Applications/iFrame Host.app (a background app with no Dock icon) and runs it in your GUI login session, so it works headless and no matter how you reach the Mac. The PIN is kept in ~/.config/iframe/pin.

Permissions (one time): System Settings → Privacy & Security → enable iFrame Host under Screen & System Audio Recording and Accessibility. They're tied to the app's signature, so they survive rebuilds. Over SSH, run security unlock-keychain before reinstalling, or signing fails.

Test without an iPad: .build/release/iframe-host probe 127.0.0.1 --pin <PIN> --screen 2732x2048@2

Client (iPad)

xcodegen generate   # creates iFrame.xcodeproj from project.yml
open iFrame.xcodeproj

Building it yourself? Change DEVELOPMENT_TEAM and the com.toddfaulls bundle IDs in project.yml (and LABEL in scripts/install-host.sh) to your own. Then run it on your iPad. A free Apple ID works, but the install expires after 7 days.

Your Mac appears under Mac on the connect screen. Enter the PIN, pick a display density, and tap Connect.

Client (Mac)

A native Universal app (Apple silicon + Intel, macOS 14+), target iFrameMac in the same Xcode project.

scripts/build-mac-client.sh   # → dist/iFrame.app and dist/iFrame-<version>-macOS-Universal.zip

It signs with your Apple Development / Developer ID identity if you have one (security unlock-keychain first over SSH), otherwise ad hoc; IFRAME_SIGN_IDENTITY=- forces ad hoc. The app isn't notarized, so on another Mac open it once with right-click → Open (or System Settings → Privacy & Security → Open Anyway).

• Display: the host makes a virtual display shaped like the window. Resize the window or go full screen and the remote desktop reshapes to fit, instead of being letterboxed. More space / Most space trade sharpness for room; Use the Mac's own display mirrors the host's screen instead.
• Input: mouse (all buttons), trackpad scrolling 1:1, and the keyboard pass straight through. Key codes are already Mac key codes, so every layout and shortcut works. While the stream has focus, every shortcut goes to the remote Mac (⌘Q, ⌘W, ⌘Space…) except ⌃⌥⌘ ones: ⌃⌥⌘F full screen, ⌃⌥⌘S stats, ⌃⌥⌘R refresh, ⌃⌥⌘D disconnect. macOS still keeps ⌘Tab and Mission Control for itself.
• Cursor: the remote Mac's pointer shape becomes this window's real cursor, so it moves with no lag.
• Don't run it on the host Mac itself against its own virtual display. The window would sit on the display it reshapes.
• Testing: open dist/iFrame.app --args -manualHost <addr> -pin <PIN> -autoConnect YES connects at launch.

Client (Linux)

Linux/ has a C client (SDL2 + FFmpeg, NVDEC/VAAPI hardware decoding) that speaks the same protocol. The Mac's virtual display takes the shape of the Linux window or monitor.

sudo apt install libsdl2-dev libavcodec-dev libavutil-dev avahi-utils
make -C Linux && make -C Linux install
iframe-client            # finds the Mac over Bonjour; --help for options

See Linux/README.md for scaling, key mapping and hotkeys.

Host (Linux)

LinuxHost/ is a separate host for Linux desktops (X11 + NVENC), in C. It's independent of the Mac host above: its own binary (iframe-linux-host), systemd user unit and PIN file. Only the wire protocol is shared, so the iPad app and the Linux client connect to it unchanged.

LinuxHost/install.sh     # build, install, enable iframe-linux-host.service; prints the PIN

See LinuxHost/README.md.

Controls (iPad)

Input	Action
Trackpad / mouse	Pointer, click, right click, two-finger scroll
Hardware keyboard	Full key passthrough including ⌘ shortcuts (except the ones iPadOS reserves, like ⌘Tab and ⌘Space)
Tap / two-finger tap	Click / right click
Drag	Move pointer
Long-press, then drag	Click-drag
Two-finger drag	Scroll
Three-finger tap	Toolbar (on-screen keyboard, stats, disconnect)

Client (Windows)

The Windows client connects to the same Mac host using H.264. It includes Mac discovery, PIN login, virtual-display settings, keyboard/mouse input, fullscreen, and live stats.

On Windows 10/11 x64 with Node.js 22.12 or newer:

cd Windows
npm ci
npm start
# Build a standalone portable Windows executable:
npm run dist

Open Windows/dist/iFrame-0.1.0-Windows-x64.exe, choose or enter your Mac's address, and enter the host PIN. The packaged app includes its runtime. See Windows controls, testing, and limitations.

Notes

• The virtual display uses CGVirtualDisplay, a private CoreGraphics API (the same one DeskPad, BetterDisplay and Chromium's tests use). A future macOS could change it; run the host with --no-virtual to stream the Mac's existing display instead.
• scripts/make-icon.py generates the icon; scripts/build-icons.sh renders every icon size and the app's accent color from it.

Known limits / next steps

• Video isn't encrypted; there's only a PIN. Use it on a trusted LAN or over Tailscale.
• Transport is TCP. That's fine on a LAN. UDP with FEC would handle lossy networks better.
• No audio and no clipboard sync yet.

License

MIT. See LICENSE.