techomancer · Oct 29, 2026 · Oct 29, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -13,6 +13,44 @@ hashes are given where a change is easiest to understand by reading the commit.
 
 ## October 2026
 
+### VNC, and screen and pointer commands
+
+- **Built-in VNC server.** `--vnc [ADDR]` or `[vnc] listen` serves the
+  guest's screen, keyboard and mouse to any VNC viewer (RFB 3.8, Raw and
+  ZRLE, desktop resizing, QEMU's key-position extension; no password, so it
+  stays on loopback by default). It serves every machine and board the same
+  way, with the window open or under `--ci` with no window. It reads the
+  frame the board composed and finds what changed by comparing frames in
+  64x64 tiles; drawing code is untouched. Standalone `iris` only. See
+  [HELP.md](HELP.md#vnc). (`src/vnc/`)
+- **Clicks land where you point.** The guest mouse is relative and X
+  accelerates it; iris now steers the guest pointer onto an absolute
+  position by reading back the board's hardware cursor (VC2, VC1, VC3, GBE),
+  learning the cursor's hotspot from the screen corner. VNC uses it, and so
+  do new monitor commands for scripts and assistants: `screen
+  info|shot|pix|wait|cursor`, `ptr move|click|drag|down|up|rel|calibrate`,
+  `key <chord>|type|names`, on every machine
+  ([debug.md](debug.md#graphics)). The MCP server wraps them as
+  `screenshot` (returns the image), `click`, `drag`, `move_pointer`,
+  `type_text`, `press_key`, `wait_screen`, `screen_info`, `screen_pixel`.
+  (`src/hostinput.rs`, `src/keysym.rs`, `clanker/iris_mcp.py`)
+- `GfxDisplay` gained `request_frame`, `set_capture`, `hw_cursor` and a
+  frame sequence number, so a consumer can follow the composed frame on any
+  board. The CI socket's `screenshot` uses it and no longer also writes a
+  `screenshot_NNNN.png` into the working directory on Newport. While a VNC
+  viewer is connected, Newport's window composes in software, and goes back
+  to GL when the last one leaves. (`src/gfx_display.rs`,
+  `src/dev/ng1/rex3.rs`, `src/ci.rs`)
+- **Empty GIO slots answer reads with a bus error** (Indy, Indigo2), as the
+  MC does once the PROM has enabled GIO time-outs. They returned all ones,
+  so the PROM's probe took an absent board for a Newport and polled its
+  status 200 000 times on every headless boot (one log line each) before
+  giving up. Headless Indy and Indigo2 now reach the IRIX serial login
+  without the flood. (`src/ip22_bus.rs`, `src/dev/mc.rs`)
+- **The O2's cursor no longer vanishes at the top and left edges.** GBE's
+  cursor position fields are 12-bit signed; they were read as unsigned, so a
+  cursor partly off the edge was drawn 4000 pixels away. (`src/dev/o2gfx.rs`)
+
 ### Host efficiency on small machines
 
 - **Threads that have nothing to do now sleep in milliseconds, not

diff --git a/Cargo.toml b/Cargo.toml
@@ -284,6 +284,8 @@ toml = "1.0.3"
 postcard = { version = "1", features = ["alloc"] }
 blake3 = "1"
 png = "0.18"
+# The VNC server's ZRLE encoding (zlib).
+flate2 = "1"
 parking_lot = "0.12"
 spin = "0.9"
 # rfd >= 0.16 dropped the async-std/tokio runtime features: `xdg-portal` now

diff --git a/FEATURES.md b/FEATURES.md
@@ -238,6 +238,13 @@ Omit `[nfs]` to disable the share; the default forward list is empty.
 | `vino.standard` | `ntsc` | ntsc (60 fields/s) or pal (50 fields/s). |
 | `vino.camera_index` | `0` | Host camera index; meaningful only for source=camera. |
 
+### [vnc]
+
+| Key | Default | Values and behavior |
+|---|---|---|
+| `vnc.listen` | `unset` (no server) | `host:port`, or a port on 127.0.0.1, to serve the screen, keyboard and mouse over VNC (RFB 3.8; Raw and ZRLE; no password). Every board, with a window or `ci`; not `headless`. Standalone `iris` only. See [HELP.md](HELP.md#vnc). |
+| `vnc.max_fps` | `30` | Most updates a second sent to a viewer (1–120). |
+
 ### [audio]
 
 | Key | Default | Values and behavior |
@@ -399,6 +406,7 @@ iris-gui does not parse this standalone option set; use its named-machine UI.
 | `--graphics` | Graphics board: `newport` (default), `xz`, `extreme`, `solidimpact`, `highimpact`, `maximpact` (or `impact:solid`, `impact:high`, `impact:max`). |
 | `--gdb-port` | Enable GDB stub on the given TCP port (e.g. --gdb-port 1234). Connect with: target remote localhost:<port> |
 | `--monitor-port` | Monitor console port on 127.0.0.1 (default 8888). Give each iris running at once its own. |
+| `--vnc [ADDR]` | Serve the screen, keyboard and mouse over VNC on ADDR (`host:port` or a port; default 127.0.0.1:5900). Sets `vnc.listen`. |
 | `--serial-port-a` | Serial channel A (tty2) port on 127.0.0.1 (default 8880). |
 | `--serial-port-b` | Serial channel B (tty1, the console) port on 127.0.0.1 (default 8881). |
 | `--test-device` | Map the bare-metal test device into GIO expansion slot 0: SIGNATURE, PUTC (guest console → stdout), DUMP (machine state → JSON) and EXIT (terminate with the guest's exit code). Off by default. |

diff --git a/HACKING.md b/HACKING.md
@@ -253,7 +253,8 @@ Every device that has real work to do runs on its own thread:
 | graphics: `REX3-Processor` / `GR2-HQ2`, `GR2-RE3` / `MGRAS-HQ3`, `MGRAS RSS` ×1-4 | consumers of the graphics FIFOs (section 7) |
 | `REX3-Refresh`, `GR2-Display`, MGRAS refresh | scanout and composition; the refresh thread owns the GL context |
 | `hptimer` | host timer callbacks (Compare, PIT ticks, HAL2 audio pacing) |
-| listeners | debug console, serial ports, CI socket, GDB stub |
+| listeners | debug console, serial ports, CI socket, GDB stub, VNC |
+| `VNC-Grab`, `VNC-Pointer`, `VNC-Client`/`VNC-Read` | with `--vnc`: frame diffing, pointer steering, one pair per viewer |
 | `machine-events` | power-off and reset requests from devices |
 | `ISP1040-0`, `ISP1040-1` | Octane SCSI request rings and disk transfers |
 | `IOC3-RX-A`, `IOC3-RX-B`, `IOC3-NVRAM` | Octane serial input per port; the DS1685 NVRAM file writer (IOC3's other deadlines are `hptimer` one-shots) |
@@ -418,6 +419,31 @@ without locks and composes it through the board's colour maps and DAC.
 Everything that touches the host GL context (including teardown) runs on the
 thread that owns it.
 
+### The composed frame and the hardware cursor
+
+`GfxDisplay` (`src/gfx_display.rs`) is what the window, the CI socket, the
+monitor's `screen`/`ptr` commands and the VNC server see of a board, whichever
+board it is. The display thread leaves the composed guest frame in
+`screen().rgba` (stride 2048, `0xAABBGGRR`, no status bar) and bumps
+`screen().frame_seq` each time it rewrites it. GR2, IMPACT and the O2 compose
+on the CPU and do this every frame. Newport composes inside the window's
+renderer, so it fills `rgba` only when asked: `request_frame` for one frame,
+`set_capture(true)` for every frame while a consumer (VNC) watches. While
+capture is on, Newport's window uses the software compositor so the copy is a
+slice copy; with no window (`--ci`) the refresh thread composes in software
+itself. `copy_frame`/`fresh_frame` are the helpers that read it.
+
+Nothing downstream looks inside the drawing code. The VNC server
+(`src/vnc/`) finds damage by comparing frames in 64x64 tiles, so every board
+is served the same way and drawing paths carry no bookkeeping.
+
+`hw_cursor` reads the board's cursor registers live (VC2, VC1, VC3, GBE):
+the glyph's top-left and its opaque pixels. `src/hostinput.rs` uses it to
+steer the guest's relative PS/2 mouse onto absolute positions (the X server
+accelerates what it reads, so positions cannot be sent), learning each
+glyph's hotspot from the top-left screen corner, where X pins the pointer
+at 0,0.
+
 ## 8. jitv2: the MIPS JIT
 
 `--features jitv2` adds a JIT compiler (`src/cpu/jitv2/`) that turns guest

diff --git a/HELP.md b/HELP.md
@@ -494,6 +494,55 @@ standalone `iris` window if you need them.
 
 The window is freely resizable. At 1× and 2× scale the display is rendered with nearest-neighbour pixel-perfect sampling; at all other sizes trilinear filtering with mipmaps is used. `lock_aspect_ratio = false` in `iris.toml` allows free resizing with letterboxing. When the guest changes video mode (PROM, then IRIX during boot) the window snaps to the new resolution times the configured `scale` / `--2x`.
 
+## VNC
+
+iris has a built-in VNC server. Any VNC viewer (TigerVNC, RealVNC, Remmina,
+noVNC, macOS Screen Sharing) then shows the guest's screen and types and
+clicks into it:
+
+```sh
+iris --vnc                    # 127.0.0.1:5900
+iris --ci --vnc               # no window at all: VNC is the screen
+iris --vnc 0.0.0.0:5901       # reachable from other machines (no password!)
+vncviewer localhost:5900
+```
+
+or in `iris.toml`:
+
+```toml
+[vnc]
+listen  = "127.0.0.1:5900"   # absent: no VNC server
+max_fps = 30                 # most updates a second per viewer
+```
+
+It works on every machine and graphics board (Newport, XZ/Extreme, IMPACT,
+the O2), with the normal window open or under `--ci` with no window. Not with
+`--headless`, which has no screen to serve. The window, the monitor and any
+number of viewers can type and click at the same time. iris-gui has no VNC
+server.
+
+What you see is the guest's screen, without the window's status bar. Only
+the parts of the screen that change are sent, so an idle desktop costs next
+to nothing. With Newport, the window uses the software compositor while a
+viewer is connected (`disp compositor gl` is refused) and switches back when
+the last one leaves.
+
+**The mouse.** The guest's mouse sends movements, not positions, and IRIX
+accelerates fast ones. iris steers the guest pointer onto the spot you point
+at by watching where the board draws the cursor, so a click lands where you
+clicked. The first time you point, the guest pointer jumps to the top-left
+corner and back: that teaches iris where the tip of the cursor is. The
+viewer's own cursor (a dot or an arrow) runs a little ahead of the guest's;
+it catches up within a fraction of a second.
+
+**The keyboard.** Viewers that send key positions (TigerVNC, noVNC, QEMU's
+extension) work with any host layout. Others send characters, which iris
+types on a US layout.
+
+**Security.** There is no password. Leave the server on `127.0.0.1` and use
+an SSH tunnel (`ssh -L 5900:localhost:5900 host`) to reach it from
+elsewhere.
+
 ---
 
 ## Configuration file (`iris.toml`)
@@ -716,6 +765,7 @@ Automation and debugging
   --serial-log <FILE>         Append everything IRIX prints on ttyd1 to FILE
   --serial-log-timestamps <MODE>  off (raw) or wall: timestamp each completed line
   --monitor-port <PORT>       Monitor TCP port (default 8888)
+  --vnc [ADDR]                VNC server on ADDR (host:port or port; default 127.0.0.1:5900)
   --serial-port-a <PORT>      Serial channel A TCP port (default 8880)
   --serial-port-b <PORT>      Serial channel B TCP port (default 8881)
   --gdb-port <PORT>           Start the GDB stub

diff --git a/clanker/CLANKER.md b/clanker/CLANKER.md
@@ -60,6 +60,7 @@ found or what to watch out for while changing code, it goes in `clanker/`.
 | `clanker/INDEX.md` | categorized index of every note |
 | `clanker/cpu/`, `jitv2/` | CPU architecture findings; the MIPS JIT |
 | `clanker/devices/`, `scsi/`, `net/`, `video/` | device models, storage, the NAT stack and its services, VINO/IndyCam |
+| `clanker/vnc/` | the VNC server, the host input layer (`screen`/`ptr`/`key`) and pointer steering |
 | `clanker/rex3/`, `gr2/`, `mgras/`, `odyssey/`, `hostgl/` | graphics boards and host GL (`odyssey/` is references only) |
 | `clanker/gui/`, `macos/` | front-ends and platform packaging |
 | `clanker/snapshot/`, `perf/`, `testing/`, `build/` | snapshots and `iris-ci`, host performance, test harnesses, builds |

diff --git a/clanker/INDEX.md b/clanker/INDEX.md
@@ -98,6 +98,7 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS
 
 | Note | Kind | Title |
 |---|---|---|
+| [empty-gio-slot-reads-bus-error.md](devices/empty-gio-slot-reads-bus-error.md) | rule | Reads of an empty GIO slot end in a bus error (ABORT_EN); all ones made the PROM poll a phantom Newport 200 000 times headless |
 | [cpal-alsa-busy-poll.md](devices/cpal-alsa-busy-poll.md) | rule | cpal's ALSA backend can spin a core: `[audio] host = "pulseaudio"` |
 | [decode-table-is-host-mapped.md](devices/decode-table-is-host-mapped.md) | rule | The physical decode table is a sparse host mapping over 40 bits: change it only through `fill`, set the default first, compare entries with `entry_eq` |
 | [mc-decodes-32-physical-bits.md](devices/mc-decodes-32-physical-bits.md) | rule | Indy/Indigo2/O2 decode 32 physical bits; IRIX PTE software bits reach PA[35:32]; fold with `MipsExecutor::phys_mask`, never decode them |
@@ -170,6 +171,7 @@ Newport (REX3): drawing semantics, HOSTRW, GFIFO, the REX3 JIT, trace and record
 
 | Note | Kind | Title |
 |---|---|---|
+| [cursor-drawn-3px-right-of-x-pointer.md](rex3/cursor-drawn-3px-right-of-x-pointer.md) | open | Newport draws the hardware cursor 3 px right of the X pointer (measured by corner pinning) |
 | [alphahost-fields-and-aline-alpha.md](rex3/alphahost-fields-and-aline-alpha.md) | rule | REX3 source alpha: host fields and antialiased lines |
 | [batched-hostrw-in-the-jit.md](rex3/batched-hostrw-in-the-jit.md) | rule | Batched HOSTRW in the REX3 JIT — and how the equivalence test lied twice |
 | [blendalpha-and-alpha-blending.md](rex3/blendalpha-and-alpha-blending.md) | rule | REX3 BLENDALPHA: what it selects, and why the `blast` billboard exposes it |
@@ -273,6 +275,15 @@ Host GL pass-through for guest GL programs (`iris-hostgl`).
 | [windows-silent-exit-0xc000041d.md](gui/windows-silent-exit-0xc000041d.md) | rule | Windows: CLI vanishes with exit 0xC000041D and no message |
 | [winit-window-calls-must-be-on-event-thread.md](gui/winit-window-calls-must-be-on-event-thread.md) | rule | Window methods must only be called on the event-loop thread (Windows crash) |
 
+## vnc/
+
+The VNC server, the host input layer (`screen`/`ptr`/`key`) and pointer steering.
+
+| Note | Kind | Title |
+|---|---|---|
+| [pointer-steering-and-hotspots.md](vnc/pointer-steering-and-hotspots.md) | design | Steering the relative mouse by the hardware cursor: X's acceleration curve, hotspots from corner pinning, busy guests |
+| [frame-capture-and-damage.md](vnc/frame-capture-and-damage.md) | design | What VNC reads (rgba + frame_seq), Newport capture and forced SW compositor, tile damage |
+
 ## macos/
 
 Mac App Store and sandbox constraints.
@@ -387,3 +398,4 @@ Plans, task briefs and reviews written for agents. Each states its own status; m
 | [networking-tab-redesign.md](plans/networking-tab-redesign.md) | plan | Networking tab redesign — design & plan |
 | [nfsudp-plan.md](plans/nfsudp-plan.md) | plan | Plan: in-core NFSv3-over-UDP server (`src/net/nfsudp.rs`) |
 | [pcap-release-plan.md](plans/pcap-release-plan.md) | plan | PCAP release + elevation plan |
+| [vnc-server-plan.md](plans/vnc-server-plan.md) | plan | Built-in VNC server: board-agnostic frame tap, damage from frame diffs, pointer steered by the HW cursor, monitor/MCP screenshot + click for agents |
diff --git a/clanker/devices/empty-gio-slot-reads-bus-error.md b/clanker/devices/empty-gio-slot-reads-bus-error.md
@@ -0,0 +1,23 @@
+# Reads of an empty GIO slot end in a bus error
+
+Status: fixed 2026-10-29 (`GioBusErrorDevice`, `src/ip22_bus.rs`).
+
+**Symptom.** Indy or Indigo2 with no graphics board (`--headless` with
+Newport): the PROM printed `Cannot open video() for output` only after
+200 000 `MC: GIO Timeout at 1f0f1338` lines, and IRIX later failed (the
+install guide recorded a UTLB miss). The GIO slot probes of a normal boot
+also "succeeded" on empty slots.
+
+**Cause.** Unanswered GIO addresses returned all ones with `BUS_OK`. The
+PROM and the kernel find boards with `badaddr()`, which expects a read of an
+absent board to fault. With no fault, the PROM took the slot for a Newport,
+wrote CONFIG and polled REX3 STATUS (offset 0x1338) for GFXBUSY to clear:
+all ones never clears, so it ran its 100 000-iteration timeout, twice.
+
+**Rule.** With `CPUCTRL1.ABORT_EN` (bit 4, "enable GIO bus time outs"; the
+PROM sets it in its startup code) an unanswered GIO read returns `BUS_ERR`
+(a data bus error exception) after recording GIO_ERROR_ADDR/STAT. Without
+it, all ones as before. Writes are still dropped. Headless Indy and Indigo2
+(IP28 checked) now boot IRIX to the serial login with `hinv` showing no
+graphics; the PROM still says `Cannot open video()` because the NVRAM has
+`console=g`, and waits in its menu instead of autobooting.
diff --git a/clanker/iris_mcp.py b/clanker/iris_mcp.py
@@ -1,6 +1,8 @@
 import asyncio
+import os
 import sys
-from mcp.server.fastmcp import FastMCP
+import tempfile
+from mcp.server.fastmcp import FastMCP, Image
 
 # Initialize FastMCP server
 mcp = FastMCP("iris-monitor")
@@ -272,5 +274,89 @@ async def traceback(count: int = 10) -> str:
     """Show execution traceback (history of executed instructions)."""
     return await client.send_command(f"dt {count}")
 
+# ---- The guest's screen, pointer and keyboard (any board) ----------------
+# These wrap the monitor's `screen`, `ptr` and `key` commands. Coordinates are
+# screen pixels, as in a screenshot. The pointer is steered onto the target
+# by reading back the hardware cursor, so X's acceleration does not matter.
+
+_shot_dir = tempfile.mkdtemp(prefix="iris-mcp-")
+
+@mcp.tool()
+async def screenshot(x: int = 0, y: int = 0, width: int = 0, height: int = 0, max_width: int = 0) -> Image:
+    """A PNG of the guest's screen as the board composes it (no host status
+    bar). Optionally cropped to x, y, width, height, and scaled down to
+    max_width (needs Pillow). Coordinates in the image are screen pixels
+    unless scaled."""
+    path = os.path.join(_shot_dir, "shot.png")
+    crop = f" {x} {y} {width} {height}" if width and height else ""
+    out = await client.send_command(f"screen shot {path}{crop}")
+    if not out.startswith("saved"):
+        raise RuntimeError(out)
+    if max_width:
+        try:
+            from PIL import Image as PILImage
+            im = PILImage.open(path)
+            if im.width > max_width:
+                im.resize((max_width, im.height * max_width // im.width)).save(path)
+        except ImportError:
+            pass
+    with open(path, "rb") as f:
+        return Image(data=f.read(), format="png")
+
+@mcp.tool()
+async def screen_info() -> str:
+    """Screen size, hardware cursor and pointer position, whether the guest
+    has enabled the mouse, and buttons held."""
+    return await client.send_command("screen info")
+
+@mcp.tool()
+async def screen_pixel(x: int, y: int) -> str:
+    """The colour (rrggbb) of one screen pixel."""
+    return await client.send_command(f"screen pix {x} {y}")
+
+@mcp.tool()
+async def wait_screen(quiet_ms: int = 500, timeout_ms: int = 10000) -> str:
+    """Wait until the screen stops changing for quiet_ms (an app finished
+    drawing, a window opened), at most timeout_ms."""
+    return await client.send_command(f"screen wait {quiet_ms} {timeout_ms}")
+
+@mcp.tool()
+async def click(x: int, y: int, button: str = "left", count: int = 1) -> str:
+    """Move the pointer to screen pixel (x, y) and click. button: left,
+    middle or right (IRIX: right opens the toolchest/root menus, middle
+    pastes/moves). count 2 for a double click."""
+    return await client.send_command(f"ptr click {x} {y} {button} {count}")
+
+@mcp.tool()
+async def move_pointer(x: int, y: int) -> str:
+    """Move the pointer to screen pixel (x, y) without clicking."""
+    return await client.send_command(f"ptr move {x} {y}")
+
+@mcp.tool()
+async def drag(x1: int, y1: int, x2: int, y2: int, button: str = "left") -> str:
+    """Press button at (x1, y1), move to (x2, y2), release."""
+    return await client.send_command(f"ptr drag {x1} {y1} {x2} {y2} {button}")
+
+@mcp.tool()
+async def type_text(text: str) -> str:
+    """Type text on the guest keyboard (US layout, ASCII; newline is Enter).
+    Runs of spaces are typed as one."""
+    out = []
+    lines = text.split("\n")
+    for i, line in enumerate(lines):
+        if line:
+            out.append(await client.send_command(f"key type {line}"))
+        if i < len(lines) - 1:
+            out.append(await client.send_command("key Return"))
+    return "\n".join(out)
+
+@mcp.tool()
+async def press_key(keys: str, action: str = "") -> str:
+    """Press a key or chord: 'Return', 'Escape', 'F1', 'ctrl+c',
+    'ctrl+alt+Delete', 'Up'. X keysym names and short aliases (ctrl, alt,
+    shift, enter, esc, tab, bs, del, pgup...). action: '' to press and
+    release, 'down' or 'up' to hold or let go."""
+    return await client.send_command(f"key {keys} {action}".strip())
+
 if __name__ == "__main__":
     mcp.run()