Skip to content

feat(windows): UIA cursor context; pace SendInput keystrokes - #1137

Open
DepengWang wants to merge 6 commits into
Open-Less:betafrom
DepengWang:feature/windows-cursor-context
Open

DepengWang wants to merge 6 commits into
Open-Less:betafrom
DepengWang:feature/windows-cursor-context

Conversation

@DepengWang

@DepengWang DepengWang commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Brings Cursor Context to Windows. The feature already exists on macOS (via AX): it reads a small window of text around the host app's caret and passes it to LLM polish, which helps with homophones and domain terms. This PR adds the Windows half using UI Automation and reuses the existing HostContextAdapter -> cursor_context_input() pipeline, so there is no parallel path.

It also includes one unrelated Windows fix found while testing (dropped characters in SendInput insertion mode). It is a separate commit and can be split out if preferred.

Changes

Cursor context on Windows

  • host_document/windows.rs: UIA reader. Prefers TextPattern2::GetCaretRange, falls back to TextPattern::GetSelection for collapsed selections only, and never guesses a caret from ValuePattern. The text window is built by expanding a range from the caret, so the whole document is never read; Core's existing window_around_cursor / plan_window then slices it to budget.
  • Gated by the same password-field (CurrentIsPassword) and process-blocklist checks as the existing edit watcher, plus a focus-consistency recheck before returning, so an Alt+Tab mid-read cannot mix text from two apps.
  • Runs on spawn_blocking with a 1s outer timeout. Any failure degrades to "no context" and dictation continues.
  • Settings: the Cursor context toggle is now shown on Windows as well as macOS (still hidden on Linux, which has no implementation). All 8 locale descriptions and the types.ts doc comment drop the "macOS only" wording.
  • Logging: the real dictation path now logs capture metadata (status, chars before/after, elapsed, app), matching what the debug probe already logged. No document text is ever logged.
  • README.md / README.zh.md: document Windows support and the VS Code requirement below.

Unrelated fix: SendInput dropped characters (unicode_keystroke.rs, commit 50a02da2)

  • With windowsInsertionMode=sendInput, mixed Chinese/English dictation lost the leading characters of a Latin word right after a CJK-to-ASCII transition ("open" arrived as "pen"). Characters inside a 16-char chunk were sent with no delay.
  • Adds a 1ms gap after every character, mirroring the existing macOS INTER_KEYSTROKE_DELAY.

Testing

Verified on a real Windows 11 machine using Settings -> Experiments & Extensions -> cursor context probe:

  • Notepad, Chrome <textarea>, VS Code: caret position correct, including with the caret in the middle of existing text.
  • Password field on a real Chrome login page: Blocked/secure_text_field.
  • Windows Terminal: Blocked/blocked_app.
  • Captured context reaches the LLM polish prompt during a real dictation.
  • SendInput fix: the dropped-character case no longer reproduces, with no noticeable typing delay.

Not tested:

  • Word, Outlook, Edge, WeChat, WeCom, Teams, password managers.
  • The UIA timeout path against a genuinely hung host app.
  • macOS regression (no Mac available). The shared files touched are core_adapters.rs, host_document/mod.rs and openless-core/src/host_document/window.rs.

Known limitation

VS Code and other Chromium/Electron editors only keep the accessibility tree's caret in sync once they detect an accessibility client. With default settings the read succeeds but can return a stale position. Setting editor.accessibilitySupport to on and reloading the window fixes it. This is documented in both READMEs.

Screenshot

Settings: cursor context toggle and encrypted sync signed in with a self-hosted token

Windows, Settings -> Permissions & data. The Cursor context (experimental) toggle is now available on Windows. Taken from a local build that also has #1133 merged, which is why the encrypted sync card below shows a self-hosted token sign-in.

🤖 Generated with Claude Code

DepengWang and others added 6 commits October 2, 2026 18:55
Adds a Windows UI Automation implementation of read_around_cursor, reusing
the existing HostContextAdapter -> cursor_context_input() pipeline instead
of a parallel path. Prefers TextPattern2::GetCaretRange (active caret only),
falls back to TextPattern::GetSelection for collapsed selections only, and
never guesses a caret from ValuePattern. The text window is built by
expanding a Range from the caret (never reading the whole document), then
sliced to budget by Core's existing window_around_cursor/plan_window.

Gated by the same password-field (CurrentIsPassword) and process-blocklist
checks as the existing edit watcher, plus a focus-consistency recheck
before returning a result so an Alt+Tab mid-read can't mix two apps' text.
Runs on spawn_blocking with a 1s outer timeout; any failure degrades to
no context, never affects dictation.

Adds budget-split/redistribution/emoji-boundary tests to openless-core's
window.rs (previously untested) and a pure is_blocked_process_name helper
with tests in windows.rs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
Shows the Cursor context toggle on Windows as well as macOS (Linux still
has no implementation, so it stays hidden there). Updates the
cursorContextEnabled doc comment in types.ts and all 8 locale descriptions
to drop the "macOS only" wording and mention password fields/terminals are
excluded on both platforms.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
Updates the Cursor Context sections in README.md and README.zh.md to
cover Windows: TextPattern2 caret-range priority, TextPattern selection
fallback, and the "skip rather than guess" rule when the caret can't be
reliably located. Password-field exclusion wording now also mentions the
Windows UIA password-control flag alongside macOS Secure Input.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
TauriHostContextAdapter::capture previously logged nothing on success —
only debug_read_cursor_context exercised the status/chars_before/
chars_after/elapsed_ms/app log line. Extends the same metadata-only log
(no document body, per design doc §18/§19) to every real dictation that
has cursor context enabled, so whether a capture actually happened is
visible from the normal log file without a devtools round-trip.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
Windows SendInput text injection sent characters within a chunk back to
back with zero delay, only pausing every 16 characters. Found while
testing cursor context (windowsInsertionMode=sendInput): dictating mixed
Chinese/English text dropped the leading character(s) of a Latin-script
word right after the CJK-to-ASCII transition, e.g. "open" arrived as
"pen" and "OpenLess" vanished entirely — well before the 16-char chunk
boundary, so the existing chunk-level pause never applied.

macOS already works around the same class of problem (its
INTER_KEYSTROKE_DELAY comment: "Chromium / Electron / Tauri themselves
drop characters when keyDown/keyUp have no delay"); Windows had no
equivalent. Adds a 1ms gap after every character, not just at chunk
boundaries — confirmed on hardware to stop the drops, with no
perceptible typing latency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
Confirmed on hardware: Chromium-based editors (VS Code) only keep their
accessibility tree's caret position in sync once they detect an active
accessibility client, so with default settings the position UI Automation
reads back can be stale even though the read itself succeeds. Setting
editor.accessibilitySupport to "on" (plus a window reload) fixed it.
Documents this in both READMEs so users don't mistake it for a bug.

Also adds a metadata-only diagnostic log (source pattern + requested vs.
actual Move distances, no document content) to host_document/windows.rs,
added while narrowing this down between VS Code/Chrome — kept because it
is cheap, useful for future reports of "wrong context" on a given app,
and fully compliant with the no-document-body logging rule.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011h9QUkoFtCoDLsJjRDB6dX
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant