Skip to content

fix(remote): load a remote session's scrollback from the remote tmux - #602

Merged
Ark0N merged 2 commits into
Ark0N:masterfrom
timkjr:fix/remote-shell-history
Oct 13, 2026
Merged

Ark0N merged 2 commits into
Ark0N:masterfrom
timkjr:fix/remote-shell-history

Conversation

@timkjr

@timkjr timkjr commented Oct 10, 2026

Copy link
Copy Markdown

A remote SSH session's local pane runs ssh -t … tmux new-session -A (or attach), so the local tmux only sees the remote tmux client's full-screen repaints and keeps a fraction of the history: measured 48 of 279 lines while output streamed, none after a reattach. The remote pane, where the shell runs, keeps all of it.

  • A full-history load (GET /api/sessions/:id/terminal?full=1) of a remote session now reads the scrollback rows from the remote pane in one ssh round trip (#{history_size} + capture-pane -S -<N> -E -1, exact =name: target, newest bytes kept by a remote tail -c).
  • The local capture reads only its visible frame (-S 0), spliced below them (PaneCaptureOptions.scrollbackOverride). The local frame and caret are kept because the live stream keeps drawing into the local pane.
  • Fail-soft and bounded: connection options from buildSshConnectionArgs(), the shared remote-ssh limiter (a slot waited for longer than 1.5 s is given up), a 5 s timeout, and a 30 s per-host back-off after any failure, which falls back to the local capture as before.

docs/remote-sessions.md updated.

Testing: new test/remote-pane-history.test.ts and route tests; typecheck, lint, frontend syntax, format check and the full suite green on this branch. Live against a remote host on my own deployment: a full=1 load takes ~170–200 ms (the ssh round trip) with no fallback warnings.

Known interaction with #601: if both land, the 'tail' truncation flag added there compares against the LOCAL pane's #{history_size}. For a remote shell that no longer describes the spliced remote history, so the banner may not offer Load full history for a remote shell with more than 10,000 remote lines. Not fixed in either PR; happy to follow up once the order is settled.

A remote SSH session's local pane runs `ssh -t … tmux new-session -A` (or
`attach`), so the local tmux only sees the remote tmux client's full-screen
repaints and keeps a fraction of the history: measured 48 of 279 lines while
output streamed, none after a reattach. The remote pane, where the shell
runs, keeps all of it.

A full-history load (`GET /api/sessions/:id/terminal?full=1`) of a remote
session now reads the scrollback rows from the remote pane in one ssh round
trip (`#{history_size}` + `capture-pane -S -<N> -E -1`, exact `=name:`
target, newest bytes kept by a remote `tail -c`), and the local capture
reads only its visible frame (`-S 0`), spliced below them
(`PaneCaptureOptions.scrollbackOverride`). The local frame and caret are
kept because the live stream keeps drawing into the local pane.

Fail-soft and bounded: connection options from buildSshConnectionArgs(),
the shared remote-ssh limiter (a slot waited for longer than 1.5 s is given
up), a 5 s timeout, and a 30 s per-host back-off after any failure, which
keeps the local capture as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Ark0N

Ark0N commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks a lot for this, @timkjr. It fixes the near-empty scrollback on remote SSH sessions by reading the history from the remote tmux pane and splicing it above the local frame, and the write-up with the measurements made it easy to review.

I ran it against real tmux (a 'remote' tmux server and a local pane attached to it, then your script and capturePaneBuffer with the override). The splice is exact: 300 of 300 lines, caret row last, and still correct after a resize. The exact =name: target and the double shellescape hold up too. Typecheck, lint, format and the full npm test are green here.

One thing needs fixing before merge, plus a few smaller ones:

  1. The 1.5 s slot-wait bound is not enforced (src/remote-pane-history.ts:154). The check runs inside the task, so it only fires after runWithRemoteSshLimit() has already handed over a slot, and the request still waits for everything queued ahead of it. With the pool saturated I measured fetchRemoteScrollback() returning null after 6008 ms. File probes can hold a slot for 20 s, and the terminal route awaits this before capturing, so a busy pool now stalls remote tab loads. Please add a bounded acquire to src/remote-ssh-limiter.ts (for example a maxWaitMs option that removes the waiter from the queue on timeout and never touches active), use it here, and add a limiter test showing that a timed-out waiter does not run and does not leak a slot.

  2. Back-off scope (src/remote-pane-history.ts:171). Every failure backs off the whole host for 30 s, including a missing remote tmux session (tmux exits 1 and ssh passes that through). One stale non-owned session then disables remote history for every other session on that host. Please back off only on transport failures (ssh exit 255, a timeout, unparseable output), the split classifyRemoteAliveExit() already makes, and just return null for a per-session failure.

  3. Seam gaps (src/web/routes/session-routes.ts:3125). Two cases drop rows without saying so: rows that scroll off the remote screen during the round trip (with a 150 ms delay and fast output I lost 13 lines), and a shared session whose latest client is taller than this pane (window-size latest; I lost 20 rows). Neither is worse than today. Please list both in the new docs/remote-sessions.md section. If you want to go further, capture the remote visible rows too and splice where they overlap the top of the local frame.

  4. Tests for fetchRemoteScrollback() itself (src/remote-pane-history.ts:143). The VITEST guard returns before the back-off, slot-wait and error paths run, and item 1 sits in exactly that code. An injectable runner would let a test pin both. A source assertion for the new -S 0 branch in test/tmux-capture-full-history.test.ts would match how the other capture flags are pinned.

  5. Nit: the @fileoverview of src/remote-ssh-limiter.ts and the 'Remote sessions over SSH' section of docs/architecture-invariants.md still describe file access as the only ssh consumer. One sentence in each about the history read would help.

Thanks also for flagging the interaction with #601. I'll settle the merge order between the two, and whichever lands second gets the small adjustment. Once item 1 is in (2 to 5 can come in the same push), this is ready to merge.

…f only on transport failures

Review follow-ups for the remote scrollback read.

The 1.5 s slot-wait bound was checked inside the task, so it only fired
once runWithRemoteSshLimit() had already handed over a slot, after
everything queued ahead of it. runWithRemoteSshLimit() now takes
`maxWaitMs`: a waiter that times out leaves the queue, rejects with
RemoteSshSlotTimeoutError, never runs and never touches `active`. The
history read uses it, and a slot timeout does not back the host off.

Only a transport failure backs the host off now (ssh exit 255, a
timeout, a spawn error, output that is not the script's), via
classifyRemoteAliveExit(). A remote command failing is about that
session only. A live run against a real host showed a missing session
comes back as exit 0 with empty markers, because the `| tail -c` pipe
takes tail's exit status, so markers that arrived without a size are
also per-session.

fetchRemoteScrollback() takes an optional runner as a test seam; tests
now cover the bounded wait, each back-off class, its expiry, and the
masked missing-session case. A source pin covers the `-S 0` capture
branch. The two seams that can drop rows at the join (rows that scroll
during the round trip, a taller shared client) are documented in
docs/remote-sessions.md, and the limiter's header and the remote-ssh
invariants now name the history read as an ssh consumer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@timkjr

timkjr commented Oct 11, 2026

Copy link
Copy Markdown
Author

Thanks for running it against real tmux. The follow-ups are in 136f115, on top of the original commit.

  1. runWithRemoteSshLimit() takes maxWaitMs: a waiter that times out is taken out of the queue and rejects with RemoteSshSlotTimeoutError. It never runs and never touches active. Limiter tests cover a timed-out waiter (it doesn't run, no slot leaks, and the waiter queued behind it still gets the slot), one served in time, and a free slot. Live, against a saturated pool, fetchRemoteScrollback() returns null at 1500 ms.
  2. The host now backs off only on a transport failure, via classifyRemoteAliveExit(), or on output that isn't the script's. Running against a real host turned up one more case: a missing session comes back as exit 0 with both markers and nothing between them, because the | tail -c pipe takes tail's exit status. That's per-session too now, pinned by a test. Live, a missing session returns null in 144 ms with no back-off, and an unreachable port backs the host off.
  3. Both seams are listed in docs/remote-sessions.md with your measurements. I also built the overlap splice you suggested: the remote screen is captured in the same round trip, and the local frame is matched against it as text. It requires a minimum overlap, so a changed spinner row can't make it place the frame too low and repeat rows. I'll send it as its own PR on top of this one once I've run it end to end, so it doesn't hold this one up.
  4. fetchRemoteScrollback() takes an optional runner as a test seam. Tests now cover the bounded wait, each back-off class, the back-off's expiry and the masked case. The -S 0 branch is pinned in test/tmux-capture-full-history.test.ts.
  5. Done in both places.

The #601 interaction is in my reply there: with both in, #601's alternate-screen fallback would discard this splice for remote shells, so whichever lands second needs to skip it when the remote override was used.

@Ark0N
Ark0N merged commit f9f00bd into Ark0N:master Oct 13, 2026
2 checks passed
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…ge-time fix)

joinScrollbackAndFrame joins two separate `capture-pane -e` outputs. The
remote history's last row can end with attributes still active (a diff row's
background, dim tool output), while the local frame is encoded relative to
the default state, so its first plain rows inherited them in the browser. A
`\x1b[0m` now ends the scrollback before the frame.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…602 integration)

A remote session's local pane hosts the remote tmux client, so it is always
in the alternate screen. #601's alt=recording fallback therefore threw away
the scrollback #602 had just spliced in from the remote tmux on every remote
shell load. The fallback now applies only when no remote scrollback was
read (the read failed, or the session is not remote).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…in a remote history load

Conflict resolved in tmux-manager.ts: #609's screen-aware
joinScrollbackAndFrame keeps #602's merge-time SGR reset before the frame,
and #609's join tests expect it. The #601/#602 route test now mocks the
remote read in #609's { scrollback, screen } shape.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…tegration)

A remote session's local pane hosts the remote tmux client in the alternate
screen, so its history_size is always 0. #583 read that as "nothing to load"
and hid the partial-history notice, but with #602 a full=1 load reads the real
scrollback from the remote tmux. The route now leaves the field out for remote
sessions, so their notice keeps the byte-based wording and its Load full
history button.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…fore a prompt upload

Conflict resolved in test/routes/session-routes.test.ts: #602's
remote-pane-history mock and #605's bounded-path-probe mock are both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N pushed a commit that referenced this pull request Oct 13, 2026
…-roll video uploads

Conflicts resolved against #605 (and #602's test mock):
- CLAUDE.md Prompt uploads: #606's rewrite is kept and #605's bounded
  workspace probe sentence follows the remote (SSH) refusal.
- test/routes/session-routes.test.ts: the remote-pane-history and
  bounded-path-probe mocks are kept beside #606's _resetPasteRateBuckets
  import, and both PRs' paste-image tests are kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Ark0N

Ark0N commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Thanks @timkjr, shipped in 1.41.1. One fix at merge: the remote history and the local frame come from two separate capture-pane -e reads, so the last remote row's SGR state (a diff row's background, dim text) carried into the frame's first plain rows. There is now a \x1b[0m at the seam, with your test expectations updated. Also, a remote session reports no paneHistoryLines, so the fullscreen fast path from #588 never skips your remote read.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants