Skip to content

Server Federation - #3

Open
DavidValin wants to merge 7 commits into
mainfrom
server-federation
Open

DavidValin wants to merge 7 commits into
mainfrom
server-federation

Conversation

@DavidValin

@DavidValin DavidValin commented Sep 11, 2026 •

Copy link
Copy Markdown
Owner
  • Federate 2+ servers, pinned and encrypted via mutual pqhybrid
  • Share unique channels registry
  • Share unique nicknames registry
  • Each server is responsible for their nicknames registration / logging
  • Tell a user logging in on the wrong server which one to use
  • Detect name collisions between servers (conflicted, refused until resolved)
  • Forward OTP Mail to recipient home server for final delivery to user
  • Relay the delivery receipt back to the sender's home server
  • Broadcast New Channels between federated servers (one shared public list)
  • Broadcast User Joins / Leaves, relayed via the home server to every server with a member in that channel
  • Proxy private-channel joins to the home server, which alone checks the password / bans / allowlist - the password never leaves it
  • Propagate channel deletion and account removal, so names are freed federation-wide
  • Release a decommissioned peer's claimed names (--forget-federation-peer)
  • Log federation events to the server console
  • Federate peer-to-peer link relay - two members on different servers are introduced through a server they share, then talk directly
  • Smoke test two real servers end to end (never yet run outside tests)
  • Check how a federated member actually renders and behaves in the client

@DavidValin DavidValin added the enhancement New feature or request label Sep 11, 2026
@DavidValin DavidValin self-assigned this Sep 11, 2026
- Public channels created on one federated server were invisible to
other servers' clients until someone manually joined by name — now
they're mirrored and announced live everywhere.
- Members of a shared channel connected to different federated servers
couldn't see each other join or leave at all — now presence is relayed
through the channel's home server.
- That presence relay didn't work between two servers with no direct
link to each other — fixed by having the home server broadcast to every
linked peer, not just the one a request came from (proven with a
3-server hub test).
- A server receiving its own gossip back (about its own client, relayed
by the home server) would wrongly re-record that client as a "remote"
member and tell them about themselves — fixed with a self-echo guard.
- Channels with only remote (federated) members and zero local members
could be incorrectly deleted by the inactivity sweep, which only checked
local members — fixed.
- The home server had no way to learn a remote joiner's identity (public
key/key mode) to pass along to other peers — added it to the join-proxy
request.
- Existing tests assumed a stale message ordering (joiner gets Joined
immediately) that the new presence notices changed — updated to expect
the new, correct ordering.
Blocker fixes
- Federated members' synthetic UserIds moved below the top bit
(FEDERATED_ID_CEILING) — they collided with client::p2p's reserved
direct-punch half, leaving every federated member permanently
"Connecting" with sends queueing silently forever.
- JoinProxyResponse now only accepted from the peer the request was sent
to — any linked peer could forge a channel grant.
- A link is now torn down when its writer fails, not only its reader — a
half-open socket left a dead sender registered as live, so nothing
redialed and all traffic vanished silently.

Resource bounds on the federation port
- Handshake timeout (20s) and a 64 KiB pre-auth frame cap
(ControlReader::set_max_frame_len), lifted once the peer is verified.

Presence correctness
- New ChannelMembership message: the home server sends full membership
at link-up, applied as a replacement — so a peer linking late (or
relinking) no longer shows a busy channel as empty forever.
- Link-down now forgets that peer's members with proper UserLefts,
rather than leaving ghosts that also pinned the channel alive.
- /ban now removes federated members and gossips it; join_remote checks
the ban list unconditionally. A ban was previously unenforceable against
anyone connected through another server.

Mail relay
- MailForward checks is_delivered first — it was resurrecting
already-delivered mail.
- New MailReceiptAck — relayed receipts were immortal and re-sent in
full on every reconnect.

Validation
- Nicknames/channel names from gossip are validated before storage
(newline injection could forge entries in the persisted directory file).
- A server refuses to start if its own server_federation_id contains a
tab, newline or comma.

Login redirect
- New server_federation_client_addr, announced over the link, so
redirects name the client port instead of the federation port (which
nothing answers on).
- Addresses moved inside the signed handshake transcript — otherwise a
relaying MITM could rewrite the address users are told to connect to.

Dial direction
- Both sides dial again; simultaneous connections resolve via a rule
both ends compute identically, with link_id-checked teardown so a
displaced link can't remove the winner's entry. Replaces the "only the
lower id dials" rule, which broke NAT deployments.
…re introduced through a server they share, then talk directly
- Cancel-unsafe read in select! (federation/mod.rs) — rd.recv() was
raced against a ping-tick inside select!; since ControlReader::recv
isn't cancel-safe, a tick firing mid-frame silently dropped bytes and
desynced the link. Moved the read loop onto its own task (like the
existing writer task) so it always runs a recv() to completion.
- Directory gossip never relayed past one hop (federation/directory.rs,
federation/mod.rs) —
NicknameRegistered/NicknameRemoved/ChannelRegistered/ChannelRemoved were
only applied locally, not forwarded, so a hub topology (servers linked
only through a shared third server) silently diverged. Made
Ownership::merge/merge_channel_entry/remove_nickname/remove_channel
report whether they actually changed anything, and relay each gossip
message to every other linked peer gated on that change (so it
terminates instead of bouncing forever).
- No sender-authenticity check on membership gossip (federation/mod.rs)
— any linked peer could send ChannelMemberJoined/ChannelMemberLeft for a
channel it doesn't own, injecting fake members. Added the same "does the
directory say this peer actually owns the channel" check
ChannelMembership already had. (Deliberately not applied to the four
directory-gossip messages above — that would break the relay fix, since
a relaying hub is never the original claimant, and directory-claim trust
is already an accepted, documented boundary in docs/SECURITY.md.)
- Stale doc comments contradicting the code:
  - dial_peer's comment claimed only the lower-id server dials; the code
has both sides dial and resolve it via connection_is_canonical.
  - README.md and RemoteIdentity's doc in proto.rs both still said
cross-server live chat "doesn't work yet" — it's implemented.
  - A misattached doc block had the ping/idle-timeout explanation
sitting on PEER_SIGNAL_MAX_HOPS, leaving
FEDERATION_PING_INTERVAL/FEDERATION_IDLE_TIMEOUT undocumented.
- Failed handshake reset the redial backoff (federation/mod.rs) — a
refused/misconfigured peer got redialed (and logged) every ~10s forever
instead of escalating. run_peer_link/dial_once now return whether the
handshake actually succeeded, and dial_peer only resets backoff on that,
not on every Ok.
- Swallowed directory-save errors (server/mod.rs, federation/mod.rs) —
four let _ = .../.unwrap_or(false) call sites silently discarded
disk-write failures, letting the in-memory directory diverge from what's
on disk with no record of it. Added a shared log_directory_save_failure
helper and wired it into all four.
- Doc sweep for remaining stale references:
  - docs/PROTOCOL.md §18.2 only described gossip reaching "every
currently-linked peer" (true before the relay fix, incomplete after) —
added the relay/termination explanation.
  - proto.rs's four gossip message docs (NicknameRegistered,
ChannelRegistered, ChannelRemoved, NicknameRemoved) likewise didn't
mention relaying — added a line to each.
  - Checked docs/SECURITY.md, docs/SPEC.md, handshake.rs, directory.rs's
module doc, tests, main.rs, settings.rs — no other stale references
found.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant