A peer is one binary on a machine with a GPU that puts that machine behind a Behavioral State Protocol door. An agent that knows only the door's manifest can read the hardware state, deploy a model, follow its lifecycle on the event stream, prompt it with a command and read the answer as an event, run a job and undeploy. Nothing is written for the agent beyond the manifest. The same door hosts workspaces, the protocol's tenants: an organization or team creates one, invites the identities who may use it, and configures clouds in it, named pools of peers whose owners admit the cloud at their console; a workload is placed in a cloud of a workspace and every fact about it names the cloud, the peer and the device. Peers find, name and vouch for each other through BEST doors too, so the same word carries to the end game: a network of peers and no centre.
best-peer is an implementation of BEST written in Rust from the spec text alone, sharing no code with any other, so the spec is proven portable and its gaps surface. It wraps an existing runtime (containers with the NVIDIA container toolkit running Ollama or vLLM) and builds no engine of its own. A prompt is a BEST command and its answer a BEST event; the peer reaches the engine's own API behind an abstraction it owns, and nothing of the engine is on the door: BEST carries the whole of it.
The plan is the epic behavioralstate/spec#27 and its slices, on the Behavioralstate board. The board is the plan: no plan file lives here. How the work is made and shipped is SDLC.md.
Done means best-validate green on the served manifest and an agent running the whole lifecycle from the manifest
alone, with the transcript kept in this repository: acceptance/workspace-from-the-manifest.md
is that record, a workspace with a GPU cloud driven on a rented host, made with the published connector and a client
that had nothing but the person's sentence; acceptance/lifecycle-from-the-manifest.md
is the first run, before workspaces.
The path for someone handed a release binary: a Linux PC with an NVIDIA card is the first-class case, and the other cases are one variable away. What the terminal shows at each step is written here so a run that differs is a finding worth sending back.
Before you start. The peer reads the card through the NVIDIA driver, so a GPU host needs the driver installed
and loaded; the peer refuses to start without it. A host with no card (a laptop, a Mac) sets BEST_PEER_GPU=none and
the CPU is the one device. Then one of two ways to run models, each with what it asks of the machine:
| Way | What the machine needs | Settings |
|---|---|---|
| Docker (the default) | Docker Engine running and reachable by your user (docker ps without sudo, or run the peer with it), and the NVIDIA container toolkit registered with it, so that docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi lists the card. Room under Docker's storage for the engine image and the model weights, a few GB. |
None: the peer pulls the engine image, starts the container, publishes its port. |
| Native | Ollama installed and running as a service on its own port. The peer pulls the model through it and claims its port; nothing is started or stopped. No jobs: run-job is refused at ingest, with the fix. |
BEST_PEER_RUNTIME=native. Ollama keeps its own bind, loopback as installed: a prompt reaches it as a command through the door. |
BEST_PEER_RUNTIME=fake is a third way for a machine with neither: the surface answers, nothing runs. Every other
variable has a default that fits these two ways; Run lists them for a setup that differs.
1. Download and run. One line installs the newest release, or upgrades the one you have:
curl -fsSL https://raw.githubusercontent.com/behavioralstate/best-peer/main/install.sh | shIt takes the release's archive for your machine, checks it against SHA256SUMS, and puts best-peer where one
already is on your PATH, so a rerun upgrades in place, else in ~/.local/bin (BEST_PEER_INSTALL_DIR chooses another
place over both; BEST_PEER_VERSION=0.0.12 pins a release). On Windows, in PowerShell, the same, the first time into
%LOCALAPPDATA%\Programs\best-peer, added to your PATH:
irm https://raw.githubusercontent.com/behavioralstate/best-peer/main/install.ps1 | iexNeither touches the data folder, so an upgrade keeps the credentials and the certificate. From then on the peer is
best-peer in any terminal (a new one, on Windows), and the steps below run it so: best-peer --version, then
best-peer to start it; best-peer --help lists the commands and the settings to start with. Or take the archive by hand, unpack it, and run it from its folder as ./best-peer:
curl -fLO https://github.com/behavioralstate/best-peer/releases/latest/download/best-peer-x86_64-unknown-linux-gnu.tar.gz
tar xzf best-peer-x86_64-unknown-linux-gnu.tar.gz
cd best-peer-x86_64-unknown-linux-gnu
./best-peer --version
./best-peerOn Windows x86_64 the archive is best-peer-x86_64-pc-windows-msvc.zip, on macOS Apple silicon
best-peer-aarch64-apple-darwin.tar.gz. The releases page
holds the same archives under names that carry the version, for pinning one, with SHA256SUMS to check them against.
It prints one line, best-peer is serving its door, with where it listens and its data folder, and then waits:
nothing else appears until a client talks to it. On Linux, a peer run without root (or the capability to mount and
the loop devices) prints one more first, this peer makes no disk for a job: it runs every other job, and refuses one
with inputs or outputs (NO_DISK), since a job's disk is one it mounts. If it exits instead, the message names what is missing (the
driver, Docker) and the variable that serves without it. http://127.0.0.1:7777/.well-known/best in a browser is
the manifest, the one document an agent needs.
2. Let someone reach it. By default the peer listens on loopback, so only clients on the same machine see it.
For a person elsewhere, listen on every interface and say the address they will reach, as https://:
BEST_PEER_LISTEN=0.0.0.0:7777 BEST_PEER_ENDPOINT=https://<address they reach>:7777 best-peerThe peer makes itself a certificate at first start, keeps it in its data folder and names the file in the line
https with a certificate this peer made: send that file to the person connecting, and their client trusts it
(NODE_EXTRA_CA_CERTS=<file> for a Node client, SSL_CERT_FILE=<file> for a Python one, --cacert <file> for
curl). A certificate of your own, from your CA or a public one for a name you hold, is
BEST_PEER_TLS_CERT=<chain.pem> BEST_PEER_TLS_KEY=<key.pem>; the peer checks nothing about it beyond that it
loads. An http:// endpoint is served on loopback alone: off it the credential would travel in the clear, and the
peer refuses to start, saying what to set. A hop that a VPN already encrypts (a WireGuard or Tailscale address) is
the one exception, and you say so: BEST_PEER_CLEARTEXT=vpn serves http:// off loopback, with a warning at start,
and lets this door's clouds reach members over http://. Behind a proxy, let it pass TLS through, or re-encrypt to
the peer's own certificate. There is no relay in the peer, so across the internet the address is one you make
reachable (a static IP with a port forward, a Tailscale IP), and the firewall lets in 7777 and nothing else: a prompt
is a command on the door like any other, on that port, with the door's credential. The engines' own ports stay on
loopback whatever the door listens on, and nothing of them is on the door.
3. Admit the agent. The person connecting says to their client "sign me in <name>, start from
https://<address>:7777/.well-known/best"; their client shows them a code like 8BB9-CB44, and they send it to you.
At your machine:
best-peer approve 8BB9-CB44Their client's next poll receives a key under that name. No accounts, no passwords: whoever controls the machine
decides who is in, as with SSH keys, and what they may do: approved with nothing more said, a key may do everything;
--may deploy,prompt --memory 40G makes one that may deploy models and prompt them but run no job, and hold 40 GiB
of device memory at most across its workloads, which it alone sees, prompts and ends. --deployments 2 --jobs 2 lets it
run two of each at once at most; a key may own 8 workspaces when not said, and --workspaces N sets how many. A key
with a share runs its jobs in a closed box (no network, a read-only filesystem, uid 65534 rather than root, ceilings
on host memory, cores, tasks and open files) unless you add --open-jobs. --expires 8h ends the key after eight hours, so access lent for an afternoon
ends without anyone remembering to take it back. To take a key back sooner, at your machine:
best-peer revoke <name>with the name it was approved under: every request carrying it is refused from then on, an event stream it holds
open ends within seconds, and what it ran or had waiting on this peer ends with it (--keep-running leaves that as it
is). The agent can also give its key back itself, with the revoke-agent command, which leaves its work running.
--accelerators-only holds a key to the accelerators, never the host's CPU.
Your own use of the card comes first. When you use the card yourself (a game, a render, a CUDA script), the
peer sees it through the driver and takes no new work on it: once processes that are not its own hold more than
2 GiB of the card or more than 30% of its compute for 30 seconds, get-hardware-state says the card is busy, a
deploy or a job sent to it is refused with DEVICE_BUSY, and a job that asked to wait (queue_seconds) waits; five
minutes after you stop, it is free again and what waited starts. Your desktop alone stays under that. What already
runs on the card runs on. Clients read busy, never what you run, and the peer keeps no record of when you used it:
your use is read live and never written to its log. To take the machine back yourself, at your machine:
best-peer hold --for 2h # every device, the CPU included, takes no new work for two hours
best-peer hold --while 4242 # while process 4242 runs; stays in the terminal, Ctrl-C releases
best-peer hold --device <id> --now # one device, and what runs on it ends now, each holder told
best-peer release # every device takes work again--until 2026-10-20T22:00:00Z names the end instead, and with neither a hold lasts until best-peer release; one by
--while ends after twelve hours at most unless you say otherwise, and needs Linux: elsewhere hold with --for or
--until and end it with best-peer release. When it ends it releases only its own hold: a device you held again
meanwhile stays held. --reason maintenance is one line every approved
client reads, and with --now the holders whose work ended read it after held by its owner; a hold needs none.
A held CPU takes no job with no device either, and --now on it ends one. --now also ends what was starting on the device, its image still pulling, as soon as it starts if the hold still
holds then. A hold is kept
across a restart.
A code waits ten minutes for you; the door holds at most five waiting from one client, sixteen from one address and 64 in all, and past that answers 503 with how long until the oldest expires. An IPv6 address is counted by its /64, which one client holds whole, and a connection whose address is not known is counted with every other such one. Loopback is not counted at all: it is this machine, so a proxy on this machine in front of the door takes the per-address ceiling away, and a proxy on another machine puts every caller behind it under one count of sixteen. The door reads no forwarded address, since any client can write one; with a proxy, set the ceiling per address there.
4. More than one machine. Run a peer on each, and one more best-peer anywhere as the door your team enters
by: a laptop will do, with BEST_PEER_GPU=none and the runtime it has (BEST_PEER_RUNTIME=fake when it has none:
its own device then serves nothing real, and the clouds are what the door is for), and
BEST_PEER_TRUST=<a's cert.pem>,<b's cert.pem> when the machines serve their own certificates. The pool is made through that door, by the agent, from the manifest alone: the
person says "create a workspace called research, then configure a cloud called lab in it with the peers
a=https://:7777 and b=https://:7777". The agent sends create-workspace at the root, reads
the workspace's endpoint from get-workspaces, and sends configure-cloud there, with the credential it signed in
with: a new sign-in is a new identity, and no member (through best-mcp 2.6.0 or later, the workspace is the connection
<name>/<workspace id>). The door signs in to each
machine as any agent would and logs the code for its owner: to admit cloud lab of workspace ws-… on a, run on that machine: best-peer approve <code>. Once admitted, the machine is up in the workspace's get-clouds and its
devices are in its get-hardware-state as lab/a/gpu-0; a deploy in the workspace names the cloud and goes to
the peer with the room, every fact comes back on the workspace's stream with the cloud and the peer named, and a
prompt in the workspace is placed on the peer that holds the deployment and answered on the same stream. A machine that stops answering is PeerLost on
that stream, with every workload it held told as ended, and PeerJoined when it answers again. The same machine
may serve a cloud of another workspace too; one cloud per workspace. Who else may use the pool is the owner's to
say: invite-member with the identity their get-workspaces shows them, and they accept-invitation. A door
started with BEST_PEER_MEMBERS set refuses to start and names these commands: the pool is configured through the
door, not the process.
5. What the agent does from there. Reads the hardware state (a card shows its memory, temperature, power,
utilisation and the CUDA version; the host CPU is listed after it), deploys a model on it, follows the events to
DeploymentReady, sends it a prompt and reads PromptAnswered, runs a job, undeploys. The run on a GPU host, in a
workspace with a cloud, the prompt as a command, is recorded in
acceptance/workspace-from-the-manifest.md; the first run, before
workspaces, in acceptance/lifecycle-from-the-manifest.md. Send back
the terminal output of the peer, whatever happened: the release notes say what is proven and what is not yet.
6. Watch it. /metrics on the same port says how the machine moves, for a collector to chart: each card's memory,
utilisation, temperature and power, each workload's phase and prompts, the door's commands and events. Approve a key
for it alone, best-peer approve <code> --may metrics, and give it to the collector: Metrics says what it
reads and how a collector is set up.
| Crate | What it owns |
|---|---|
crates/door |
The BEST surface, from the spec text: the manifest, the catalogues and schema documents, command ingest with refusals that say what to send and an idempotent id, the event log on disk with its paged history and its live stream, queries, workflows, and sign-in by the device flow with the guidance verbatim; a door that hosts tenants declares their manifests' template and serves each tenant's capabilities as a door of its own. Who approves is an Approver the peer chooses. |
crates/gpu |
The accelerator abstraction: a vendor-neutral snapshot of the host's devices (Gpu), each naming the compute APIs it offers (CUDA, ROCm, oneAPI, Metal, Vulkan, CPU) so a workload asks for an API and memory, never for a vendor. Ships the Fake tests run against. |
crates/runtime |
The container runtime abstraction (Runtime): resolve a reference at its registry to one pinned by digest without pulling (resolve, ImageRef::pinned), an image the registry does not serve not found whatever the engine holds, so that pulling, running and removing by digest never touches a tag on the engine; whether the engine holds a digest under its repository however spelled (holds), the engine's answer alone, so a caller wanting the registry's answer resolves first, pinned references included; pull with progress, or held to limits on the engine's disk (pull_bounded with PullLimits: a bound on what the pull puts there, a floor on what it leaves free; TooLarge or NoRoom when passed, RoomUnknown for a floor where the room cannot be read); resolve and pull with a registry's credential (resolve_with, pull_bounded_with with a RegistryToken: a bearer token for one registry, named by its host, refused before anything is asked for an image of another registry, its token never shown; a runtime that cannot carry one refuses it rather than pull without); start with devices handed over by id and the ports published, name a container for its caller and find it by that name (the newest when several carry it) or say that the runtime cannot, stop, observe, follow what a container writes (out and error apart, from its first byte to its end), and what the engine does on its own told as events. Ships the Fake. |
crates/engine |
The inference engine abstraction (Engine): which engines serve a model (Ollama, vLLM), what to run for each, how a served model is made ready, with progress, and how it is asked to answer a conversation. Ships the Fake. |
crates/peer |
The best-peer binary: the door over the three abstractions, with the peer's own commands, queries and events, its state folded from the events it told; the workspaces it hosts, each a tenant door with its clouds of peers, folded from their own logs; best-peer approve and best-peer revoke, the operator's side of sign-in; and best-peer hold and release, the owner taking the machine back. |
crates/host |
The host itself as a device: a Gpu adapter reporting the CPU and system memory, so a peer with no accelerator serves the whole lifecycle. |
crates/nvml |
The Gpu adapter for NVIDIA cards through NVML: memory, temperature, power, utilisation and the CUDA version, read live, and the processes on each card, so the peer tells its owner's use from its own; the peer refuses to start without the driver unless told the host has none. |
crates/docker |
The Runtime adapter over the Docker Engine API: NVIDIA devices handed over through the container toolkit by their NVML UUID, and the processes of its containers as the engine's top lists them. A reference is resolved through the engine's distribution endpoint, which asks the registry and pulls nothing; a credential travels in the resolve's and the pull's X-Registry-Auth as a registry token, which the engine sends that registry as a bearer token; one pinned by digest is held only when the image it finds lists that digest under its own repository, however spelled (docker.io/, index.docker.io/, library/ and an implicit registry all match), which is the engine's answer alone, so a caller wanting the registry's answer resolves first, pinned references included; it is pulled with no tag made, and is removed as that reference alone, leaving any tag on the image. The peer itself still pulls by tag. A pull held to limits reads the room under the engine's root directory at its start, every half second and at its end, and is given up by dropping the engine's stream; an image it made past them is removed, a reference the engine held given back to the image it named, and an image the engine held before the pull and still holds is never held to the bound by its unpacked size, since none of it landed. An exited container is removed once its followers have read all it wrote, ten seconds at most, and the engine keeps at most two files of 10 MB of each container's output on the host. |
crates/native |
The Runtime adapter for engines already running on the host outside any container (an installed Ollama): a deploy binds to the engine's own port; jobs need Docker. |
crates/engine-http |
The Engine adapter over the engines' own HTTP APIs. |
The three adapters are thin and proven on a host by hand, with the proof on the slice's issue; they are never unit-tested and CI runs on the fakes. Accelerator adapters (NVML, ROCm) are separate crates too, and join the same way.
cargo run -p best-peer # on a host with the NVIDIA driver
BEST_PEER_GPU=none cargo run -p best-peer # on any other machine: the CPU is the one device
curl http://127.0.0.1:7777/.well-known/best
npx -y @behavioralstate/best-validate http://127.0.0.1:7777Or take a release, as in Try it on your machine,
with install.sh or install.ps1 or by hand: one binary per machine (Linux x86_64 on glibc 2.35 or newer; Windows
x86_64; macOS Apple silicon), with SHA256SUMS and a build provenance attestation
(gh attestation verify <archive> --repo behavioralstate/best-peer).
best-peer --version says which it is; best-peer --help lists the commands, and --help after any of them
(best-peer approve --help, best-peer hold --help) says what it takes.
BEST_PEER_LISTEN (default 127.0.0.1:7777) is where it listens; BEST_PEER_ENDPOINT is the consumer-facing base URL
the manifest advertises, required when the peer listens on every interface. An http:// endpoint is served only
when both the listen address and the endpoint's host are loopback (127.0.0.0/8, ::1, localhost); otherwise the
peer refuses to start, naming which is off, unless BEST_PEER_CLEARTEXT=vpn says a VPN already encrypts every hop to
and from it, which also lets its clouds reach members over http://. An https:// endpoint is served by the
peer itself, with rustls: BEST_PEER_TLS_CERT and BEST_PEER_TLS_KEY name the operator's certificate chain and
private key (PEM files, checked for nothing beyond loading); without them the peer makes a certificate at first
start, tls/cert.pem under its data folder, named at start for clients to pin, and made again when the endpoint's
host changes or fewer than thirty days of it remain, which the start-up line says. BEST_PEER_DATA (default
~/.best-peer) holds the event log, the credentials as hashes, the admin token, that certificate, and the record of
where the door is that best-peer approve reads, so it reaches the peer over the same TLS, pinned to that
certificate and no other. The event log is a file per UTC day, events/2026-10-19.jsonl, so a day can be copied,
kept or removed on its own; an event goes to the day of its own time, or to the file being written when that time
is older, so the files in name order are the log in its order. An events.jsonl left by an older peer is read before
them and never written again, so an upgrade keeps the history with nothing to move. Going back to an older peer over
the same folder is not: it reads events.jsonl alone and would number its events over the day files', so move
events/ aside first. Each workspace keeps
its own log the same way under workspaces/<id>/events/. accepted.jsonl, the record of the commands the door
accepted, stays one file.
Every request to the door, a workspace's included, is one line in the peer's log at info: where it came from (the
client's address and port, so the log holds the addresses of everyone who reached the door), the method and the
path as it arrived (never the query string), the status, the holder when a credential opened it, refused or not,
and how long the answer took. The key is never in it. A request the client gave up on before its answer is a line
too, request abandoned with the status -; an event stream's line is written when it opens, and its ms is the
time to the headers, not how long it stayed open. The log is loud by design: an agent's polls, a cloud's ticks and
every stream reconnect are lines. The lines are under the target best_peer_door::router, so
RUST_LOG=info,best_peer_door::router=warn quiets them and keeps everything else. The peer writes its log to
stderr and keeps no file of its own: the record lasts as long as whatever runs the peer keeps stderr (journald under
systemd, by its own size and age limits), so an operator who counts on it as an audit trail sets that retention.
Every door hosts workspaces, the BEST tenants. The root manifest declares their template
(/.well-known/best/{tenantId}), and a workspace's own door is at /workspaces/<id> under the root, with its own
log, catalogue, stream and correlation space; a credential opens the workspaces it belongs to and nothing of the
others, and get-workspaces at the root lists them with each one's endpoint and manifest. At the root:
create-workspace (whoever sends it is the owner; it needs the workspaces scope, and a key owns 8 at most, or the
count its approver set with --workspaces N, refused past it with WORKSPACES_EXCEEDED), archive-workspace (the
owner, once its clouds are archived: it ends for good, closed to every member, its log kept and the credentials it
kept deleted, and it no longer counts against the owner's), invite-member (the owner, naming the invitee's identity,
the opaque id of their credential that get-workspaces shows them), accept-invitation and leave-workspace
(refused while one still holds a workload there); owner and member are the only roles, and the owner stays. The
root's events tell a credential the membership facts addressed to it by identity (the workspace it made, the
invitations it received or sent, its own joins and leaves, and the archive of a workspace it owned) and nothing of
another workspace, whatever its grant; a workspace's own stream tells its members its membership, its clouds and its
peers, and each holder its own workloads, and ends when the workspace is archived. In a workspace: configure-cloud names a cloud, a
pool of peers as peer and endpoint, and makes or changes it (one machine serves one cloud per workspace, and a
peer still holding a workload is not removed); archive-cloud retires an empty one, and get-clouds lists them
with each peer's state. The cloud signs in to each member by the device flow and logs the code to approve there;
the credentials and its place on each member's stream live in workspaces/<id>/clouds/<cloud>/members.json under
the data folder. The workspace's get-hardware-state lists every cloud's devices as cloud/peer/device with the
peers and their state; deploy-model, run-job and pull-image take the cloud (a device names it too) and
are placed by the peer's own rule across its members and sent there as the cloud's command; what a member tells
comes back on the workspace's stream under the caller's correlation id with cloud and peer named; a prompt is
placed on the member that holds the deployment, as the cloud's own command there, and its answer relayed the same
way. BEST_PEER_TRUST lists PEM files to trust beyond the public roots, for members serving their own
certificates. A member named by an http:// endpoint off loopback is refused at configure-cloud
(CLEARTEXT_MEMBER), and nothing is sent to one, since the cloud's credential would travel in the clear, unless this
door's operator set BEST_PEER_CLEARTEXT=vpn. A cloud recorded with such a member before (under an older peer, or
while vpn was set) stops the peer at start, naming the workspace, the cloud, the peer and its endpoint, and how to
name it by https:// instead. Every thirty seconds each cloud squares its state with its members: one that does not answer within
fifteen seconds is told as PeerLost, with its workloads told as ended and its commands in flight failed;
PeerJoined when it answers again, and what it still runs of the cloud's by then is ended there, so the two agree;
a workload a member no longer lists at two ticks running is told as ended. A job is the exception: a member lost under
it queues its order again, once, and fails it the second time (see Deploy). After a restart the cloud
listens again for every workload and order it had sent, so a job that ends while the door was down, or after, is
told with its exit code, not as lost. A credential's share is counted across the whole door: its workloads at the
root and in every workspace together, orders that wait included.
BEST_PEER_GPU (default nvml) is how the peer sees the hardware: through the NVIDIA driver's own library, every
card with its memory, temperature, power, utilisation and the CUDA version the driver offers, read live at every
query, and the host CPU with the system memory listed after the cards; a host without the driver is refused at
start, with the fix. none says this host has no accelerator, and the CPU is the one device.
The container engine is Docker on the same host, reached through its socket or named pipe; the peer refuses to start
without it unless BEST_PEER_RUNTIME says otherwise: native deploys to an engine already running on the host
outside any container (an installed Ollama; no jobs), fake serves the whole surface against the in-memory fakes
(what CI runs).
A job's disk, for its inputs and outputs, is a file system of its own on a loop device, capped at its size and
allocated whole when it is made (16 MiB at least), under disks/ in the data folder. The peer makes it where Docker
runs on Linux and it can mount one: run as root, or with the capability to mount and the loop devices. A peer that
cannot says so at start and makes none, so anywhere else a job with inputs or outputs is refused (NO_DISK) before
any room is reserved. A disk is never promised the last of its file system: 1 GiB, or a twentieth of it when that is
more, stays free for the peer's own log and the engine: the free room and the size get-hardware-state tells are each
what is left after that, so the size is the largest disk the peer could ever make. At
start the peer removes every disk there that no running job has, such as one whose job ended while it was down. Its inputs are fetched and its outputs sent over https://
only; BEST_PEER_STORE_CLEARTEXT=loopback also allows a store on this machine's loopback over http://, for a test
or a developer's own store, on the cloud's door and on its members alike.
A deploy or a job pulls an image named by tag each time, since a tag may have moved, as before; one pinned by digest
(name@sha256:..., handed to Docker whole) names one image for good and is pulled only when Docker does not hold it:
Docker holds it when it keeps that digest under that name, as a pull by tag or by digest of that repository records it,
and not for an image loaded by hand, which records no digest, nor for the digest under another name.
BEST_PEER_PULL=missing pulls any image only when Docker does not hold it: for a host with no way to a registry,
whose images were put there by hand, a held image then runs as it is and is refreshed only by pull-image, which
always pulls. Either way a credential may run any image Docker holds, the operator's own and locally built ones
included, by naming it: a host shared with a tenant keeps in Docker only the images it would let that tenant run.
The engines' images are pinned: ollama/ollama:0.40.6 (0.40.6-rocm on ROCm) and vllm/vllm-openai:v0.32.0, so a
new release upstream changes no deploy until a host has proven it. vLLM's needs an NVIDIA driver that offers CUDA 13
(580 or newer; get-hardware-state says what the driver offers): on an older one its container is refused at start,
and DeploymentFailed says to update the driver or name another build; BEST_PEER_OLLAMA_IMAGE and BEST_PEER_VLLM_IMAGE
name others, or a mirror. BEST_PEER_READY_SECONDS (default 900) is how long an engine may take to answer after its
container started, BEST_PEER_LOAD_SECONDS (default 3600) how long a model may take to load, and a prompt is given
ten minutes to be answered. A pull from Ollama's registry that says nothing for five minutes while the engine does not
have the model (longer while it checks a large model's digests) fails the deploy as stalled, and one whose stream
breaks, or that the engine gives up on, is resumed, twice at most, so a deploy never waits the whole load on a pull
that stopped. BEST_PEER_ENGINE_HOST (default 127.0.0.1) is where the peer
itself reaches the engines' ports, which are bound there and nowhere else whatever the door listens on: on loopback
when the container engine is on this host, on that host's interfaces when it is elsewhere, so a container engine
on another host sits on a network you trust. The peer alone reaches them: a prompt is a command on the door, and
the peer carries it to the engine. Prompts are answered sixteen at a time per peer; the rest wait their turn,
accepted, in order. How many deployments and jobs a key runs at once is set when it is approved
(--deployments, --jobs); a peer started with the old BEST_PEER_DEPLOYMENTS_PER_HOLDER or
BEST_PEER_JOBS_PER_HOLDER refuses to start and says so. BEST_PEER_COMMANDS_PER_MINUTE (default 60) is how many
new commands one credential with a share may send in any sixty seconds across the door, and
BEST_PEER_STREAMS_PER_CREDENTIAL (default 4) how many event streams it may hold open; a number below 1 is refused at start.
A container runs confined: no capabilities, no new privileges, a ceiling on tasks (every thread counts as one), the
NVIDIA driver's compute and utility capabilities alone when it is handed a card, whatever its image asks, and a small init as its first process, which reaps what the image leaves behind and passes a stop on to it. The job of a
credential approved with a share, a tenant's, runs code the owner did not write, so it also gets a closed box: a
read-only filesystem with a private /tmp of 1 GiB, which counts in its memory, no network, and a ceiling on host
memory with swap counted in, half the host's cores and 65536 open files at most; a volume its image declares is a
private /tmp-sized tmpfs too, never a volume on the host. It runs as uid 65534 and gid 65534 (nobody), whatever user
its image names and whatever the host's Docker settings, with /tmp as its HOME unless the job sets one; /tmp,
those volumes and a job's disk are where it may write. No container dumps a core, which the
host's own core handler would write outside it, and each is given by name the seccomp profile and AppArmor's
docker-default the engine has, as its info says them. The owner's own jobs, from a credential with no share, keep the network, a writable
filesystem and the host's memory, as before, and run as their image's own user, root for most, which a writable
filesystem needs; so do a tenant's whose approver said best-peer approve --open-jobs,
which only the approver can say, never the credential or the job. JobStarted says boxed.
BEST_PEER_JOB_MEMORY (default 4G) is the box's host memory ceiling and BEST_PEER_JOB_PIDS (default 4096) every
job's task ceiling, BEST_PEER_ENGINE_MEMORY (default none) and BEST_PEER_ENGINE_PIDS (default 4096) an
engine's; none is no ceiling, and a memory ceiling under the 6M a container starts with is refused at start. Host
memory is not device memory: a boxed job that reaches its ceiling ends in JobFailed naming it, whatever
memory_bytes it asked for. The engines keep their network, which they need to pull models, and a writable
filesystem, which they keep the models on. Capabilities, privileges and the box are never opened by a variable. On a
workspace's cloud, a job runs on the member as the cloud's credential: boxed when the member's owner approved the
cloud with a share and did not open its jobs.
What stands between a workload and the host is a property of each device, and get-hardware-state says it as
isolation, weakest first: process for an engine installed on the host (BEST_PEER_RUNTIME=native), container
for the confinement above, and user-namespace when the container engine also maps its users to unprivileged ones on
the host, which Docker run rootless or with userns-remap does, so root in a container is nobody outside it; the peer
reads it from the engine's own info, and the user namespace is the operator's to set up there. vm, a virtual
machine with the device passed through, is a level no peer offers yet. deploy-model and run-job may ask for one
as isolation: the workload is placed only on a device offering at least that, and refused at once
(ISOLATION_UNAVAILABLE, with what is offered and the fix) where none does. On a workspace's cloud each device says
what its member says, a member that says nothing counting as process, and an order that asks for a level waits only
for a device that offers it.
A device takes new work only while its owner is not using it, and get-hardware-state says so as availability:
free, or busy with until where the peer knows when it frees and reason where the owner gave one. The peer
reads each card's processes through the driver every five seconds, the peer's own containers set aside by the
container engine's top: processes not the peer's that hold more than BEST_PEER_BUSY_MEMORY (default 2G) of the
card or more than BEST_PEER_BUSY_UTILISATION percent (default 30) of its compute for BEST_PEER_BUSY_AFTER (default
30s) make it busy, and BEST_PEER_FREE_AFTER (default 5m) under both frees it; the start-up line says them.
BEST_PEER_BUSY=off turns this off, for a server that runs another GPU service of its own; the other four are
still checked at start. An engine installed on the
host (BEST_PEER_RUNTIME=native) cannot name its processes, so a card it serves on is not read while it does. The
owner's use is never an event: the log would otherwise become a dated record of when the owner sits at their own
machine. best-peer hold and best-peer release are the owner's own act, with the admin token as approve and
revoke take it, told as DeviceHeldV1 and DeviceReleasedV1, which only a credential approved for all reads on the
stream. A busy device is refused at ingest with DEVICE_BUSY (until and reason in the details where known); a
run-job with queue_seconds waits for it; a cloud places on another member, a size on its menu is not
available_now while every device that meets it is busy, and an order whose job a member's hold --now stopped
waits again, once, as for a member lost.
BEST_PEER_MODEL_STORE names a store of models beside the registry: Ollama's library (blobs/ and manifests/,
what OLLAMA_MODELS holds after an ollama pull). A deploy on Ollama of a model the store holds is served from it,
the store mounted read-only at /models, and pulls nothing; a model the store lacks is pulled from the registry as
before. The peer looks for the model's manifest in the store itself before it starts the engine, so the store is a
directory at the same path for the peer and the container engine, which it is when they share a host; one the peer
cannot read refuses the start. vLLM is not served from a store: it pulls as before. A store is for containers: with
BEST_PEER_RUNTIME=native or fake the peer refuses to start. An embedding model cannot be deployed from it, since
a deployment answers conversations. A tenant told that a model failed reads of "the operator's model store", never
its path.
GET /metrics is the door's telemetry, beside the BEST surface and never on it: what is a fact is an event and a
query, and what changes every second is a series a collector scrapes, the peer keeping no history and charting
nothing. It answers in the Prometheus text format, or OpenMetrics when the collector asks for it
(Accept: application/openmetrics-text), so Prometheus, VictoriaMetrics, Grafana Alloy, Telegraf, the OpenTelemetry
Collector and anything else that reads the format can scrape it. It is served on the door's own listener, under its
TLS, behind its credential in X-Api-Key or as Authorization: Bearer, on loopback too; nothing of a conversation
is in it, only counts. A workspace's door serves none: it fronts other machines.
A key reads there what it reads elsewhere: every card's series, unless it is held to the accelerators, when it sees
no CPU; its own deployments and jobs, unless it was approved for all. A key approved --may metrics is what a
collector holds: it reads every holder's workloads, with their holder ids, and the door's own counts, and may do
nothing else: every other path refuses it, commands, queries and events alike, so a key that leaks from a
collector's configuration opens nothing more. metrics stands alone: an approval that names it beside another
scope is refused, since a key that could act would read more of the others here than it may anywhere else. The door's counts leave out what no key reads at the root, who made a
workspace and who joined one. The series, every one named best_peer_… in base units:
| Series | Labels | What |
|---|---|---|
best_peer_info |
version, driver |
The peer |
best_peer_hardware_readable |
Whether the driver answered this scrape (0: the device series are absent) | |
best_peer_device_info |
device, vendor, api, name, apis, slicing |
What each device is |
best_peer_device_memory_total_bytes, …_free_bytes |
device, vendor, api |
Device memory, free as the driver reports it |
best_peer_device_memory_promised_bytes |
device, vendor, api |
Memory promised to every holder's workloads there (all or metrics keys) |
best_peer_device_utilisation_ratio, …_temperature_celsius, …_power_watts |
device, vendor, api |
Where the driver reports them; a MIG instance has none of its own |
best_peer_device_energy_joules_total |
device, vendor, api |
Energy used since the driver loaded, where it says: what a stretch used is the difference, which sampled power misses |
best_peer_disk_free_bytes, …_total_bytes |
The room for jobs' disks, where the peer makes them | |
best_peer_deployment_info, best_peer_job_info |
the workload, device, holder (and model, engine) |
What each workload is, while it lives |
best_peer_deployment_phase, best_peer_job_phase |
the workload, phase |
Each workload's phases as a set, 1 for the one it is in and 0 for the others; gone when it ends |
best_peer_deployment_memory_promised_bytes, best_peer_job_memory_promised_bytes |
the workload, device, holder |
Device memory promised to it |
best_peer_prompts_total, best_peer_prompt_tokens_total |
deployment, outcome or kind |
A deployment's prompts answered and failed, and the tokens the engine counted, while it lives; a prompt that found its deployment gone is a PromptFailed in best_peer_events_total, not here |
best_peer_orders_waiting |
The queue's waiting orders, as the scraping key reads them | |
best_peer_commands_accepted_total, best_peer_commands_refused_total |
command (and code) |
Commands accepted and refused at the door since it started (all or metrics keys) |
best_peer_events_total, best_peer_event_streams |
event |
Facts told since the log began, by type, placements and failures among them; streams open now (all or metrics keys) |
The host's CPU is a device too (api="cpu"): sum(best_peer_device_memory_total_bytes{api!="cpu"}) is the
accelerators' memory. Where the peer sits, its workspace and its cloud, is not a label on its own series, since one machine
may serve the clouds of several workspaces: the collector adds what it knows of the target, or the cloud door does.
A door that hosts workspaces serves its clouds' members' series beside its own, each labelled workspace, cloud
and peer, so one scrape of the door sees the whole pool: every reconcile tick (thirty seconds) each cloud reads a
member's /metrics with the credential the member's owner approved for the cloud, and keeps what is the cloud's
to show: the machine's own series (what it is, its devices, its disk) and, of its workloads, those the cloud placed
there, never another holder's nor the member's door counts. A member that does not answer within five seconds, one
removed from the cloud, or one that no longer admits it, has no series until it does, its absence being the
signal; the workspace, cloud and peer labels are the door's own, whatever a member's series say. A collector's
key reads every cloud's; a key approved for all reads the machines of the workspaces it is a member of; any other
key, none. A Prometheus job for a peer, with the key in a
file only the collector reads and the certificate the peer made named for it:
scrape_configs:
- job_name: best-peer
scheme: https
authorization: { credentials_file: /etc/prometheus/best-peer.key }
tls_config: { ca_file: /etc/prometheus/best-peer-cert.pem }
static_configs: [{ targets: ["gpu-host.example:7777"], labels: { cloud: lab } }]Everything an agent needs is in the catalogues. get-hardware-state says what devices are here and what each offers;
get-workloads says what already holds memory on them. deploy-model names a model, an engine (ollama runs on
CUDA, ROCm or CPU; vllm on CUDA) and the device memory it needs; the peer places it on a device offering an API the
engine runs on with that much room after what is already promised, or refuses at once with the reason and what to do.
On a workspace's door the same commands take the cloud to place in (or a device as cloud/peer/device, which
names it), the devices are every cloud's peers', and every row and event carries cloud and peer.
The facts follow on the event stream under the command's correlation id, in order: ImagePulling, ImagePulled
(held: true when the image was already here and not pulled, ImagePulling then told once, at 100),
DeploymentStarted, WeightsLoading, then DeploymentReady, or DeploymentFailed. While an image is pulled,
ImagePulling says 99 at most and may fall back, since the engine learns of more to fetch as it goes; 100 comes with
the pull's end, just before ImagePulled. What the engine does on its own
later (OutOfMemory, DeploymentEnded), and its end when you take its key back, is told under the same correlation id. An engine whose container exits before
its model is ready fails the deploy: DeploymentFailed carries its exit code and the last lines it wrote. prompt names a ready deployment and
the conversation so far, and PromptAnswered follows under its own correlation id with the answer whole, or
PromptFailed with the reason and the fix. run-job runs an image once, a tenant's in a closed box, and tells
JobStarted and JobFinished with the exit code and output, the last lines the job printed (twenty at most, 2000
characters at most), as does a JobFailed that follows its exit (its outputs not stored); more than that belongs under
/out, which goes to the workspace's store. undeploy-model and
stop-job end them.
On a workspace's door run-job is an order, and a cloud with no room now takes it to wait its turn, as the
serverless GPU platforms do. It is refused at once only when no device of the cloud's members that answer could ever
hold it. JobQueued comes first, holding the order whole, with its id as job and waiting: true when it waits;
the cloud places orders first come, first served, as soon as a member has the memory free, and an order that does not
fit yet holds back none that does. JobPlaced names the member it was sent to, then the member's facts follow, and
JobStarted names the order. queue_seconds bounds the wait, a day when the order names none (past it, JobFailed
says it was never placed), and deadline_seconds the run from JobStarted (past it, the job is stopped and
JobStopped says why, as it does for a job or an order whose key you took back); both are counted from the facts' own times, so a restart neither resets nor extends them. stop-job names a waiting order by its id,
and it ends in JobStopped before it started. A member lost under an order queues it again once, JobQueued
telling attempt: 2, and fails it the second time. get-workloads lists the orders not started yet as queued,
oldest first. An order that waits counts against its credential's share and its count of jobs, as an admitted one
always has. Every member lists its host's CPU after its cards, with the host's memory, so an order the cards cannot
hold runs on the CPU rather than waiting, unless it names a device or its key is approved --accelerators-only, or
it orders a size from the cloud's menu whose compute floor is a GPU's API. The queue is the cloud's state,
folded from these facts; docs/queue.md says what it does not do yet that Verda's batch jobs and
RunPod Serverless do.
A peer's own door keeps the same queue, in the same code, for the orders sent to it directly, and waiting there is
asked for. A run-job that sends queue_seconds or deadline_seconds goes through it, JobQueued coming first; one
that sends queue_seconds waits, with waiting: true, while no device of this peer has its room, and is started as
soon as one has, first come, first served, an order that does not fit holding back none that does. queue_seconds
ends a wait unplaced, deadline_seconds stops a run, stop-job ends an order that waits, get-workloads lists the
waiting ones as queued, and a restart keeps them. A run-job that does not send queue_seconds is refused at once
when there is no room (INSUFFICIENT_MEMORY), as before. That is what a cloud relies on: it sends its members nothing
that asks them to wait, so a full member refuses, and the order stays in the cloud's queue, free to go to whichever
member frees first. What an order waits for is a device it fits, whatever the device is: a whole card, a MIG instance
or the host's CPU.
A cloud can sell by sizes instead of hardware. Its owner sets a menu with set-menu: each size a name, its floors
(the device memory it needs, the compute API at the least version, the host memory of the device's machine), the
share of a card's compute it stands for, and a price per second, in one currency for the menu:
{ "cloud": "lab", "currency": "EUR", "sizes": [
{ "name": "small", "memory_bytes": 8589934592, "compute": { "api": "cuda", "version": "12" }, "share": 0.5, "price_per_second": 0.0005 },
{ "name": "large", "memory_bytes": 42949672960, "compute": { "api": "cuda", "version": "12.6" }, "share": 1, "price_per_second": 0.002 }
] }get-menu answers each size with whether the cloud's members offer it at all (offered) and whether one could
start it now (available_now). run-job names a size instead of memory_bytes and device, and is placed on a
device that meets the size, the smallest that does, so a larger one stays free for a larger size. It is queued as any
order is when none has the room yet. A size no device of the cloud meets is refused with the sizes that are met, and a
size the menu lacks with the sizes it has. JobPlaced and JobStarted carry the receipt: the size, its price per
second and the currency.
A client deals in sizes and never sees the hardware. Nothing the workspace door serves about an order by size names a
card, a device or a MIG profile: not the receipt, not get-menu, get-workloads or get-usage, and not the order's
events, on the page or on the stream, save what the job itself printed (output), which is its own and is relayed as
it was printed. The door's log keeps the device each order went to, for the cloud's own count
of what is held, and leaves it off what it serves. Pinning stays the owner's: memory_bytes and device work as
before. get-hardware-state and get-clouds are the operator's views of the machines and still name them.
A size takes a device whole: nothing else runs there until it ends. A device is a whole card, or one instance of a
card split by MIG (an A100, H100 and newer). On such a card, the peer reports each instance as a device of its
own, with slicing: mig and its own memory, in place of the card. So a split card serves several sizes at once, for
one client or for several, each on an instance that holds its memory, compute and faults to itself. A consumer card
such as a 5090 has no MIG and is sold whole: on one, a job's share of the card was held by nothing, and a fault in one
job ended its neighbours' work (docs/sharing.md). The machine's operator splits a card before the
peer starts, as root, once per boot:
nvidia-smi -i 0 -mig 1
nvidia-smi mig -cgi 1g.10gb,1g.10gb,2g.20gb,3g.40gb -CThe first turns MIG on, and the second makes the instances; nvidia-smi mig -lgip lists the profiles a card offers.
The peer reads whatever instances there are. A card with MIG on and no instances offers nothing.
What the orders by size used, and what they cost, is get-usage, read from the workspace's own events. Each job is
billed from its JobStarted to its end, at the price its receipt carries. A menu price changed while a job runs cuts
it into intervals, each billed at the price it had, and a later change never rewrites what was billed. A job on a
member that was lost is billed to the last moment that member was heard, an event or a keep-alive on its stream
(PeerLost's last_heard), or, where the loss does not say it, to the last fact the member told, never to the moment
of the loss, and says end_inferred. The factor, the share of compute a job had against what its size gives it
uncontended, is 1, since a job has its device to itself, a whole card or a MIG instance. Jobs are named by size, never
by the hardware they ran on. from and to bound the window (a from after to is refused), cloud and job
narrow it, and the totals are per currency. A member reads its own jobs; the workspace's owner reads every one. An
order by memory_bytes is not priced.
The workspace's owner may cap what its orders by size cost together, with set-spending-cap ({ "amount": 50, "currency": "EUR" }, or { "amount": null } to clear it). Under a cap an order by size sends deadline_seconds, so
its most is its price times its deadline. It is refused when it is taken if what was spent, what the open orders may
still spend and its own most would pass the cap, with each named, and refused on a cloud whose menu prices in
another currency. Three things can still pass a cap after an order is taken: a price raised while its job runs, an
order queued again after its member was lost, and an order taken with no deadline before the cap was set, which runs
on unbounded until stop-job ends it.
A job reads and writes through the workspace's store, an S3-compatible bucket such as Verda's object storage. Its
owner sets it once with set-store: the endpoint (https://), the bucket, the region, and the store's
access_key_id and secret_access_key. The credential stays with the workspace's door: it is kept in a file of the
workspace's own, readable by the peer's user alone, and is never recorded, never in an event, and never sent to a
member; get-store shows the store without it. An order names its inputs by key, each with the path it lands on
under /in, and its outputs by a key prefix:
{ "cloud": "lab", "size": "small", "deadline_seconds": 3600,
"inputs": [{ "key": "scenes/shot-12.blend", "path": "shot-12.blend" }],
"outputs": { "prefix": "renders/shot-12" } }The store is the workspace's: every credential that may run jobs there reads any key in it and writes under any
prefix. The cloud refuses two inputs that land at one path or one under another (INVALID_INPUT_PATH) at once, but
asks the store nothing while it takes the order: once the order is taken, before JobQueued, it asks the store each
input's size, eight questions at a time for the whole workspace and a minute for the order. An input the store does not
hold (NO_SUCH_INPUT), one it does not answer for (STORE_UNREACHABLE), inputs not answered within the minute
(INPUTS_LATE) and inputs that together exceed the disk (INPUTS_EXCEED_DISK) end the order in JobFailed, the code
as its code and the inputs named. An order is served from the store it was taken for: one whose store is replaced
while it waits fails when its turn comes.
The member it places the order on gets only URLs the cloud signed for that order: one to read each input, one to
write the outputs. Before the job starts, the member makes it a disk of its own, of disk_bytes (a size names its
default in the menu, and an order may name its own), fetches the inputs there and tells InputsFetched. The job
reads /in, read-only, and writes /out. When it exits, its directories, regular files and links under /out are
packed as one tar archive and sent to <prefix>/<order>.tar as they are packed, so nothing is written beside the disk,
and OutputsStored says so, with the key, before JobFinished. A link is kept as a link, never followed; a pipe, a
device or a socket it left is not an output. A store that does not take it ends the job in JobFailed, its exit code
kept. The order's deadline_seconds covers the sending too: one still running past it is cut and the job ends, its
outputs not stored (JobFailed at a peer's own door; through a cloud, the cloud stops it and JobStopped says so).
A stop while the outputs are being sent ends the job at once in JobStopped, its outputs not stored, and cuts the
sending at its next read; a store that stops answering fails the transfer, ten minutes and a second a mebibyte on,
which also bounds a sending without a deadline. The disk goes with the job, whatever its end, and a peer restarted under a running job finds
its disk again and still sends its outputs. A job with outputs
sends deadline_seconds, six days at most, because the URL its outputs are sent with lives seven days at most. A
member says the room it has for disks in get-hardware-state as disk, and an order is placed only on a member with
its disk free, and waits for one; a disk no member could ever make is refused (NO_DISK_ROOM). At a peer's own door
run-job takes the URLs themselves (inputs with a url, a path and their bytes when known, outputs with a
url and a key), records none of them on its log, and does not wait; where the outputs go is kept beside the
job's disk, readable by the peer alone, for a restart, and goes with it.
The peer's state is the fold of the events it told. After a restart it is rebuilt from the log and squared with what the engine still runs; a workload that ended while the peer was down is told so then.
BEST carries the whole of it. A deployment is prompted with a command, prompt, naming the deployment and the
conversation so far (messages with their role and content, and the engine's knobs: max_tokens, temperature,
stop), and the model's answer is a fact, PromptAnswered, under the prompt's correlation id, with the answer whole,
the model that answered and the token counts the engine reported; PromptFailed says why there is no answer and what
to do. The peer reaches the engine's own API behind the Engine abstraction, on the loopback port the runtime
published, and nothing of the engine is on the door: there is no inference endpoint, no path under a deployment, no
header the engine would read. A prompt needs the prompt scope and a deployment that is ready and the caller's,
refused at ingest otherwise with what to do. The answer is one fact, not one per token: a token is not a fact about
the machine, and streaming partial output is not here (behavioralstate/spec#54 says what is and what waits). What the
engine is asked and what it answers are in the door's log like every other fact, read by the holder under its own
correlations and by nobody else (behavioralstate/spec#53), and kept until the holder says otherwise.
forget is how the holder says so (behavioralstate/spec#68). It names one prompt's correlation, a deployment (every
prompt of the holder's to it) or a time (every prompt of the holder's answered before it; where one correlation holds
prompts answered on both sides of the time, the answers before it are forgotten and the prompts' messages kept, since
which prompt each answer was for is not told apart, and Forgotten names those correlations as held_back to forget
by correlation), and removes the content: the
messages the holder sent, the answers' content, and what an engine said when a prompt failed. Each event and command
stays as an envelope, its id, type, time and correlation with its deployment, model, token counts, holder, cloud and
peer, marked forgotten, so a page of events, a replay of the stream and a restart serve it without its content, the
state folds the same, and what a prompt used is still there to count. It is rewritten in the log's own files, so
nothing of it stays on disk. On a workspace's door the member peer that answered each prompt is asked to forget its
copy too, and Forgotten, which completes the command, says what was removed, what stays, and each member asked,
with how many it took. A prompt still in flight keeps its content until it ends (Forgotten counts them as
in_flight), and a forget sent again then, or after a member could not be reached, removes what is left. A
correlation the holder's commands did not use is refused, as a read of it would be: nobody forgets another's prompts.
Where the boundary lies: the engines' own ports are bound to loopback whatever the door listens on (in native mode
the installed engine keeps its own bind), and the peer is the one thing that reaches them. A prompt on a deployment
that is not ready, or whose undeploy-model was accepted (its phase is ending in get-workloads from then, and
Undeployed follows), or that nobody made, is refused at ingest with what to do; an engine that does not answer
within ten minutes is PromptFailed with the fix. The door promises that a ready deployment answers prompts from
DeploymentReady until the deployment ends: the undeploy, or DeploymentEnded or OutOfMemory telling what the
engine did on its own, within 30 s of it where the engine told nothing itself. What the model says is the model's.
Every command of this peer follows one pattern, and nothing else is needed to drive it:
POST /commandsanswers201with the correlation id, once the command passed its schema and the peer's current state (a deploy no device has the memory for is refused here, not failed later). A correlation id belongs to the credential that first used it on this door, and a command'sidserves as its correlation when it names none: another credential naming either is refused with the field and the fix, since what a credential reads is the facts under its own correlations. A chain of commands under one correlation is one credential's. From that moment the command ends in a completing event, through a restart if need be: a command accepted and not yet run when the peer stops is run when it starts again, through the same state check, or failed with aCommandFailedthat says why; one whose workload had already started and did not survive the restart is failed with the fix. The door keeps the command whole until it completes, then only its id, hash and correlation, which still answer a retry as before, in memory and after a restart alike.- Everything that happens is an event carrying that correlation id, in the order it happened. The command's
schema document lists what it
producesand, among those, whatcompletesit: the events after which the command's outcome is known (DeploymentReadyfor a deploy, orDeploymentFailed,DeploymentEndedorOutOfMemorywhen it ended before it was ready;JobStartedorJobFailedfor a job, and alsoJobStopped, for an order stopped before it started), so a consumer stops waiting for the command and a restart does not run it again; the others are progress on the way (ImagePulling,WeightsLoading, withprogress_pctwhere the engine says).CommandFailedcompletes any command. - What the command made then has a life of its own, told under the same correlation id (
JobFinishedwith the exit code,OutOfMemory,DeploymentEnded), so one filter follows it from the first event to the last. - A consumer that was not listening catches up with
GET /events?correlationId=..., followingnextCursorto the end, and follows from there withGET /events/streamandLast-Event-ID. There is no other channel for progress or outcome, and nothing to poll.
A credential with a share holds 4 event streams open at once at most, counted across the door and every workspace
on it (BEST_PEER_STREAMS_PER_CREDENTIAL); opening a fifth ends its oldest, which reads to its client as a stream
that closed, so it reconnects with Last-Event-ID and misses nothing. Nothing is refused: a client whose connection
died without a close (a dropped Wi-Fi, a laptop asleep) still holds its stream until the door's writes to it fail,
which can take minutes, and its reconnect takes the ghost's place instead of meeting it. Reconnecting costs the
replay since its last id. One stream without correlationId follows everything the credential may read. A cloud
holds one stream on each member. A credential without a share, the owner's, is not counted.
A credential approved with a share (--memory), the tenant's kind, sends 60 new commands in any sixty seconds at
most, counted across the door and every workspace on it (BEST_PEER_COMMANDS_PER_MINUTE): the one past that is
refused with TOO_MANY_COMMANDS, 429 and Retry-After, and was not accepted. A credential without a share, the
owner's own, is not paced, so an evaluation or a batch of prompts runs as fast as the peer answers. A retry of an
accepted command is answered as it was and is not counted, nor is a command refused before it is whole or by its
admission (no room, no such workload). On a member, a cloud is one credential: when the member's owner approved it
with a share, every tenant of the cloud shares the member's pace, and a member that refuses the cloud's command is
told on the cloud's stream as the command's failure, with the member's reason, so a member's owner who hosts a busy
cloud raises the pace or approves the cloud without a share.
Every string and list a command carries has a bound its schema document states (maxLength, maxItems), refused
past it with SCHEMA_VALIDATION naming the field. A request body weighs 256 KiB at most on the door and 4 KiB less
on a workspace's, refused with PAYLOAD_TOO_LARGE and limit_bytes: a cloud passes a command on to a member's door
in an envelope of its own, so whatever the cloud took, the member takes.
An agent signs a person in the way every BEST service is signed in to: it reads the manifest, posts to
deviceAuthorizationUrl, gives the person the link, and polls tokenUrl. The published connector does this by itself.
Who may use a peer is decided by whoever controls the machine it runs on, as with SSH keys: no accounts, no passwords, no identity provider. The page the person opens shows the code and says what to do; on the peer's machine:
best-peer approve ABCD-EFGHThat presents the machine's admin token, which only its owner can read, and the agent's next poll gets its credential:
a key shown once, kept hashed, bound to the connection name the person gave. The token is checked before anything
else, so without it an approval learns neither whether a code is pending nor which scopes the door has. An operator
who wants an identity provider instead plugs one in behind the Approver trait, which says whether a proof is the
authority's (attest, asked first, and for a revoke) and admits a registration (admit); the door never knows which
stands behind it.
Where a credential belongs is the workspaces it created or accepted an invitation into, on this door, and
get-workspaces says which, with the identity the credential is known by there, an opaque id another workspace's
owner names to invite it; the token answer's endpoint is the root, where that query is. A credential that has not
joined any workspace still uses the machine's own surface at the root.
What a credential may do is set at the approval and travels with the key. The manifest's authentication block
lists the scopes: deploy (pull-image, deploy-model, undeploy-model), jobs (run-job, stop-job), prompt
(prompt), workspaces (create-workspace) and metrics (reading the whole door's metrics, and nothing else); all is every scope and every holder's workloads, and what a key approved with nothing said is, so the owner's own box changes
nothing. Reading is every key's but a metrics key's: the hardware state, and its own workloads and events. best-peer approve <code> --may deploy grants scopes; --memory 40G sets the most device memory the key's workloads may hold together,
counted across its deployments, jobs and commands admitted but not yet started; --expires 8h ends the key after
that long (8h, 30m, 2d, or seconds; 366 days at most, and 8 hours at most for a session credential, which an
agent asks for when it cannot keep a secret: more is refused, never cut). best-peer revoke <name> takes a key back
at once, by the connection name the agent signed in under or the holder id the events show, with the peer running
and the same admin token; an agent takes back its own with the revoke-agent command, sent under it. Either way the
key opens nothing from then on, on this door and every workspace door here, and an event stream open under it ends
within seconds, as one does when a key runs out. A key you take back takes its work on this peer's own door with it:
its orders that wait end before they start, its jobs are stopped and its deployments ended, each told with the reason
that its credential was taken back, and one that was starting or queuing meanwhile is ended as soon as it is told.
One the container engine does not stop is named in what revoke prints and tried again until it stops, across a
restart too: the credential's line on disk says its work ended with it. Orders on a
workspace's cloud are the cloud's and are not ended by it. best-peer revoke <name> --keep-running leaves the work as
it is. A key that runs out, or one the agent gave back
itself, leaves its work running, as an agent that ends its session after ordering a long run expects; what it left is
the owner's to end. A revoke the
disk did not keep says so: the key is closed until the peer restarts, its work is ended all the same, and running the
revoke again writes it. Every command document says which
scope it needs. A command the key may not send is refused at ingest with FORBIDDEN and what it was approved for;
a deploy or a job past the share with SHARE_EXCEEDED, what is held and the room left; a workload another key
started, by name, whether a command, a prompt or the events are asked for, and a refusal names only what the key may
see. get-workloads shows a key its own, with held_bytes against share_bytes; DeploymentStarted and
JobStarted name the holder, an opaque id of the credential, never the key. A key with a share runs its jobs on
a device, where its share counts them: a job without memory_bytes, which would run on the host with no device and
hold the host's memory and processors outside the share, is refused with JOB_NEEDS_DEVICE; a key with scopes and
no share is not held to it. In a workspace the rule holds for the cloud's jobs, and a member's owner who approved the
cloud with a share holds every tenant of the cloud to it on that member. A key approved with --deployments N or
--jobs N runs that many deployments or jobs at once at most, counted across the door like the share, every
workspace's included, and the one past that is refused with DEPLOYMENTS_EXCEEDED or JOBS_EXCEEDED: what a
workload claims in memory_bytes is its client's own word, so the share bounds what is promised and the counts how
many, while neither bounds what a workload uses on a card that cannot be partitioned. A key approved without them is
not counted. On a member the cloud's own credential is one holder, so a member's owner who approved the cloud with a
count counts every tenant of the cloud together, and a deploy the cloud accepted can then be refused by the member,
told on the workspace's stream as DeploymentFailed with the member's reason. A key approved with --accelerators-only
sees and uses the accelerators alone, so the host's CPU and memory stay the operator's: get-hardware-state shows it
only those, and a device that offers the CPU alone, or a host with no accelerator, is refused with
ACCELERATORS_ONLY and what it may name instead. A share does not imply it: a key approved with --memory alone may
still be placed on the CPU device, within its share. In a workspace the rule holds for the cloud's devices, and a
member's owner who approved the cloud with --accelerators-only holds every tenant of the cloud to that member's
accelerators. Another sign-in is another holder, as the spec has it: what a key that expired or was
replaced left running is the owner's to end with a key approved for all. The spec has no per-command scope field,
so the requirement is in the description text: it is best-peer's own.