[earakely-scale]

What

A computer env now composes the computer app envs it lists. In env.toml:

type = "computer"
computer = "ubuntu-24.04"
driver = "reference"
apps = ["notes", { env = "clock", install = "build" }]

Each app is built into the computer's image after the driver, in the order listed, each under its own folder (/opt/agentenv/apps/<environment name>) with its own venv, programs and server, as an app deployed alone already is. install = "build" is the default and the only value for now; "boot" is refused as not supported yet.

Design: the M1.3 sections of "Computer Envs and Computer App Envs" ("Including an app", "Composition, end to end", "Several apps"). Issue: FD-4367.

How

• ComputerEnv.apps. put(apps=[...]), from_toml and the wire form carry them; composed_apps() returns them, so the composer, boot readiness and app settings, which already work over that list, take every app. The wire form writes apps only when there are some, so existing computer envs keep their stored form and bundles still reuse them.

• A toml ref with a key beside it. EntityRef(beside=("install",)) lets an apps entry be { env = "clock", install = "build" }; the resolver accepts install there and still refuses any other key.

• Gate 1 over several apps. Each app is checked against the base and the driver as before, now including a port the driver listens on (check_app, so an app deployed alone gets it too). The apps are checked against each other: no arch they all run, one listed twice, or two sharing an environment name, a folder, a supervisord program name or a port, each pair named once, for the first thing they'd share. Every problem of one app is named with its index, apps[1]: ....

• Folders by environment name. An app's folder, its supervisord programs and its conf file take its environment name, which a computer already requires to be unique, instead of its id's last part. Apps whose ids end alike (@a/mail, @b/mail) can share a computer, and a bundle prefix no longer makes a folder's name differ from the app's name. A driver keeps its id's, and its programs and their file are named under driver- (agentenv-driver-<name>.conf, driver-<name>-<service>) as an app's are under app-, so a driver and an app never share a conf file or a program, whatever their names.

• A part undoing what an earlier one installed. An app's files, venv and programs go under its own folder, so the only steps that can write elsewhere are run scripts (as root) and install steps (as the session user). When two or more parts (the driver or an app) have such steps, the builder saves the built image and lists what each step's layer holds, found from the newest layer down as the layer a shell ran exactly that command for. Only certain losses are refused, failing the pair at its build stage, before the push:

  • a later part deletes a file or folder an earlier one left (a whiteout, or a folder made opaque);
  • it points an earlier part's link elsewhere;
  • it makes an earlier part's file, link or folder another kind;
  • it replaces an earlier part's program (an executable file) with other content.

  failed at build: a part undoes what an earlier one installed: /usr/local/bin/tool: @local/~/kit/notes replaces @local/~/kit/mail's program
  

  Any other file two parts wrote differently may be an edit that kept the earlier part's change (two apps adding users to /etc/passwd, a cache rebuilt from every app's entries), so it's said on a progress line, naming the parts, and not refused. Scratch locations (/tmp, /run, /var/tmp, /var/cache, /var/log, .cache, __pycache__) are never compared. apt packages install in one shared layer that's no part's, so files the package manager owns never count.

  • The rules live in utils/clashes.py (compare over each part's id and what one of its commands wrote, in build order). A path is compared with the last other part to write it, whoever wrote it since, until it's deleted or a loss is recorded on it.
  • Where it runs: Builder.written(tag, commands). This machine's Docker saves the image and reads it in-process (utils/layers_in_image.py, standard library only). A builder VM runs the same script in a container of the image, on the image's own Python, so the VM needs none.
  • With one part or none having such steps, nothing is saved.

• Image keys cover what an image is made of. The key covered composer.py's source, so every edit to it, a progress line or a check, built every computer env again. It now covers the base, the Dockerfile, the build files with their executable bits, the lock, the form and the arch, and composer.FORMAT (replacing builder.FORMAT), raised only for a change that alters an image some other way, its labels or how it's built. Changing the key builds every computer env once more after this lands, as this PR's changes to the composer would have anyway; later composer edits won't.

Not in this PR

• Each app's data plane, its child card and /svc/<app>/agentenv through the driver, and the probe running each app's export: M1.4 (FD-4369). Its design doc, CUA Env M1.4: App Data Planes Through the Driver, has seven choices to review.
• Apps installed at boot (install = "boot").

Testing

Unit: 12,257 passed; the one failure is the macOS-only Hermes pipe test, as on main. Several apps' tests are in a file of their own, test_computer_apps.py; the clash rules are direct tables of clashes.compare. New tests:

• An image's key changes with a build file's executable bit and with composer.FORMAT.
• Two apps compose after the driver in their listed order, one of them with an id ending mail in a folder named for its environment name, notes, each with its venv layer and programs; the env stores and reads them back; listing them in the other order makes another key.
• Gate 1 refusals: listed twice, a shared environment name, a shared folder, a shared program name, a shared port, a port the driver listens on, an arch one app doesn't share, no arch all the apps share. Apps given as an iterator are checked and stored alike.
• An env folder naming its apps, by bare name and as a table with install; refused: install = "boot", an app listed twice, another key beside the ref.
• The layer reader on a saved image (a base layer with no history, empty-layer entries, a gzipped layer, files, programs, links, deletions, an opaque folder, setup.sh beside setup.sh.extra, one command run twice, a base layer of the same command), in-process and as the script a VM runs; a command missing or out of order is refused.
• Certain losses (a deletion, an opaque folder, a link pointed elsewhere, a file made a link, a program replaced) fail both pairs at build with nothing probed, also when the later part first copied the earlier one's file unchanged; a file already deleted before a folder is made opaque isn't counted. Edited files and a rebuilt binary cache are said on one progress line and not refused, more than ten summed up; the same content, scratch paths and files only one part wrote say nothing. One part with run scripts saves nothing. On a builder VM, the comparison runs in a container of the image before any push.

Integration, laptop (local Docker, localhost:5000): computer_app_test gains a computer env desk with two apps, notes (PPA package, download, run script, install step, launched window) and clock (tzdata, a run script, a service on its own port). Through agent-env run, 1 passed in 6 min 20 s: each app runs on its own settings from its own folder, clock answers on its port with tzdata's zones, the card is the desk's, both apps' readiness passes, only notes launches a window, and both arches compared what the two run scripts wrote before the push. The comparison took about 3.5 s on that image (docker save 2.9 s, reading 0.6 s), and the real run-script layers held only each app's own output, so nothing needed leaving out.

Modal VM, emulated: builder_modal_vm_test (slow, remote_sandbox-gated) now also reads what each RUN of its arm64 image wrote, in a container of that image under QEMU in an amd64 Modal VM: jq from the apt layer listed as a program, the driver's venv and installed.json from the next. 1 passed in 7 min 6 s.

Modal VM: the same bundle through bundle register --sandbox modal_vm,local, on S3 and ECR. The desk's amd64 built in a Modal VM, ran its comparison in a container of the image there, and was ready in 2 min 23 s; arm64 built natively on the laptop in 56 s. task run --env-sandbox modal_vm then deployed it on Modal VM, and the check returned the same as on the laptop.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no new blocking issues were found.

Summary

Computer environments now build their listed apps into one image, in order.

• Computer environments build their listed apps in order.
• Computer builds compare files written by the driver and apps.
• Computer image keys track the inputs that shape the image.

The earlier findings were rechecked against the current code. Their fixes remain in place. No new actionable issues were found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Base[Base computer] --> Driver[Install driver]
  Driver --> Apps[Install apps in listed order]
  Apps --> Compare[Compare files written by each part]
  Compare -->|Certain loss| Fail[Fail build]
  Compare -->|No certain loss| Push[Push image]
  Push --> Ready[Check readiness]

Loading

_{Reviews (11) · Last reviewed commit: "test(computer): the two-app desk checks ..." · Reviewed by Greptile}