Skip to content

fix: never commit .brain/ into a vault, and write manifests atomically - #236

Merged
joedanz merged 1 commit into
mainfrom
fix/never-commit-brain-dir
Oct 30, 2026
Merged

joedanz merged 1 commit into
mainfrom
fix/never-commit-brain-dir

Conversation

@joedanz

@joedanz joedanz commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • Incident 2026-10-29/30: a hard reboot of embark-brain left the vaults' .gitignore empty. git add -A tracked .brain/index.db (118 MB), and both the cycle and every agent's vault-sync then committed it every run. The brain disk filled (38 GB), three manifests were truncated by ENOSPC, and every agent sync failed (the live index changes between commit and pull). Repaired by hand on the live boxes; this makes it impossible to repeat.
  • holds.commit_vault_strays and vault-sync: exclude .brain/ by name, and untrack it if it is already tracked. They no longer trust .gitignore.
  • compiler.write_manifest: temp file + os.replace, so a cut-short rewrite of the busy key leaves the old manifest whole.

Test plan

  • New tests/test_brain_dir_never_committed.py (4 tests, all failed before the fix): the cycle with no .gitignore, the cycle untracking a committed index (live file kept), a manifest write cut short at half (old bytes intact), and vault-sync end-to-end with a committed index and a brain index that rewrites it every run (pull and push succeed twice, the server never tracks .brain/, the agent keeps its index).
  • Full suite: 1,989 passed. ruff@0.16.0 clean.

https://claude.ai/code/session_01LdXVKJurmhkcFxdUJBLKcr

A hard reboot of embark-brain (2026-10-29) left every vault's .gitignore
empty. The next `git add -A` tracked .brain/index.db (118 MB), and from
then on the cycle and every agent's vault-sync committed a fresh copy each
run: the brain disk filled (38 GB of history from a 143 MB master), three
manifests were cut off mid-write by ENOSPC, and every agent sync failed
because `brain index` rewrites the tracked index between the commit and
the pull.

- holds.commit_vault_strays and deploy/agents-box/scripts/vault-sync
  exclude .brain/ by name and untrack it if it is tracked, instead of
  relying on .gitignore.
- compiler.write_manifest writes a temp file and renames it into place, so
  an interrupted busy-key rewrite leaves the old manifest whole.

Claude-Session: https://claude.ai/code/session_01LdXVKJurmhkcFxdUJBLKcr
@joedanz
joedanz merged commit 2de3d20 into main Oct 30, 2026
6 checks passed
@joedanz
joedanz deleted the fix/never-commit-brain-dir branch October 30, 2026 13:09
@joedanz joedanz mentioned this pull request Oct 30, 2026
1 task done

This branch was successfully deployed

1 active deployment
Preview — 97bc5067 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant