Repository navigation
feat(memory): open-source the v0.3.4 AML engine as an opt-in backend - #174
Merged
Merged
Conversation
Grivn
marked this pull request as draft
October 30, 2026 20:18
Grivn
force-pushed
the
codex/open-source-v0.3.4
branch
from
October 30, 2026 20:31
052f000 to
31bf838
Compare
Grivn
marked this pull request as ready for review
October 30, 2026 20:34
Add a separate revisioned raw-record store and an explicit memory-agent backend: bounded BM25 retrieval, System 2 query planning, JEV evidence judgments, and a time-ordered View. Background layered consolidation links its topics, values and instructions back to the original records. Include the per-user Add/Search adapter, supervised workers, provider budgets and retries, authenticated inspection, metrics, deployment tooling, and the deterministic tests of the evaluated baseline. Records retain opaque payloads and revision/CAS semantics; JEV request limits use UTF-16 units, and historical evidence keeps its correction/time boundaries. Keep the current public CLI search helpers independent from the frozen retrieval primitives from this first milestone. Adjust the architecture oracle and remove a deployment-test dependency on excluded research docs. Internal research/prototype files and reverted experiments are not imported. Evaluation-Commit: b2d0b0e05e5185f45b77d8e04f128aecf4bfd4ee Archived-Milestone: 13dda8a
Consolidate full user and other-role messages while reading the first 240 UTF-16 units of each assistant message by default, matching the reference reading policy. Preserve complete raw messages for storage and retrieval. Expose validated role-reading overrides and add request-role, model-cost, backlog and delivery observability. Add role-boundary and runtime-policy regressions together with the implementation. Evaluation-Commit: ae4562efbe477ab3e5dc19a4c546033e95c79a7d Archived-Milestone: 7351d71
Write service logs into UTC hourly files and expose authenticated query and export routes. Wire log directories through the service command and deployment unit, with rotation, filtering and path-boundary coverage. The Add/Search wire contract and the memory policy remain unchanged. Evaluation-Commit: 66ec975c95cea9748f0162e206db67ba6dced768 Archived-Milestone: 228ef55
Expose each stored conversation line's role in inspection and count reads whose fully judged evidence set is empty. Reporting no evidence does not add an abstention gate or discard the existing response. Provide pattern/checksum-based output masking behind safety.mask, disabled in the evaluation profile. The experimental forgetting, abstention and profile-extraction switches are absent from this version. Include the safety package's architecture dependency assertion with its introduction rather than deferring that existing test correction to the next version. Evaluation-Commit: ddf9b154bb34e726d91d19a258541500592fc043 Archived-Milestone: 53ecd19
Allow Adds to finish while their user's consolidation is backing off. Before a degraded Search, catch up the lexical index and rank through SQLite BM25; avoid rebuilding a whole in-memory words index for every concurrent fallback. Defer unused words statistics after background builds and keep watchdog probes independent from budget-file I/O. Recover worker/store-open panics and close partial resources so retries neither strand a user nor leak stores. Carry the catch-up, backoff, cleanup and worker-liveness regressions with the fixes. Keep the original v0.3.4 memory policy and industrial profile. Evaluation-Commit: c09e4a8b1a94b0d8e3bc74259fc40a5cb6ecb4cd Archived-Milestone: 14f4c36
Document explicit opt-in commands, unchanged native CLI/storage defaults, and the five evaluation-source mappings. Preserve the full sanitized history on archive/aml-v0.3.4-development while keeping this import reviewable. Include the reference implementation and V8/fdlibm license notices in binary/npm packages, update deployment and security guidance, and retain only public documentation and fixtures. Native-helper and public test-path compatibility adjustments already belong to the first milestone. No evaluation tag, deployed service, private data or release version changes. Archived-Commit: 261df69
Use a platform-absolute temporary log directory. Give the startup test a bounded 30-second readiness wait and report early process failure instead of losing the underlying error behind a generic timeout. Keep the shutdown deadlines, request cancellation and budget settlement assertions unchanged. This is the same tested harness adjustment as the archived integration, with no production behavior changes. Archived-Commit: 31bf838
Grivn
marked this pull request as draft
October 30, 2026 20:51
Grivn
force-pushed
the
codex/open-source-v0.3.4
branch
from
October 30, 2026 20:51
31bf838 to
35d9f83
Compare
Grivn
marked this pull request as ready for review
October 30, 2026 20:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Expose the v0.3.4 AML memory engine through explicit
mnemon memory-agent ...commands while keeping the current native CLI as the default.remember,recall,search, setup/hooks, nativemnemon.db, and the existing Web UI. New records use a separate store and require explicit commands; there is no automatic migration or engine switch.The separate hosted frontend, UI prototypes, internal research plans/history, private evaluation data, and credentials are excluded. No release version or tag is created.
Why
Publish the evaluated implementation as a usable optional memory module before making further algorithm, prompt, or engineering changes. This keeps the evaluation baseline reviewable and gives existing CLI users an unchanged default path.
Provenance and scope
The engine baseline is original evaluation tag
v0.3.4, commitc09e4a8b1a94b0d8e3bc74259fc40a5cb6ecb4cd. All five original evaluation tags remain unchanged.The PR now has seven commits: the five v0.3.0–v0.3.4 source milestones, public-release integration, and Windows test portability. Each milestone includes its implementation and regressions; native CLI compatibility and public test-path corrections are present from the initial import.
The complete sanitized development history is preserved on
archive/aml-v0.3.4-development, pinned at31bf838a2cbda3b7f3437994c3c9e55e5a07c52d: 361 original rebased commits plus two follow-ups, with authors, messages, empty commits and three merges retained. Internal research documents and UI prototypes remain excluded throughout the public history.b2d0b0e05e5185f45b77d8e04f128aecf4bfd4ee92e5a4bfd8fc37280ba2782df8208577768e9696ae4562efbe477ab3e5dc19a4c546033e95c79a7d34b28ab4493d45429b841c812aeb57a84eede77566ec975c95cea9748f0162e206db67ba6dced76802d2c53fe2c95352f6a5cf6f8ff508b7a962ca81ddf9b154bb34e726d91d19a258541500592fc04376c84c007ceafcf43f4060363c61faba8a0113d0c09e4a8b1a94b0d8e3bc74259fc40a5cb6ecb4cd470dbc1975bd5457c215d1c51f76c6ecfb5b2638These are source mappings, not replacement evaluation tags. Rebase and commit consolidation change public SHAs. A future normal merge should retain the seven public commits.
Compared with the previously reviewed head
31bf838a, every final file is byte-identical except the history/provenance section ofdocs/memory-service.md. Production code, prompts, configurations and tests are unchanged. Each of the five milestone engine snapshots also matches its original evaluation sources; the final baseline comparison covers 90 production files/profile, and all 243 pre-existing files undercmd/memory,internal/memory, andwebmatch the public base.This is a first import: approximately 37.3k added lines of source/configuration, 40.7k of tests, and 3.9k of documentation/notices. Later v0.3.6 changes and concise-prompt experiments are not included.
Validation
All seven commits build. Each of the five evaluation milestones passed engine, adapter, native-search and architecture regressions plus the deployment-cap consistency check. The final head passed the full
make testsuite. The existing safety-package architecture assertion is grouped with v0.3.3's introduction of that package, instead of arriving in v0.3.4. Historical development commits in the archive are not individually retested.make test(includinggo vet ./..., native/AML units, architecture and deployment-script suites).go test -race -p 2 ./internal/records ./internal/memoryagent ./internal/aml ./cmd/memory -count=1.go build -o mnemon ..bash scripts/e2e_test.sh: 262 checks passed.npm test --prefix npm/cli: 13 tests passed.npm pack --dry-runverification of all seven packages, including both third-party notices, plus an actual host binary/launcher check. Other-platform binaries in this local packaging probe were layout fixtures; cross-platform builds are checked separately by CI.mnemon.db; an explicit optional write createsrecords.db, leaves the native search response unchanged, and does not insert its record into native search.git diff --check.Windows harness adjustments use a platform-absolute log path and a bounded startup wait with early-exit diagnostics. Shutdown deadlines and request/budget settlement assertions are unchanged. The two affected tests also pass locally under the race detector.
Linux and Windows CI both passed on the seven-commit head
35d9f8315d71ea9c248eb441117b6dfa3512caa6: CI run #387. This includes the deterministic suite, the existing Web workspace, Windows build/command/storage boundaries, PowerShell hooks, and npm tooling. There are no unresolved review threads. This PR is ready for maintainer review and remains unmerged until the maintainer approves.No paid provider evaluation, deployment, tag publication, or npm release is part of this PR. Existing AML scores identify the recorded evaluation baseline, not a new evaluation of this integration.
Checklist
make test)