engram
Memory for AI assistants, inspired by the brain.
64%vs 61%
Found the right past message more often than the best standard method, on a public test it was never tuned on.
The technical write-up
Vector stores fail on agent memory in two specific ways: they cannot separate confusable near-duplicates, and they cannot answer a question whose answer was never in any single stored item. engram implements the biological account of both — dentate pattern separation, CA3 attractor completion, a conjunctive binding code, context reinstatement and a neocortical schema store built by prioritised replay. It is measured on LongMemEval-S with the official retrieval protocol and on LoCoMo, which no decision ever saw, against BM25, dense and tuned hybrid retrieval across three encoders, with paired intervals. It ties the strongest hybrid on the first and leads on the second. The gain comes from one mechanism, and the rest do nothing measurable on either benchmark; the write-up says which, and what v0.1 got wrong.
- 0.993 → 0.907
- CA3 completion, 400 → 4,000 stored patterns
- 0.114 → 0.021
- code overlap under neurogenesis
- LongMemEval-S · LoCoMo · 3 encoders · paired CIs · Python
- 2026 · Memory and consolidation
- episode → dense encoder → dentate gyrus → CA3 → prioritised replay → schema storeexpansion · sparse code · traces
- episode → conjunction indexterms · pairs
- query → conjunction indexweighted intersection
- dentate gyrus → lateral inhibition
- schema store → eviction on promotion
The failure it starts from
Take 125 incident reports of the form “On {day} the {service} service suffered {fault} during the rollout.” Mean pairwise cosine under MiniLM is 0.529 and the maximum is 0.998 — to a sentence encoder these are nearly the same sentence, because they are. Ask “What went wrong with the {service} service on {day}?” and dense cosine finds the exact report 4.0% of the time.
The obvious fix is pattern separation: expand the embedding into a much larger, sparse population, as the dentate gyrus does. It decorrelates as advertised — inputs at cosine 0.77 produce codes overlapping at 0.220 — and it does not fix retrieval: 4.8% on the same set. Expansion is injective: it can only separate what the input already distinguishes.
The finding that reframes this is a different run: with a lexical hashing encoder, dense cosine is already at the ceiling. The slot confusion is not a property of pooled vectors. It is what a semantic sentence encoder does to lexical detail, and the conjunctive pathway is how engram gets that detail back.
Conjunctive coding
CA3’s recurrent network codes for combinations — this object, in this place, at this time — not for the elements separately. A conjunction is a distinct addressable unit, so auth-on-Monday is a different memory address from auth-on-Tuesday however similar the surface forms.
The implementation is a sparse binding code: each content term is a unit, and so is each unordered pair of content terms fewer than four apart — the unordered-window feature of Metzler & Croft’s sequential dependence model. Units hash into a 2³² space and are weighted by their own document frequency in the store, so a rare conjunction of two common words counts as rare. One sparse matrix-vector product scores every episode.
On the incident reports it finds the right service and day every time, and the exact report 20% of the time with MRR 0.457 — both the ceiling, since the queries name two of three attributes and five reports are genuinely tied.
LongMemEval-S, held out
| BM25, official tokens | 0.513 |
|---|---|
| engram v0.1 | 0.647 |
| BM25 | 0.673 |
| dense | 0.691 |
| RRF hybrid | 0.743 |
| z-score hybrid | 0.764 |
| engram | 0.767 |
| metric | BM25 official | BM25 | dense | RRF hybrid | z-score hybrid | v0.1 | engram |
|---|---|---|---|---|---|---|---|
| session R@5 | 0.513 | 0.673 | 0.691 | 0.743 | 0.764 | 0.647 | 0.767 |
| session NDCG@5 | 0.528 | 0.659 | 0.611 | 0.694 | 0.721 | 0.623 | 0.699 |
| session R@10 | 0.630 | 0.758 | 0.825 | 0.875 | 0.878 | 0.799 | 0.869 |
| turn R@10 | 0.577 | 0.703 | 0.703 | 0.767 | 0.802 | 0.714 | 0.802 |
| turn NDCG@10 | 0.552 | 0.675 | 0.622 | 0.706 | 0.735 | 0.647 | 0.716 |
LongMemEval-S cleaned, official retrieval protocol, 343 test questions after a hash-defined 20% dev split carried every tuned choice. all-MiniLM-L6-v2. R@k is recall_all@k.
engram ties the strongest baseline — a BM25 + dense z-score hybrid whose weight was tuned on the same dev split — on session recall@5 (+0.003, 95% interval −0.023 to +0.029) and trails it on turn NDCG@10 (−0.019, −0.034 to −0.005). It beats BM25 by +0.093 and dense retrieval by +0.076, and v0.1 by +0.120. Frozen on MiniLM’s dev split and applied unchanged, the same configuration gives the same picture with bge-small and e5-small.
The scorer and the BM25 baseline are reimplementations of the reference harness; a script downloads the originals at run time and confirms they agree on 300 random rankings.
LoCoMo, never tuned on
| MiniLM | z-score hybrid 0.614 → engram 0.640 (+0.025) |
|---|---|
| bge-small | RRF hybrid 0.644 → engram 0.680 (+0.036) |
| e5-small | dense 0.690 → engram 0.693 (+0.003) |
| encoder | BM25 | dense | RRF hybrid | z-score hybrid | engram |
|---|---|---|---|---|---|
| MiniLM | 0.543 | 0.507 | 0.586 | 0.614 | 0.640 |
| bge-small | 0.543 | 0.635 | 0.644 | 0.638 | 0.680 |
| e5-small | 0.543 | 0.690 | 0.656 | 0.636 | 0.693 |
Recall@10, 10 conversations, 5,882 turns, 1,535 questions in categories 1–4. Run once, with the configuration frozen on LongMemEval-S dev.
The LongMemEval test split was scored more than once during development, so this is the clean check. engram is best with MiniLM (+0.025 over the z-score hybrid, conversation-cluster interval +0.005 to +0.051) and with bge-small (+0.036 over RRF), and ties dense retrieval with e5-small (+0.003).
What carries it
| no dense pathway | -0.122 (-0.066) |
|---|---|
| no conjunctive pathway | -0.076 (-0.093) |
| rank fusion | -0.023 (-0.018) |
| all pairs (v0.1) | -0.009 (-0.002) |
| terms only | 0.003 (0.003) |
| BM25-weighted units | 0.015 (-0.005) |
| no reinstatement | 0 (0) |
| after 40 cycles of sleep | 0 (-0.001) |
Without the conjunctive pathway, engram is the dense baseline to three decimals. Keeping its terms and dropping its pairs changes nothing measurable on LongMemEval-S (+0.003): on conversational memory it is a lexical channel, restoring the specificity the sentence encoder removed. BM25 weighting of units scored higher on test than the frozen choice (0.781); the dev split preferred IDF-cosine, and the dev decision stands.
Reinstatement and consolidation move nothing here. LongMemEval and LoCoMo ask for evidence turns; neither asks what usually happens or what came next, which is what schemas and reinstatement are for.
The synthetic benchmark
| conjunctive binding | -0.665 (-0.09) |
|---|---|
| temporal reinstatement | -0.315 (-0.121) |
| pairs (terms only) | -0.035 (0.016) |
| schema store | -0.005 (-0.003) |
| sleep / consolidation | -0.005 (-0.003) |
| dense semantic | 0.002 (-0.082) |
| theta context | 0 (0) |
| neurogenesis | 0 (0) |
| setting | recency | BM25 | flat vector | hybrid RAG | v0.1 | engram |
|---|---|---|---|---|---|---|
| token-literal | 0.006 | 0.664 | 0.083 | 0.270 | 0.652 | 0.987 |
| paraphrased | 0.006 | 0.119 | 0.057 | 0.160 | 0.201 | 0.282 |
Overall top-1, 5 seeds, 2,314 queries. The top result must be the right episode; abstraction, which has no single episode, is scored by answer.
Four tasks, each built to isolate one mechanism: episodic, interference, abstraction and temporal. On literal queries v0.2 is at ceiling, so that setting is now a regression check rather than evidence. Reinstatement is the only mechanism that answers sequence questions — without it the temporal task falls from 0.981 to 0.009 — and every retrieval baseline scores about zero there, because it returns the anchor.
What v0.1 got wrong
- Answer-level scoring
- A query counted as solved when the top result mentioned the right project. With a 70% home-project prior a wrong episode carries it about 40% of the time, and on the temporal task the anchor itself carries it in a third of queries — exactly the 0.333 the baselines were reported at. Scored by key, v0.1’s literal-query lead over BM25 is a tie.
- The binder’s truncation
- It kept the alphabetically first 24 terms, documented as “the earliest, the topical ones”, and dropped one-character tokens, so “week 7” lost its 7 while “week 14” kept its 14.
- Passive decay
- It deleted exactly the unconsolidated traces the rest of the system promises never to delete. No benchmark wrote enough to trigger it.
- Tables no script produced
- The incident, dentate and CA3 tables did not reproduce from the committed scripts, and CA3 completion is not flat in store size: 0.993 with 400 stored patterns, 0.907 with 4,000.
Things that did not work
- The schema store with a lexical encoder
- With the hashing encoder, removing the schema store raises abstraction from 0.287 to 0.613. Lexical centroids merge different projects, and the exemplar a schema returns need not name the modal one.
- Lateral inhibition in the dentate gyrus
- Random sparse projections are already near-orthogonal: Gram off-diagonal 0.020 against activation σ 1.00, about 196× below the selection margin. It perturbs about 12% of selected units and buys no separation (mean overlap 0.1277 → 0.1284).
- Eviction coupled to promotion
- Releasing a trace as soon as a schema absorbed it destroyed episodic detail the moment generalisation began — catastrophic forgetting, produced by the architecture that exists to prevent it. Eviction is capacity-driven, and an unconsolidated trace is never dropped.
- DG/CA3 as a ranking pathway
- 0.048 on the incident reports, no better than cosine. Kept for what it is good at: storage separation and fragment completion.