AI brain
A personal AI with lifelong memory. I tell it things; it remembers them, forever, cheaply. Journaling for lazy people. I've used it daily for three months — about 350 memories so far.
Long-lived agents are still an "unsolved" problem, but mostly because the common answer is stuffing ever more context into every call. This goes the other way: retrieve a small, relevant slate per turn, and let an async process do the actual remembering. It's a cost optimization as much as an architecture — and it may be a transitional one. If context keeps getting cheaper, brains like this stop being necessary. Today they aren't.
Shape
Two halves that never block each other: a reader on the hot path of every chat turn, and a bookkeeper that runs after the response stream closes and costs zero latency.
Hot path — one turn
- Open the response stream immediately, then do everything else behind it. Context (folders, entities, self-model, rollups) loads in parallel, and every load is failure-tolerant — any piece can come back empty without taking down the turn.
- Embed the message twice: once raw, once blended with the previous assistant reply. The blend is why terse follow-ups like "what did she say about it?" still retrieve — the previous turn carries the topic the pronoun points at.
- Capture verbatim before any interpretation. The journal entry is immutable and is the source of truth; everything else in the system is an index over it.
- Retrieve with five parallel legs, not one vector search: folder-routed vector search, a global safety net (so a routing error can't blind it), a keyword/full-text leg for rare exact tokens cosine misses, a contextual leg using the blended embedding, and an entity-linked leg. Results merge by best score with a per-folder dominance cap, so one hot topic can't eat the whole slate. If the message names a known entity, that entity's memories jump the queue — which is what makes "tell me about Yasmin" surface memories that only ever called her "the ex".
- Expand one hop out through the knowledge graph from the top seeds, with a per-relation policy: contradicts pulls the other side and flags both so the model reconciles instead of silently picking one; evidence_for pulls the grounding; caused pulls the why; elaborates just tags the seed with a count the model can drill into via tools; supersedes pulls nothing — stale memories are already penalized at match time. Pulled memories score at half their seed's score: they supplement, never outrank.
- Assemble the prompt in two blocks — a stable cached block (persona, memory rules, rollups, folder and entity maps) and a volatile block (current time, the retrieved slate) — with the cache breakpoint on the stable block only. The tool loop streams up to 8 rounds over ~19 tools, advancing a cache breakpoint each round so a long search-then-add chain only re-bills the new suffix.
One bug worth confessing: user and assistant rows were originally saved in parallel, and the assistant row often won the race by microseconds. The next turn would then read the reply before the message that caused it, and the model hallucinated from the memory store instead of the conversation. Sequential saves; worth the ~30ms.
Cold path — the bookkeeper
After the stream closes, one small-model call extracts every atomic fact from the entry — tiny ones included. The design bet is that a half-life field (1 to 3650 days) encodes importance, so omission is never the right call. Then: embed and dedup (≥0.9 cosine blocks, including within the batch); the 0.8–0.9 near-duplicate band gets a second tiny LLM call to decide the relation — supersedes, contradicts, elaborates, or nothing — and writes the graph edges. That's the key move: the live agent almost never bothers to create edges on its own, so the bookkeeper builds the graph the hot path later walks. It also soft-archives fully decayed memories (rows survive forever) and refreshes weekly/monthly rollups, rate-limited so a chatty session doesn't re-summarize twenty times. Everything logs-and-swallows — a bookkeeping failure can't break chat.
The decisions, compressed
- Capture verbatim first, interpret later — the journal is immutable ground truth.
- Extraction is async — the user never waits on bookkeeping.
- Retrieval is multi-leg — cosine alone misses rare tokens, pronouns and aliases; each leg patches a specific failure mode.
- The graph is built by the bookkeeper and read by the agent — agents don't reliably do relationship bookkeeping on their own.
- Decay replaces deletion — half-lives plus access boosts earn persistence without explicit signals, and nothing is ever hard-deleted.
- Every subsystem degrades to empty — no single failure takes down a turn.
What I can't claim
That it works well. The quality of a memory system like this is close to unverifiable — there's no ground truth for "the right memories surfaced," and I suspect that's the real reason nobody has convincingly "done" lifelong agent memory yet. My only eval is using it every day and noticing when it's wrong. Known gaps: graph expansion is a single hop, not proper traversal, and some decisions I'd redo. It's a working system with honest limits, not a solved problem.
Code goes public once I've cleaned it up — link will appear here and on the front page.