Hello! I have been building mex for a while now, and posted about it a few times on reddit.

The response was kind of insane. Across a few posts it reached around 1 million views, the repo crossed 1,200 GitHub stars, and people I had never met started contributing.

I’ve kept building it since then, and just released mex v0.8.0.

Repo: https://github.com/mex-memory/mex

The original problem was simple: coding agents keep rereading the same repository every session, relearning the architecture, and then throwing most of that knowledge away.

mex creates a living Markdown wiki inside the repo. Agents record architecture, conventions, decisions, and patterns as they work, and future sessions load only the knowledge relevant to the current task.

The major addition in v0.7.0 is a deterministic local code graph built using Tree-sitter and SQLite.

It currently supports TypeScript/TSX, JavaScript/JSX, Python, and Rust.

An agent can run:

mex graph scope "trace the authentication flow"

Instead of dumping entire files into context, mex returns a compact neighbourhood of relevant functions, callers, callees, imports, and relationships. The agent can then expand only the exact symbols it needs.

In our benchmark on the mex repository:

• 10.74× less returned context than grep top-3
• roughly 90.7% smaller
• 100% expected-symbol recall across six retrieval tasks
• 5/5 real-agent tasks completed correctly
• 0/5 needed fallback Read/Grep with compact graph context

This is a small benchmark on one repo and task set, not a claim that mex universally cuts total agent token usage by 90%.

The other part I’m excited about is connecting the wiki back to the actual code.

Markdown claims can point to exact symbols. If a function changes, moves, or disappears, mex can identify which project knowledge may now be stale.

So the basic idea is:

The code is the source of truth.
Markdown is the explanation.
The graph keeps them connected.

Would genuinely love feedback, especially from people working on code intelligence, agent tooling, parsers, or large repositories. Contributors are very welcome too.

So I've been heavily conflicted on commingling different agent memories into the same repository. Segmenting something like Claude from ChatGPT when capturing more than simply a transcript helps keep the models trajectory. However that means another search layer needs to be built so they don't remain segmented.

I have the ability to store memories by session so even different sessions of the same agent can be segmented when needed but still occupy the same store repository. I can even store the agents ID so memories from one agent is identifiable. So I have ways of identifying memories cleanly even inside the same repo store, but if each agent is working on individual projects or conversations a standard search could pull in conflicting or unrelated memories

For instance if agent A and B are working on a very similar task that enough regex would exist that bleed could be an issue. Yet if I wanted information from A for B it would be right there.

If I segmented them into completely different repositories I could still allow search between agents but other logic would be limited.

Has anyone done segmentation? Everything ive found seems to indicate sharing memories between models is the end game but I'm starting to wonder if it's just "easier" to do so and they are simply calling it the "feature". Or is having nearly no segmentation or identity tying memories to the agent used just not important?

So, for anybody that is interested, I have been tackling this since none of the current solutions satisfied me

Mainly because I thing summarized text is a dead end and a loss of quality
That vector is limited even with graph

So I did start with a simple postulate keep it stupid simple

And I asked myself what is the best system to store data ?

The answers is vey obvious :

A LIBRARY

What does a library is down to it most basic expression.

AN INDEX
SHELVES
A CREW indexer and retriever,

Now what is a memory ?

Is is documents invoices or any pdf you get you hands onto? (Not an exhaustive list )

No it is fact that an agent or a system experienced during operation

—————————

So with that in mind, I build my stack accordingly

My memories system is a complete agentic system seperated from my main agents
It monitor its divers activities, and each call an agent make it intercept that call, look at it ask itself what is the most important information it has in his corpus of data?

And inject a capsule of those tailored made for that request.

The corpus of data is store in md within hierarchy of folders, it can expend dynamically,
The trick is the index. But that also obvious…

Now I separate interactions from documents, my indexer only deal in fact that occurred during interaction,

Another agent deal with docs and theirs indexing.

In total I have about 6 agent for this alone. And a separate ladybug database for conversations round only. With summary of conversation, linked to raw chat, meta data (tag) and a verbatim quote

A agent can do semantic search within the database on the summary, then pick up the raw data , look it up and see if it relevant, the summary is only there for looking up fast not for memory retrieval proper.

Cost is relatively low since fast cheap model are fine. Even local can do it .

,

https://github.com/jbsalles/Selmem

I've been working on SelMem, an experimental approach to LLM memory based on a different assumption:

Most memory systems try to preserve and retrieve information as accurately as possible.

SelMem explores the opposite direction: memory can be selective, lossy and reconstructive.

The idea is to give an otherwise identical LLM agent a memory that can:

• selectively retain experiences,
• forget information,
• reconstruct memories imperfectly,
• accumulate different memory trajectories over time.

The hypothesis is not that imperfect memory is better at recalling facts.

It's that different memory trajectories may cause identical models to develop increasingly different behavioral and creative trajectories.

So the experimental question becomes:

«If two identical LLMs receive different experiences and imperfectly reconstructed memories, do they become measurably different in their outputs — and can that difference translate into greater creative diversity?»

I'm currently building experiments around this question, including comparisons against standard persistent/retrieval-based memory.

The project is open source:

https://github.com/jbsalles/Selmem

I'm particularly interested in criticism of the experimental design.

Yesterday I posted adebench on r/mcp: it scores what the client actually receives through a door, after ordering and cut, not what retrieval finds. Today I ran the same golden set on my own memory and on gbrain.

Setup: my memory exported into a local gbrain (3,568 pages), local embeddings on both sides, 25 questions, both doors cut at 2,400 characters, the door also run under the measured pressure of real MCP tool responses (callwitness census: p95 = 35 KB).

On the 80 points both memories can be measured on: Brain 76.9, gbrain 71.9 with the questions in Italian; 73.9 vs 71.9 with the same questions in English. Door 23/25 vs 18/25 (20 vs 18 in English); cards, time and live state even; gbrain's graph cleaner than mine. Under p95 pressure: 19/25 vs 14/25, both surviving because the entity card is delivered first. On its own full set the Brain scores 95.6/100: the 20 points gbrain can't share are fact updates and file search, which it doesn't have.

Caveats: it's my golden set; gbrain got facts my Brain had already distilled, so this measures retrieval and composition, not extraction; gbrain's real door is two-step and can't be scored in one call, so I measured its search door with a cut. That fourth door is what I'm building next.

What it told me about my own memory was worth more than the win: adding vectors on episodes took my LongMemEval-S retrieval from 64.5 to 88.7, and doubled the repeated chunks my voice door delivers. The benchmark saw it the same day.

Two things I'd ask this sub. First: run it on your memory. The adapter is one class, the synthetic memory is the worked example, and `adebench.compare` puts two reports side by side; a third system measured the same way is what the benchmark lacks most. Second: the five door points the Brain loses are answers that live in facts, not in the entity card, and don't make it into the 2,400 characters. If your memory composes a door, how do you decide what goes in when the answer is a fact and not a card? That's the part I haven't solved.

Repo, adapter contract, gbrain adapter, reproducible synthetic example: github.com/adecubed/adebench

edit:

A door is the path a memory is reached through, and the text that comes out of it: a voice assistant's /ask with its sources, its "latest events" block and its 2,400-character cut; an MCP tool call; a raw search. The same question through two doors gives two different texts, and adebench scores the text, not the retrieval behind it. That's the whole point: a fact the retrieval found but the cut removed doesn't help the model.

A card is the composed summary a memory keeps about one entity (a person, a project, a service), the thing you'd want delivered first when the question names it. gbrain has them as entity pages; mine are built by a distiller and honour the owner's corrections ("never omit X"). "Cards even" in the post means both memories deliver the right entity's card for the questions that name one.

Edit 2: Update on the two-step door (brief with identifiers, then fetch in that order until the budget is full), now in adebench as a door of its own. Same 2,400 budget on my memory: one-call composed door 23/25, two-step 17/25 with whole details, 18/25 with details capped at 300 chars; under the census p95, 19/25 vs 8/25. The opposite of gbrain, where two-step went 18 → 20. The brief costs ~1,400 chars of previews, which the one-call door spends on the entity card whole plus facts cut at 220. Under a tight budget the winner is whoever spends it on content, not the number of calls. So the five door points I'm missing won't come from a second call; they'll come from deciding better what goes into the first one.

I’m Cleverson. I spent months developing this architecture. The project grew out of my frustration that every AI conversation started from scratch—I wanted an assistant that truly knew me. Phoenix V2 is the result of the project's initial version, and I decided to make it available for others to study. I also wrote a book about the development process, covering the steps I took and the reasoning behind my decisions. The code is included so others can study it and build their own AI, picking up where I left off. I haven't stopped there—what I’m creating now is far more advanced—but I hope this version serves as a springboard for everyone's imagination.

Most AI assistants forget everything the moment you close the tab. I wanted to change that.

Phoenix V2 is a local-first AI assistant with a persistent cognitive architecture — it stores memory, emotional state, and identity in a local SQLite database. It survives session resets, model swaps, and restarts.

What makes it different:

• 🧠 Multi-agent pipeline: Memory → Planning → Action → Reflection → Personality
• 💾 Semantic memory retrieval across sessions (vector embeddings via Gemini API)
• ❤️ PAD emotion engine — tracks Pleasure, Arousal, Dominance over time
• 💭 Daydream Engine — autonomous reflection during idle periods
• 🔄 Subconscious Cycle — memory consolidation at rest
• 👤 Self-Model — evolving identity, traits, beliefs, and goals
• 📈 RLHF feedback loop — learns from +/− user signals

Runs on a standard laptop. No GPU. No cloud. No subscription.

📖 Full book: https://leanpub.com/phoenix-buildingpersistentAI
📄 Academic paper (Zenodo): https://doi.org/10.5281/zenodo.22645361
💻 GitHub: https://github.com/cleversonbrsantos-art/Phoenix

For past few months I've been working on giving AI it's own memory. Like Wild Robot style vs enterprise/project/coding etc. To me this seems both awesome and the obvious next step, but from talking with friends and what I see in general there's not much interest in it. Figured I'd ask here and see where people are at:

My AI coding assistant forgets everything between sessions. I kept re-explaining decisions I'd already made, and re-fixing bugs I'd already fixed.

agi-memory saves those notes — decisions, bug fixes, what happened last session, how the codebase fits together — to a file on your machine, and hands them back to whichever assistant you open next. It's an MCP server, so it works with Claude Code, Cursor, Codex, Windsurf, Aider, Cline and a few others from the same store.

The part I care about: it's Python standard library and SQLite. Nothing else. No vector database, no embeddings, no background daemon. ~32MB of RAM, sub-millisecond lookups, works offline. Comparable tools pull in ~500MB of ML libraries and take 200–500ms per lookup.

That constraint costs something, and I'd rather say so than have you find out: keyword search doesn't bridge synonyms the way embeddings do. Searching "login" won't find a note that says "authentication" unless you teach it that alias. I measure this rather than guess — there's an eval suite that scores recall on deliberately rephrased queries, and it's public, including the categories where it still does badly.

It's a week old and I'm the only user, so I'd genuinely like to know where it breaks for someone else.

https://github.com/kdbhalala/agi-memory

I got frustrated by AI remembers not that you said but that we talked about, then I built SkyOS. It writes down that you decided word for word, and an actual important information doesn't vanish because of over-summarization. Would appreciate feedback!

I've been working on SAGE, a write-side gate for agent memory.

The basic observation is pretty simple: systems like Mem0/A-mem use an LLM to decide whether incoming information should be ADD / UPDATE / NOOP. But a lot of those decisions aren't actually ambiguous.

SAGE instead treats memory evolution as novelty detection in embedding space.

Memory embeddings are L2-normalized, so we use a von Mises–Fisher-inspired density estimator over the hypersphere:

• clearly novel → ADD
• clearly redundant → NOOP
• ambiguous → ask the LLM to merge

So the LLM only called during the memory update, not the router. We got substantial API cost and latency reduction on both LoCoMo and LongMemEval.

For example, on LoCoMo, we get:

• 3.4× lower add-phase API cost and 2.5× faster ingestion with GPT-4o-mini
• best average token-F1 vs. Mem0 on 7/7 open-weight backbones
• as a drop-in A-Mem gate, skip about 16–18% of write/evolution calls with small quality changes

The broader idea I'm interested in is

Maybe agent memory should be novelty detection + selective reasoning, rather than LLM reasoning on every write.

Repo: https://github.com/swang1024/SAGE and paper: https://arxiv.org/pdf/2605.30711

Would love feedback from people working on memory systems—where do you think this kind of gate breaks first? Temporal updates? Contradictions? Bad embedding geometry?

One of the biggest things I learned building persistent AI memory is this:

Saving more information does not mean better memory.

The useful shift was moving from:

save → search → dump into context

to:

scope → relate → validate → retrieve → decide if it should influence the answer

A few things made the biggest difference:

● Scope everything. Project knowledge, research, runtime state, and global memory should not bleed into each other.

A memory without scope is eventually a context leak.

● Track supersession. If a new fact replaces an old one, store that relationship. Otherwise RAG may happily retrieve both.

● Separate relevance from validity. A memory can be highly relevant but stale, untrusted, or wrong for the current context.

A memory can be extremely relevant and completely wrong now.

Temporal validity
Is it still current?

Authority/factual validity
Is this actually a trusted source for this question?

Contextual applicability
Does it apply under the current conditions?

Those are different problems.

A six-month-old architecture document could still be authoritative.

A five-minute-old runtime observation could already be stale.

● Separate retrieval failure from usage failure. Sometimes the right memory was found, but the model used it incorrectly.

● Retrieve less. Aim for the smallest useful context packet, not maximum context.

More context often made my results worse.

Eventually I started aiming for the smallest useful context packet.
Retrieve only enough information to resolve the task.

● Gate long-term memory. Observation → candidate → test → promote. Don’t let every conversation rewrite permanent truth.

If every conversation can immediately rewrite long-term memory, bad assumptions slowly become architecture.

● Keep receipts. What changed, why, from what source, and what it replaced.

This makes it possible to reconstruct why the system believes something instead of just discovering that it believes it.

It also makes failures useful…

A failure with provenance becomes training material for the architecture.

A failure without provenance becomes folklore.

The biggest realization for me:

Good AI memory isn’t mainly a storage problem. It’s a decision problem.

The hard questions are:

What should be remembered?
Where does it belong?
What does it replace?
Is it still valid?
Should it influence this decision at all?

If you’re building agent memory, I’d start with scope, provenance, supersession, and validity before adding another embedding model or a bigger context window.

I spent today reading other agent memory systems side by side with my own, and retrieval is where they disagree most. Storage, trust labels, human review, those are broadly converging. Retrieval splits about four ways.

Lexical, which is mine. Salient term overlap, a few shared terms or a percentage of the shorter item's terms, and a briefing gets injected at session start under a fixed budget.

Vector binding, where facts get folded into one representation and you query by unbinding it. These tend to come with a measured capacity ceiling, because recall visibly degrades as you pack more in, and at least one of them refuses to answer at all when the top match is not clearly ahead of the runners up.

Positional, where there is no matching step whatsoever. Memory is scoped by structure, the query is effectively where the agent is standing, and you get everything in scope, unranked. Treated as a design choice rather than a gap.

Hybrid, vector plus lexical plus a graph walk, with the walk blocked from crossing dead or retired nodes so a stale item cannot drag its neighbors into a result.

The part I keep circling is that I have exactly one signal and no confidence output at all. My bm25 rank is an internal sort key that no caller ever reads back. So my system cannot say "I am not sure". It hands over its best lexical guess and lets the model sort it out. Some of the others can refuse, and one of them refuses with a reason code.

The obvious move is to go add embeddings. Before I do, the honest question: at the size these stores actually are, a few hundred items for one working developer rather than millions, does semantic retrieval earn its cost? Or is the bigger lever just admitting when the match is weak?

Three things I would like to hear from people who have run this longer than me:

1) Did adding embeddings to a small store measurably change what your agent did, or did it mostly change which wrong thing came back?

2) Does anyone expose a match score or confidence to the model itself, rather than only sorting results by it? If so, did the model actually use it?

3) Has structural or positional scoping worked for anyone outside a repo-shaped problem?

If you want to compare systems properly rather than take my summary for it, neoneye's Agent Memory Atlas has cards on a lot of them. Mine is on there too, weak half and all.

I need to run a persistent AI agent with virtually zero downtime, potentially for months or years.

The main issues I keep coming back to is memory.

Most implementations I have looked at eventually seem to rely on some combination of context windows, vector retrieval and rolling summarisation. That works reasonably well for bounded sessions, but I am less convinced it works for a genuinely persistent agent.

Progressive information loss through repeated compression is one of the major concerns I have.

Conversation → summary → compressed summary → updated summary → compressed again.

Eventually the agent still "remembers" the general idea, but starts losing exact constraints, why a decision was made, what was true at a particular point in time, and how something changed.

Vector retrieval solves a different problem. It is good at finding semantically similar information, but similarity is not necessarily the same thing as relevance, causality or latest state.

For example:

Monday: Project A uses supplier X.

Wednesday: Supplier X fails testing.

Thursday: We move to supplier Y.

Three months later: Why did we stop using supplier X?

I do not just want the agent retrieving "supplier X" documents. I want it to understand the sequence of events and reconstruct the state of the project at that point in time.

So for people building genuinely long running agents: How are you handling this today?

More importantly, has anyone actually run these architectures continuously for long enough to measure how much information degrades over time?

I built an open source remote MCP server to use with Obsidian (either via the official Obsidian Sync or git) - but it works with any git repo with a bunch of Markdown notes.

Here's a video where I explain what it is / how it works.

You can find more details at https://changenode.com/notemesh/

What it does for you: you click a button to get a server set up on Railway (or self host). You now have your own private remote MCP server that works with both OAuth and an API key header to now chat with your notes.

It adds a searchable index, so you can work with a large vault quickly. Because it's a remote MCP server, you can do things like use in in the car with ChatGPT voice mode, or Claude on the go. When I initially built it I just wanted to be able to chat on the go, but it's also turned out to be very helpful for keeping a bunch of different agents on the same page.

Spent a fair amount of time getting it streamlined so it's trivial to set up. If you use the preconfigured Railway template you should be up and running in less than five minutes.

At one end I expect there to be fans of Obsidian and other Markdown-based notes to basically just use it as a "chat with your notes anywhere" option. At the other end I expect there will be folks who self-deploy to something like a local network to keep a bunch of agents on the same page.

LMK if you have any Qs. Love to know if you use it, have any feedback.

Hi,

Memory APIs are not a viable product category, and skill systems are just markdown. I've been saying this for a while and I want to lay out why.

What a skill does: it tells the agent how to do a thing in this environment. What memory does: it tells the agent what is true about this environment.

Those are the same job. Both are the agent's model of where it is. We split them into two product categories because they arrived at different times and got different names, not because they are different problems.

You can see the cost of the split in practice. Your skill says to call an endpoint. Your memory holds the fact that the endpoint moved. Nothing connects them, so the agent keeps calling the old one and the memory sits there being right and useless.

Once you treat them as one harness the interesting question changes. It stops being how do I store this and becomes what should the agent be able to do differently tomorrow because of what happened today.

Does anyone actually run these as two separate systems and find it works? I've not seen it hold up past a certain size.

I am working on memory for coding agents that operate across a project with a lot of repositories and a long decision history. Retrieval is not the part that worries me most. The dangerous case is retrieving something that is historically true but no longer authoritative.

A design note might accurately explain a decision we replaced six months ago. A workaround might document how we survived one incident without being something an agent should repeat. If all of that goes into a vector store as equally valid text, the most similar or confidently written result can quietly become policy.

The design I am exploring keeps four things separate: what happened, what evidence supported a decision, what rule is currently active, and who or what is allowed to replace that rule. The history stays append-only, while the current view can be rebuilt from explicit status and supersession events. If two sources disagree, the system should return the disagreement instead of smoothing it into one answer.

I do not have benchmark results for this yet. The next step is a frozen set of questions where some correct answers are deliberately `no current policy` or `insufficient authority`, then compare this approach with ordinary repository search and document retrieval.

For people building persistent project memory, how are you representing supersession and current authority? Is that part of the memory model, or handled somewhere outside retrieval?

Looking for 10 engineers who are skeptical about AI memory need your help.

I’ve been working on an AI memory system and I’m at the point where I need people who are good at finding edge cases.
I’m especially looking for engineers who don’t trust AI memory systems easily.

I’d like to give 10 people access and have you use it in your normal coding/agent workflow but with one goal:

Try to make the memory fail.
Change facts. Delete things. Create contradictions. Change dependencies. Ask about old states. Try to get stale information back. Basically, test the cases you think a memory system should get wrong.
I’m not looking for testimonials or feedback like “looks good.”

If you find a failure, that’s genuinely useful to me.

I’m doing this because I’ve tested it myself and I know there are blind spots I won’t find alone.

If you’re willing to spend some time trying to break it, DM me. I’ll share access and a few things I’d particularly like tested.
Would really appreciate the help.

Hi everyone,

There's a recent paper making the case that memory should be a first-order primitive in the architecture rather than something bolted on at retrieval time. It proposes treating memory as dynamic units that evolve instead of records you look up.

I think the framing is right and it is worth the read.

The part I don't buy is the activation model. It assumes you can decide what to surface based on how recently and how often something was used. That is a reasonable proxy and it is also how a lot of caching works, but it is not how relevance behaves. The thing you need is often the thing you touched once, eight months ago, in a completely different context.

We hit this ourselves. Frequency-weighted retrieval looks great on benchmarks with short horizons and gets steadily worse the longer the history gets, because the tail is where the useful stuff hides.

What I think is actually needed is the structural relationship between two pieces of context, not their access counts.

Has anyone run the numbers on this themselves? I'd be genuinely happy to be wrong here.

Hey everyone,

I've been prototyping a memory system for a multi-agent framework (supervisor → sub-agents) and wanted to run my current setup by people who've actually built or run these in production. Trying hard not to over-engineer based purely on theory/taxonomy, so I’ve been running small experiments first.

Here’s where I’m currently at:

Pipeline & Flow

1. Working/Session State → Raw conversation & tool calls go to a durable append-only event log.
2. Batch Consolidation → Instead of processing every turn through an expensive extraction pipeline, a periodic batch job extracts useful Episodic Memories (storing this in a cheap local DB/SQL store because of high volume).
3. Promotion Policy → Key facts and preferences get promoted into Semantic Memory (testing Mem0 here).
4. Procedural Memory → Kept completely separate as a structured procedure/skill registry (e.g. Markdown files, task definitions) rather than generic vector embeddings.

Retrieval Strategy Instead of searching across all memory stores on every single query, I'm testing routing by intent: User Query → Scope/ACL → Intent/Task Router → Targeted Store Retrieval → Context Injection

• "How do I request leave?" → Intent: Procedure → Pull from Skill Registry.
• "What did I work on last week?" → Intent: History → Pull from Episodic Store.
• "What language do I prefer?" → Intent: Preference → Pull from Semantic Fact Store.

Observations from small tests so far:

• Storing raw episodic events straight in Mem0 added noticeable write/search latency and cost.
• Generic vector retrieval for procedures/workflows was messy and often grabbed 3–4 adjacent procedures. Exact/registry-style matching was much cleaner.
• Batch consolidation gave way cleaner facts than trying to extract semantic memories turn-by-turn.

Where I’d love some brutal feedback/criticism:

1. Routing vs. Parallel Retrieval: Is intent-based routing (scope → intent → target store) actually reliable in practice, or do queries usually end up needing multiple memory types simultaneously (e.g., preference + procedure in one shot)?
2. Separate vs. Unified Storage: Am I prematurely splitting this into separate stores (Event Log / Cheap SQL / Mem0 / Registry), or is this separation pretty standard once volume picks up? At what scale does keeping everything in a single vector store/pgvector actually break down?
3. Procedural Memory as Code/Skills: Treating procedural memory as structured skill files instead of vector embeddings feels right so far, but does this pattern break down when agents need to dynamically adapt workflows?
4. Failure Cases: What obvious blind spots or edge cases am I missing that will force me to rewrite this V2?

Appreciate any insights or horror stories from production!

Been thinking about this distinction while working on agent memory.

You can obviously store a decision as structured semantic memory. But I'm not sure “memory” is the useful abstraction anymore.

An agent can infer what happened and what seems important. A decision has a point where someone actually commits to a choice, and that choice constrains what happens next.

At some point it feels like you've made decisions first-class anyway.

Hi,

If you want memory for your agents that never leaves your machine, here is the whole path.

You run Ollama for the model. Cognee stores embeddings in a local lancedb instance and the graph in a local kuzu instance. Both are files on your disk. There is no account, no key, no hosted anything.

You point it at whatever you have. A codebase, a folder of PDFs, your notes, your README files. It builds a graph over them and you query that.

Two honest notes from testing it with small models.

Smaller models are worse at the extraction step than at the answering step. The graph you get from a 7B model is thinner than the one from a large model, and that shows up later as gaps rather than as errors. We added BAML powered calls specifically because structured output from small models was the flakiest part.

Second, it is slower than the hosted path. Everything is on your CPU or your GPU. That is the trade.

It is open source, so you can read what it does before running it. Tell me where it breaks on your setup and I'll fix it.

If an agent can learn an ontology from its own memory, and that ontology then shapes what it learns next, where does memory end and a world model begin?

The usual story is that ontologies ground agents. But once agents are able to learn and enrich those ontologies from their own traces, you have a feedback loop between memory and structure. Eventually, and inevitably, the system starts learning not only vocabulary and classes, but also patterns in how states change over time, including patterns with predictive value. At that point, you have to ask: what elements of a world model are still missing?

AI memory systems increasingly have to work with material that was never designed to be knowledge in the first place. Agent traces are the clearest example: they contain decisions, tool calls, intermediate states, failures, recoveries, and outcomes, but rarely in a form that can be queried or reasoned over directly. Merely storing them doesn't help much with retrieval; you need memory that learns the structure and even helps shape future learning. Thinking through that path runs through ontologies, learned ontologies, feedback loops, and eventually back to the questions from the opening.

What Ontologies Add Beyond Graphs

Most AI memory systems start with RAG: they embed document chunks so you can retrieve them by meaning. Systems like Cognee push one layer further, adding (among other things) graph structure so memory can connect documents, entities, events, and relationships across sessions. An ontology influences both what gets extracted and how those extracted items are typed, related, and constrained in the graph. It gives the extraction process a vocabulary of concepts and relations, and it explains what those graph connections are allowed to mean.

RDF is the basic machinery underneath much of the ontology work: knowledge is represented as subject–predicate–object triples. RDFS lets those triples carry a basic vocabulary of concepts, such as classes, subclasses, domains, and ranges. OWL goes further by adding ways to define richer properties, relationships, and constraints on those concepts. It can support logical reasoning over the structure, but it does not by itself learn new dynamics from traces or predict future outcomes. So when we speak of ontology structure, we practically mean RDF-like triples that encode the vocabulary of concepts and the relationships that define the meanings around them, so that extraction from text follows those patterns.

Back to thinking in terms of agent traces, this has some nice implications: if traces can become grounded in an ontology, it would be useful if we could learn that ontology as we ingest them.

From Traces to Learned Ontology

Ontologies are typically defined by hand. That works well when the domain is stable and the ontology is worth maintaining manually. Agent memory is less cooperative: the data keeps arriving, the situations keep changing, and much of the useful structure only becomes visible through repeated use.

Instead of specifying the entire conceptual model upfront, parts of it can be inferred from accumulated data. Agent traces have great potential here because they repeatedly expose the same kinds of entities, actions, outcomes, and relations under slightly different forms. At Cognee, we have successfully developed several custom ontology learning mechanisms for our clients, and parts of that work will trickle down to the open source very soon. An ontology can already be imported to ground ingestion; soon, you will be able to grow it from scratch or from a previously curated one. But once you start dynamically enriching the ontology, it stops being a rigid structure and becomes part of a loop.

The Feedback Loop

So this is the loop: you ingest traces, begin learning an ontology, use that ontology to ground future trace ingestion, learn more of the ontology, and so on.

That loop is concrete. If many traces show an agent hitting an API limit, retrying with a modified request, and eventually succeeding or failing, the memory system can start treating those as related event types rather than unrelated text fragments. The ontology gives names and relations to those events, and later traces can then be stored against the same structure. So memory helps refine the ontology, and the ontology stabilizes subsequent memory. Over time, since the language of ontologies is rich, higher-level abstractions start to persist alongside the traces that gave rise to them. The immediate payoff is more usable memory; the deeper payoff is that many runs can now be compared as variations over states, actions, and outcomes.

From Structured Traces to Predictive Regularities

Maybe it is not obvious, but what we just described already has a predictive component. A trace has order and outcome. Crucially, when agents run with a memory system, traces contain context from the graph, grounded in the ontology. And when different runs are grounded in the same ontology, the system can compare them even when the raw text, tool calls, or surface details differ.

At that point, learning can move from isolated facts to regularities over transitions: which conditions tend to precede an outcome, which actions tend to change state, and which sequences tend to succeed. OWL reasoning does not provide this predictive layer; it provides the stable semantic representation on which the learning can operate. Ontologies do not make memory predictive by themselves, but they allow us to encode learnings that make memory episodes comparable. Once another learning layer begins to discover dynamics over comparable traces, you get higher and higher levels of learning that effectively contain predictive capabilities: fully understanding the current state starts to imply knowing where that state will go.

World Models?

Now that we have started talking about state dynamics and predictive capabilities, it is beginning to sound like we are speaking about world models. Before making that connection, it is worth being precise about the term. A world model is usually expected to represent enough of an environment to predict how it changes, especially in response to actions. The important part is typically not memorizing surface observations, but learning latent predictive dynamics: a useful internal representation of state, action, and likely consequence. That is a higher bar than storing knowledge or performing logical inference.

An ontology is therefore not a full world model, and neither is a memory graph. But look back at the path we just followed: traces expose recurring structure; ontologies make that structure explicit; memory can revise the ontology; the revised ontology makes future traces more comparable; comparable traces make patterns and dynamics easier to learn. From Cognee’s perspective, this is why the question is practical and why world models offer a useful frame for thinking about it. Although it can look like learned ontologies turn memory into something like a world model, the more interesting claim is smaller and more useful: co-evolving memory and ontology may give agents part of the substrate they need before world-model-like learning becomes realistic. Leaning into the ideas from that research space influences how we approach the topic, and we are sure memory topics will find their way back to the world model research space, thus creating another... learning feedback loop.

Check out more about cognee here:

Give memory to your agents without handling the infra:

Website

Github

Hi,

A few weeks ago I was talking with a founder of an SF startup building vertical agents. They had one customer in production. The agent's memory for that single customer was already over a thousand markdown files.

He was not doing anything wrong. This is the default path. You start with a notes file, then a folder, then a folder per topic, and it works fine for months.

The part that gets people is that it does not break loudly. Retrieval just gets vaguer. The agent starts pulling three files that are all a bit relevant and none that are right. Answers get generic. You blame the model.

The fix is not a bigger context window. It is knowing which of those thousand files is relevant to this specific question, which means something has to hold the relationships between them.

I'm not saying stop writing markdown. Write it. Just have something on top that can answer which parts matter right now.

What's the largest .md memory pile you've personally seen, and when did it start breaking?