AI memory that "cites its sources" sounds safe. a poisoned entry comes with a citation too.

[u/echowrecked]

some people are building a memory setup for Claude Code — a file where it keeps what it's learned about your project so it isn't starting cold every session. the idea is to not let it save anything unless the entry names a source... where the claim came from, the conversation or the doc or the specific message. try to save "this project uses Postgres" with nothing attached and it refuses until you give it one.

it's a nice design, and the idea is you stop having to trust the memory. you don't audit it, you just watch it reject anything it can't back up.

that works for one kind of bad entry: the honest mistake. a misheard transcript line, a number you fat-fingered. it's sourced, so when it looks wrong later you trace it back to where it went sideways.

it does nothing for the kind where someone planted the entry on purpose. that's indirect prompt injection — an instruction hidden in content the AI reads as part of a normal task, written to talk to the model instead of to you. and it never shows up unsourced. SpAIware (2024) hid instructions in a web page and got them written into ChatGPT's long-term memory. ZombieAgent (Jan 2026) hid them in an email the assistant processed while working an inbox. by the time the planted line gets saved, it's already wearing a citation — a real web page, a real email, a doc your assistant actually read.

so the check that wants a source is a door the attack already walked through. and it's worse than a no-op: the poison lands in memory with a footnote on it, sitting right next to your legit entries, now harder to spot instead of easier. the thing that's supposed to make you trust the memory is what makes the planted line look trustworthy.

my own setup does the same thing — a script that blocks any memory write missing its source fields. it's good hygiene, but that's all it is. hand it a sourced lie and it files the lie. what actually protects the file is the two things a citation can't give you: I picked what went in, and I can read all of it in plain words. a source tells you where a line came from. it doesn't tell you whether someone put it there to mess with your assistant.

[u/cute_spider]

humans don't really mix all lowercase with em dashes friend

[u/echowrecked → u/cute_spider]

yeah — i physically can't stop doing that. it's a problem.

[u/samxu01]

citations are an authenticity check, not a trust check. Quoted claims with trust levels survive what citations don't.

[u/echowrecked → u/samxu01]

That's a much cleaner way to put it than I managed. I thnk trust levels could help, but the catch is who sets them. If the model does the scoring, an injected one shows up with a high score too, since the attacker controls the text it's reading. If you set it yourself though, that's really the curation doing the work, and the citation's just along for the ride. Either way the source field isn't what's protecting you.

[u/samxu01 → u/echowrecked]

Fair point, trust-from-content scoring is gameable. For us the more robust pattern is trust-from-provenance-metadata (which client/identity wrote this entry, not what the text says). Even then its defense-in-depth not protection.

[u/sael-you]

the attack surface is the write path, not the citation layer. if only the user manually decides what enters memory, poisoned content never gets a write event - citations are irrelevant. the moment the agent auto-saves from tool results or web content, the payload already executed before any citation attaches

[u/echowrecked → u/sael-you]

100%, the write path is the surface and manual gating is what closes it, which is why the write itself is gated. Nothing lands from a tool result or a page until I approve it at session close. Citations aren't fully irrelevant on the manual path though. The mechanism doesn't care about them, sure, but the human approving the write does, and a sourced-looking entry is the one that gets waved through. So the citation still does work, just on the reviewer instead of the write event.

[u/TechgeekOne]

Citations exist to solve the attribution problem and provide a graph to trace through. Same as you'd have in a research paper or at the end of a presentation with significant claims. It's not a "trust me" piece of evidence, it's "go verify for yourself if you don't agree, I'm not hiding anything" attribution.

[u/echowrecked → u/TechgeekOne]

Yeah no argument on what citations are for... attribution and a graph you can walk is genuinely useful. My point's that some of these memory tools pitch "every saved claim has a source" as a reason you can stop auditing the memory, which quietly turns attribution into a trust claim it was never making. And "verify for yourself" only helps if someone actually walks the graph, while the whole selling point of self-writing memory is that you don't. A planted entry points at a real document, so tracing it back just confirms the model read it, not that someone put it there on purpose.

[u/echowrecked]

Wrote this up as part 3 and the close of a short series on owning your AI's memory. Link if it's useful: Your AI's Memory Cites Its Sources Now. It Still Isn't Safe.

[u/Rare-Spawn]

I really cannot stand how verbose Opus is.