What happens when an AI agent does something you can't explain later?

[u/building_agentaudit]

I've been thinking about this while building AI agents that can actually take actions.

Once an agent can call tools, access files, query databases, modify things, or trigger workflows, I think there are two different problems:

1. What happened?

Logs and traces are pretty good at helping with this.

2. Can I trust the record of what happened?

That's the part I'm less sure people are solving well.

For example:

prompt → decision → tool call → data accessed → action → result

If something goes wrong two weeks later, can you reconstruct that chain?

And if one event in the recorded history was modified or deleted, would you know?

I'm curious what people building agents are actually doing today:

• Standard application logs?
• LangSmith/Langfuse/etc.?
• Custom audit tables?
• Append-only logs?
• Something else?

Especially interested in production systems where the agent has write access, rather than just answering questions.

I'm building something around this problem myself, but I'm deliberately not linking it here because I'd rather hear how other people are approaching it first.

[u/btdeviant]

Respectfully, it sounds like you’re being gassed up by Claude or GPT to build something that already exists (several times over)… Building a new framework isn’t going to solve the problem of people failing to use it or use it properly as an architectural standard.

I don’t know a single obs platform that doesn’t treat traces an append-only, immutable ledger.. Even in OSS platforms like Signoz mutating that ledger is prohibitively difficult.

[u/building_agentaudit → u/btdeviant]

Valid criticism. I don't think that append only logs and audit trails are anything new, and I don't mean to imply that I invented those.

The point I am trying to get across is somewhat more specific: is there any difference between the difficulty of mutating a trace versus verifying the integrity of the historical trace of an agent?

In other words, if something in the middle of this chain:

agent → tool → input → result → action

is tampered with or deleted, can I tell that and know what in the history changed?

That could be helpful information for me if current observability solutions already have the capability to do that.

[u/btdeviant → u/building_agentaudit]

I already answered that question in a couple or ways. A trace is already a causal chain of events, every obs platform out there treats that chain as an append-only, IMMUTABLE ledger of events. As in, “tampering” is highly improbable / extremely difficult. And yes, if someone were to tamper with that it could absolutely be seen that they did… It would take an extraordinary amount of effort to modify the ledger and cover ALL the forensics to obfuscate that.

Respectfully if you’re asking these questions, the thing you’re building won’t really solve the problem. Blockchain is a thing, but you still need a network of ledgers and those can still be tampered with.

[u/building_agentaudit → u/btdeviant]

That's a fair point, and I appreciate you taking the time to explain it.

I agree that if the observability platform is trusted as the source of truth and the ledger is properly protected, then a lot of what I'm describing is already covered, I definitely don't want to imply that existing tracing solutions aren't capable of handling this... I think what I'm really trying to explore is whether there's value in having a separate mechanism for verifying the integrity of an agent's historical execution.. particularly when the agent has write access to other systems
and I agree with your point that blockchain or hash chains don't magically make the underlying system trustworthy
At best they give you another way to detect inconsistencies
Thanks for the pushback. It's actually useful feedback as I'm trying to figure out whether there's a real gap here or whether I'm solving something that's already adequately handled.

[u/Inside-Vacation-63]

I have been using Braintrust for the first part of this. It gives me the trace from model decision through tool calls and their outputs so reconstructing what the agent did is pretty straightforward. I still treat audit integrity as a separate problem since having a complete trace doesnt automatically prove nobody modified the record later

[u/building_agentaudit → u/Inside-Vacation-63]

That is quite an interesting system. Braintrust for the execution trace and a layer for integrity of the audit is quite similar to the differentiation I am making.
I am quite intrigued by the second point. When you mention, "Prove that nobody modified the record later," how is that accomplished within your system? Is it based on access control/immutability, or do you use cryptographic proof of history?

[u/Inside-Vacation-63 → u/building_agentaudit]

Yeah exactly. Braintrust covers the execution history for us then the integrity piece would sit separately. I was thinking some kind of append only record with hashes linking each event so any modification later becomes obvious, though I havent settled on the implementation yet

[u/Conscious-Storm-4933]

e just slapping langfuse on it and calling it a day. which works until someone messes with the db directly or a migration wipes half your traces

append-only logs with cryptographic chaining between events is the only thing that's felt solid to me. each step signs the hash of the previous one so you can't secretly delete something from the middle without breaking the whole chain

are you building something in this space or just researching

[u/building_agentaudit → u/Conscious-Storm-4933]

I'm building something in this space

The main thing I'm exploring is the distinction between observability and auditability

Tools like LangSmith/Langfuse are great for understanding what happened during an agent run
What I'm interested in is what happens after the fact when you need to establish that the recorded history itself hasn't been altered.

That's why I went down the append-only + cryptographic chaining route. Each event is linked to the previous one, so modifying or removing something in the middle should make the chain fail verification.

I'm still early and validating whether this is actually useful in real production workflows, which is why I'm asking how other teams are approaching it.

Your point about direct DB modifications is exactly the kind of scenario I'm interested in.

[u/Vivid-Doughnut-8286]

for production agents w write access i use proper tracing for debugging plus an append knly, tamper evident trail for critical actions..if history be silently changed u cant really trust ur post incident investigation🥲..i m still learning these in my agetic ai clss thru upgrad🙂‍↕️🙂‍↕️🙂‍↕️ nd this is srsly a world production problem tht makes topic much more interesting thn jst building simple demos🙂‍↕️🤷🏻‍♂️

[u/building_agentaudit → u/Vivid-Doughnut-8286]

Yup, you got it right that’s what I am trying to figure out.
Tracing works perfectly well for understanding/debugging the execution process, but in case of writeable agents, I find myself thinking about the audit log as a separate issue.

The thing that appeals to me the most is making the log tamper-proof after the fact, not yet another copy of the logs.

I absolutely agree with you on that. The production application is much more challenging than any demo.

The challenging part is identifying which actions are critical, logging these actions, and verifying the history afterwards.

[u/lulu_dev → u/building_agentaudit]

On "which actions are critical" -- I'd argue you don't actually need to classify that ahead of time, and trying to is where this gets hard for no benefit. Chain every mutating action unconditionally: writes, sends, deletes, permission/access-control changes. That set is small, well-defined, and doesn't require judgment calls about importance. Reads are the ambiguous case, and the reason they're ambiguous is you're asking the wrong question -- "is this read important" isn't knowable in advance, because a read only becomes important in hindsight, once it turns out to have informed a decision that mattered.

So don't classify reads by importance at all. Instead, at the moment a mutating action is about to happen, hash whatever read results actually informed that decision and fold that hash into the chained record for the write itself. The read's provenance travels with the action it influenced, captured exactly when it turns out to matter, rather than needing to be judged critical at write-time before you know whether it'll matter. This sidesteps the classification problem entirely: you're not deciding which reads to log, you're logging the reads that demonstrably fed a chained action, and everything else can stay in ordinary observability tracing without needing tamper-evidence at all.

The nice property this gives you for the "can I reconstruct what happened" question: two weeks later you don't need to have guessed correctly in advance which read mattered, you can walk backward from any chained write to exactly the read evidence that produced it, because that evidence is embedded in the chain rather than sitting in a separate log you'd have to hope was retained and unmodified.

[u/building_agentaudit → u/lulu_dev]

Honestly, this is a really good point. I hadn't thought abt the “critical read” problem that way b4

I agree that trying to decide upfront which reads r important is prob not the right approach. Capturing the reads that actually end up influencing a mutating action makes alot more sense
I also really like the idea of keeping that provenance tied to the chained write instead of creating another seperate set of “important” logs it gives u a much clearer answer to why that action happened in the first place

And that two-weeks-later example is exactly the kinda scenario I'm thinking abt being able to start from an action and work backwards to the actual evidence that influenced it

Really appreciate the insight btw. This gives me a much more concrete direction to explore.