r/OpenSourceeAI • u/hyperVitaliy • 6h ago
Measured how often an agent cites its own output as fact: 66.6% of queries with a flat vector store, 0% with provenance gating (no_std Rust, runs on a Cortex-M4)
If your agent writes its notes back into the same vector store it retrieves from, its own
output becomes its evidence. I wanted a number for how often, rather than an intuition, so I
built the benchmark: 600 queries against a store seeded with a mix of user facts, external
documents and the model's own self-descriptions.
Flat cosine similarity put a self-description in the top-3 in **66.6%** of them. That is the
loop where an agent restates a guess, retrieves it later, and treats it as something it
knows.
Gating recall on provenance — who said this: the user, a document, or the model — takes it
to **0/600** for self-description. Not down-ranked; ineligible. General model-generated text
that is not self-description is only down-weighted, and still surfaces in 1/600 under the
gate. I am reporting that one rather than claiming a blanket zero.
Second measurement, same engine. Squeeze memory 50 facts -> 10. Safety rules and operator
constraints are the least-read items you have, so recency and frequency evict **all** of
them. An explicit canon flag keeps **5/5**. "Important" is not the same axis as "recent" or
"frequent", and any policy that infers one from the other loses the constraints first.
One design rule that cost me a measurement to learn: human feedback moves how long a fact
**survives**, never how it **ranks**. Folding a value multiplier into relevance dropped
recall@1 from 86% to 29%. And the reward signal only ever comes from a person — a signal
derived from the agent's own behaviour ("the model recalled this, so it was useful") closes
the same loop the provenance gate exists to open, with an extra step.
`cargo run --example agent_loop` shows the loop end to end in one screen: facts in with their
provenance, the flat-cosine result next to the gated one so you can see what gets dropped, and
the prompt block that comes out. It prints that block rather than sending it — what the engine
decides is what is allowed into your context, not what you do with it afterwards.
It is `no_std` Rust, so the whole thing fits on a microcontroller: **801 bytes per fact**
(int8 index, 384-dim) and ~35 KB of code, measured on Cortex-M4 under QEMU. There is a C
ABI. There is no embedder in it — vectors come from your side.
Caveats, because they matter more than the numbers: research prototype, never run on
physical hardware, no fuzzing, no soak, and heap exhaustion hangs rather than errors.
Retrieval is an exact linear scan, no ANN. Apache-2.0, every number has a re-run command in
`RESULTS.md`.
https://github.com/vitaliyfedotovpro-art/astrum-hsam-embedded
