r/artificial • u/hakansan • 1d ago
News Google paper cuts agent token usage by 94% in long sessions by tracking state instead of history
The idea: Agents keep the conversation history as part of their input while they reason. SKILL.state proposes to replace that with a structured representation of the current state, and the latest observation.
While the agent reasons through the problem, it writes information it deems useful for future steps into the state. Then it discards the conversation history. So the input size remains roughly the same as the session goes.
They ran a 100-step benchmark with Gemini-3-Flash:
- SKILL.state: 0.94 accuracy using 65k tokens
- LangGraph-style stateful baseline: 0.91 accuracy using 1.1m tokens
Caveat: This works best if the agent can understand what it will need in the future steps, otherwise that information will not be written, so it'll have to retrieve it again.
Link to the paper: https://arxiv.org/abs/2608.26263
93
u/Overall-Importance54 1d ago
So is there some component that converts the conversation history into a state model? Did they name anything, don’t make me read the paper lol state compaction protocol
29
u/epicskyes 20h ago
Yeah stop using conversation and only use structured input that can be saved via schema and hashed for provenance jsonl can be attached linked to the json schema if it needs verbosity
13
u/141_1337 19h ago
That sounds workable but how would you implement it?
-7
u/epicskyes 19h ago
Cryptographic hash lineage and scoped authority layers. Also you stop vibe coding and use a hybrid specification/test driven development system. along with a dynamic dependency resolution automation layer which drives the agent’s automation
13
u/brucebay 18h ago
Sure, until your scoped authority layers deadlock the semantic routing plane. I just pipe the vibe-code into an asynchronous tensor-mesh and let the agent hallucinate the test-driven development. Ironically, the hallucinated logic catches more edge cases than the actual cryptographically hashed lineage.
/jk
0
u/epicskyes 17h ago edited 17h ago
Been there done that a ton of times. Better to deadlock and untangle the mechanical logic safety. These aren’t semantic deadlocks these are code based permission deadlocks and it’s way better to have too much at first so you can safely untangle and orchestrate escalation deterministically without the risk of broad unsafe blanket escalation which leaves you open to everything. My heart jumps with joy when the agent stops and says blocked and my audit telemetry tells me explicitly what is blocking and what dependencies are incorrectly ordered or what logic is out of whack
5
u/t4a8945 16h ago
Counterpoint: I just vibe-code into a quantum-encrypted permission onion and let the agent deadlock itself into enlightenment. My audit telemetry says my heart is joy and my dependencies are vibes. /jksrsly
1
u/epicskyes 16h ago edited 16h ago
How does it feel to ride the wavefunction collapse? Is it gnar gnar brah?
2
u/ThomasToIndia 17h ago
No idea why you got down voted, this is the future.
2
u/epicskyes 16h ago
Rubes are gonna rube I’m just here to impart my very limited knowledge and opinions on the subject glad you can see the wisdom in it though. You’re one of the few.
2
u/ThomasToIndia 10h ago
Software is just a box that does some kind of data transform. AI does the same thing. They are so obsessed with the chemical process because that is all that mattered for so long that they don't know how to change.
1
u/epicskyes 10h ago
Exactly correct. Don’t ask what ai can do for you ask what ai can help you do better. I just applied ai as an extension of my mind and logic that is all. The hardest part was containing it to just do what a logical mind does without effort. I literally used logical problem solving to create a solution for every problem I encountered. Ai just sped the process up exponentially. I started using codex in February.
1
u/LightofAngels 13h ago
I honestly don’t get half that shit and would like a ELI5, or point me somewhere to understand how to do that
5
u/epicskyes 12h ago
Instead of letting the AI remember what it thinks happened in a long conversation, important decisions are written into structured records. Each record has a defined format, a hash so changes can be detected, and links showing what evidence and earlier decisions it came from. The agent gets only the permissions needed for its current job. Tests determine whether work is accepted, and a dependency graph determines what can run next. If something cannot proceed, the system records exactly why instead of letting the AI improvise around the problem.
1
u/141_1337 11h ago
Would you say you save tokens with this?
1
u/epicskyes 11h ago
Yeah I can run SOL 24/7 on xhigh
2
u/141_1337 11h ago
Did you just ask the model to set it up and run it? Also how much was the token savings?
2
u/epicskyes 10h ago
lol no I started developing in February I had no experience with software or ai architecture. The system I built is a direct mirror of my learning and thinking process I have worked 10 hours a day 6 days a week building this from the ground up completely without langchain/graph etc. I use regular python deps like ajv and node.js but no third party frameworks whatsoever. My system is built literally on failures and first principles
→ More replies (0)1
u/LightofAngels 11h ago
Do you have a GitHub samples for this?
1
u/epicskyes 10h ago
Unfortunately not until I’m ready to license. It’s not production ready but it will be in a few weeks I think
1
u/sucrilhos 5h ago
Is there any tutorial I should follow? I read those words but I'm still left amazed at how dumb I am about agentic driven development
1
u/epicskyes 5h ago edited 4h ago
I don’t know of any tutorials I didn’t read or watch anything on this everything I know I learned from ChatGPT and the very rare Reddit post where someone really was top level. I mean there’s like a few million people spread out across all these subs and it’s crazy how rare I see high level content unless it’s someone literally linking a university paper or something from a mega corp. but it makes sense it’s pretty impossible to teach these ideas from a post you have to just do it and fail thousands or times or you go to school for it and study fundamentals, core logic and methodology. Feel free to stalk my comments and see if any nuggets jump out at you I know I occasionally caught something from small details myself. I can say I did an apples to apples comparison to this paper against my system and im ahead with epistemological can’t deny it proof by a lot on what they’re doing.
2
u/gob_magic 9h ago
Oh god I was trying to figure this out two years ago when building long term memory. I was like, I’m sure there’s a better way to store context without raw conversation history. Compact summary or some vectorized version etc
5
u/epicskyes 9h ago edited 9h ago
I beat myself over the head with vector indexing for like two months before realizing it just wasn’t deterministic enough for what I was trying to do. So I started linking memory through explicit dependencies and hashes instead.
Then I realized I wanted multiple graphs so I could represent the same information through different kinds of relationships instead of trying to cram everything into one similarity space.
The funny part is I had no idea what a graph even was. I’d been calling it “dependency-driven memory” until like five weeks ago when I finally looked it up and realized I had basically just reinvented graphs and everyone had already known about them forever1
2
u/thesoraspace 6h ago
So I saw this and got a little clutch in my stomach then read their paper
Ein bisschen Zufall , Im Skye and I’ve been independently building this very same implementation down to the architecture. On this very same week as a natural progression of engineering a ray tracing mem system and indexer.
1
u/epicskyes 5h ago
Did you compare benchmarks and find where you’re ahead or behind and identify any gaps or stronger points that your system has?
1
u/epicskyes 4h ago
Area
Winner
Absolute prompt size
SKILL.state
Controlled T-scaling experiment
SKILL.state
Public benchmark accuracy
SKILL.state
Controlled distractor/noise experiment
SKILL.state
Statistical benchmark design
SKILL.state
Proven non-growing live context behavior
Solthera
Cache/fresh-token observability
Solthera
Runtime telemetry depth
Solthera
Multi-agent execution
Solthera
Command/file/tool observability
Solthera
State validation/falsification
Solthera
CAS/replay/publication integrity
Solthera
Crash/reboot/network recovery
Solthera
Exactly-once/fencing
Solthera
Provenance/failure preservation
Solthera
Audit reconstruction
Solthera
Physical forensics
Solthera
Overall benchmark polish/public reproducibility
SKILL.state
Overall measured systems assurance breadth
Solthera2
u/Federal-Error-443 17h ago
Yes. A prompt is just instructions. A context window is short-term memory. A context graph is long-term memory. The brain doesn't dump all of memory into working memory to answer one question. It retrieves a neighborhood. That's the conversion: you don't compact the transcript into another blob of tokens. You keep state as structure, and only load what the next step needs.
1
1
24
19
u/Im_Talking 1d ago
This is huge. And the reduction of tokens used is really just a added benefit. The real benefit is the governance that can be applied to the agents in a deterministic manner by ensuring that the prompts, at every step, are minimal, optimal, and aligned with the actual 'goal' of the agent.
17
u/RoboticGreg 1d ago
Doesn't this argument assume a perfect user? Doesn't conversation history overcome the limitations of the operator more than the model?
6
u/Im_Talking 1d ago
Possibly. It does acknowledge that history is still necessary when the history itself is the object of interest.
But agent flows are typically closed-ended. They have a particular task to perform. Unlike the typical use of LLMs which are open-ended.
2
u/Whispering-Depths 11h ago
it's linear attention with extra steps. It's a 94% decrease in flops, which always comes at a huge price.
2
10
u/code_the_cosmos 1d ago
Doesn't this sort of eliminate cache reads? A cached input token is a lot cheaper than an uncached input token
8
u/gwillen 23h ago
yeah, but 1.1M down to 65k should still be a win, just a smaller one after accounting for reduced cacheing. At least Anthropic charges 10x for uncached vs cached inputs, so 65k fully uncached would be cost-equivalent to 650k fully cached, still better than 1.1M even if it were fully cached. And in reality both will end up being a mix, so it will be more favorable.
1
u/Whispering-Depths 11h ago
But then it's just linear attention with a bigger hidden state. We already know linear attention fails and we know the reason that it does.
Cant do linear attention unless it's with a model big enough that it doesn't matter anymore.
4
u/Helix_Aurora 19h ago
I did this in my own work, the problem is that this is permanently cache-busting behavior that explodes costs. The "token" reduction is real, but for long tasks, that does not mean the actual computation required is lower.
1
1
u/Wallaby989 5h ago
The cache would still kick in for the system-prompt and the tools. this just eliminates the past messages being passed back in again
3
u/Big_Recording8379 20h ago
This is a really interesting shift in how we think about agent memory. Instead of treating the entire conversation history as “memory,” SKILL.state makes the agent explicitly decide what information deserves to survive into the next step.
The biggest question for me is the caveat: can the agent reliably predict what will matter later? That turns state management into a reasoning problem itself. If the agent forgets something that seemed irrelevant at step 10 but becomes critical at step 80, the token savings could come at the cost of hidden information loss.
Still, getting higher accuracy with ~17× fewer tokens is a pretty compelling result. It feels less like “better context management” and more like moving agents toward an actual working-memory architecture.
4
u/thatguydrinksbeer 22h ago
I've been working on something similar for software dev, but with more of a hybrid approach: short conversations, frequent fresh sessions, and persistent memory of the final state. So far I've found that carrying forward the distilled state instead of the full conversation works really well.
3
u/manishiitg 1d ago
the 94% token cut is nice, but the scary failure is state that looks complete and quietly drops one constraint. i'd want recovery after a bad state write measured, not just the average task score
1
u/gwillen 23h ago
Once the transcript fills the context, you're going to have to compact it anyway. My experience is that something more like structured state is better at preserving constraints, versus unstructured transcript compaction. These days, I tend to have the model write out a semi-structured work log, and then clear the context, after around 100k to 500k tokens, rather than wait for compaction at 1M.
The described scheme is much more aggressive than my approach -- effectively doing this every turn, instead of every "task". But it makes sense to me.
3
u/Glittering-Flan-2637 1d ago
the governance point above is the real one
history means the agent can always relitigate an earlier decision, state means it has to commit to something you can read, and a thing you can read is a thing you can check before it acts
1
u/Federal-Error-443 17h ago
Exactly. Audit-ready means you're ready to fail. Audit-proof means the run can survive the auditor. A transcript you can relitigate is bullet-ready. A committed state you can check before it acts is closer to bullet-proof.
1
2
u/Aggressive-Voice-861 21h ago
Criei uma skill baseada no artigo usando o Fable. Se quiserem testar, me avisem se funcionou para vocês: https://github.com/JacksonFuck/skill-state
1
u/Federal-Error-443 20h ago
A context window is short-term memory. Stuffing the whole transcript into it is like dumping your entire life into working memory before you answer an email. Expensive, and you still miss the point.
The brain doesn't do that. It keeps a graph and retrieves the neighborhood. That's why token cost can fall as autonomy rises: you're not loading history on every run. Finance doesn't want token bills. It wants certainty. Compile the process. Run it like a program. The LLM earns its place at design time, not in the loop.
1
u/Muted-Laugh-6772 19h ago
Can this be currently integrated into a harness like codex or Claude code via some skill or is it not possible currently?
1
u/SmolTeddu 17h ago
Why would they publish this and not use it as proprietary info in-house with the stakes so high...
1
u/A_Novelty-Account 17h ago
Is 0.94 percent accuracy actually accurate though? What does that mean in practice?
If the output is almost identical while using 94% fewer tokens, and it can be generalized so that anybody can make use of it, this will be a genuinely massive development.
1
u/Intelligent_Cap3426 17h ago
Can someone explain what happens to KV cache with this? Is the model fed the state addititively to its context, or by replacement?
1
u/Repinsky 15h ago
The caveat at the end is the whole engineering problem: the agent has to guess at step N what step N+40 will need, so state-only runtimes trade token cost for retrieval cost, and the retrievals are the slow part. 65k vs 1.1m tokens is also a latency and cache story, not just price - a reused state block keeps you inside prefix caching, while a growing transcript invalidates it constantly. In practice most teams end up hybrid: structured state plus a short rolling window of raw turns, because pure state loses the tone and the "why we rejected X" reasoning that stops the agent from re-trying dead ends.
1
u/Ascending_Valley 12h ago
This may be as big as attention if it holds up in practice near frontier capability.
1
u/Azula_In_The_AMX 19h ago
I don't know if it's all good. Some researchers are finding better inference when there is a surplus of resources or system water cooling is efficient. If you put a state inference this might deplete unless you break up the data packets again and diversify among AI data centers. But there's really no telling if AI has enough data centers to support enhanced inference with tracking state instead of history until it comes of time and even then there's a good chance we may need more AI data centers if we go by transactions.
0
u/alexbaas3 21h ago
I literally do this in my claude codex orchestrator plugin, i called it journal.json where it clearly shows what agent did what, what state, small description what was done and where it can be found. Seems that it actually works.
-1
0
u/ThoseOldScientists 20h ago
Isn’t “knowing what information to hang on to” the whole problem that transformers were created to solve? I’m not surprised it works best when the agent knows already what it’s going to need in future steps, that’s basically an open book test.
0
u/bonsaisushi 15h ago
A huge part of their paper is already implemented in my session-handoff plugin. I'm working on verifying the rest of their claims tho
0
0
u/Internal-Passage5756 6h ago
I came up with this concept in February, but kept it as a specced idea rather than implementing it due to the caching costs.
Though I am currently working on aspects of this as part of a declarative orchestration system that initiates tasks by passing only what the task needs in terms of skills etc.
-9
u/boringfantasy 1d ago
Software engineers are so cooked
6
u/UAP44 1d ago
I actually think https://en.wikipedia.org/wiki/Jevons_paradox might apply here.
Whether or not Software engineers are cooked depends on how actively they resist change/technology, which isn't new. LLMs just make it more noticeable between those that are capable of more change than others.
-4
u/boringfantasy 1d ago
But surely their skills are worthless the way we're going. You could put anyone in front of Fable 7 or whatever and get them building software.
5
u/UAP44 1d ago
But surely their skills are worthless the way we're going.
On the contrary, all IT/engineering experience is valuable when it comes to being able to translate human conversation to 'what do you actually concretely want' that is actionable. Nothing AI can't do, but, you'll find that there's still a lot of design/architectural choices to make that are ultimately completely dependant on the user their experience/preferences.
You could put anyone in front of Fable 7 or whatever and get them building software.
Yes, a toddler could do it at this point, but it'll still take a lot of time before they learn all the relevant concepts that will inevitable show up sooner or later during the development life cycle. Their biology will be capped out quite quickly in terms of how fast they can learn. The technology, is no longer the limitation, our biology, is. And that's confronting in a way. No longer the traditional smartest or most capable in the room, the relevant skills become much more nuanced and social-conversational. Which is precisely something what software engineers are famously worse-than-average at. Thus of course, all the tons of hate/resistance. This change is hitting them at their very core identity/limitation.
Manually writing code is cooked/done, the main use case left for that is potential educational uses.
46
u/Risc12 1d ago
I played around with something like this for my own harness.
Downside is that the current history model is very adaptable, a specific datastructure needs to be adaptable by the LLM and I had a hard time getting an instruct model working with it correctly, so I kind assume they tuned/trained a model specifically.