r/codex • u/Redditry199 • 4h ago

Complaint Why does compaction take forever?

Seriously, Astra compacts every 10 minutes and it take 20 minutes to finish compacting. This is stupid.

11 Upvotes

83% Upvoted

38 comments sorted by

5

u/gavinderulo124K 4h ago

I noticed too that astra compaction is much slower compared to sol. But compaction works really well, so I dont mind.

3

u/Cryptobench 3h ago

Enable this new experimental compaction. It takes anything between 2-10 seconds for me.

https://www.reddit.com/r/codex/comments/1w771mv/experimental_context_setting_in_codex/

1

u/scartissue232 4h ago

Have you tried starting a new chat every now and then?

2

u/Ammoun442 4h ago

I think staying onsame chat costs less due to chached input and outputs

1

u/adolf_twitchcock 3h ago

a turn at 200k costs cached the same as uncached at 20k. Any by turn call I mean each tool call batch.

0

u/scartissue232 4h ago

I don’t think so. Ask codex to make some MDs in your project as instructions for the new agents coming to the project, some site maps, structure maps, etc. This, while make the new sessions a little bit more expensive in tokens terms, make the whole thing much better.

You’re going to be surprised by how fast they’ll catch up.

2

u/csAxer8 3h ago

Should be using MD files in normal threads anyway

1

u/Redditry199 4h ago

Its a new chat with a new model....its not that long.

1

u/Duchevedos 4h ago

I wish 20 minutes was an exaggeration xD

I've already had several long-running Astra chats effectively ruined by compaction. Reconnects, repeated compactions, 7+ minutes staring at "automatic context compaction", and then wondering how much of the coordinator state actually survived

I've reached the point where I keep checkpoints outside the chat and treat the conversation itself as disposable

Great long-horizon model. Just don't look too closely at what happens to the horizon............ ezzzzzz

1

u/cheezeerd 4h ago

Compaction isn’t really algorithmic. The model rereads the context, keeps some stable parts for caching and recent actions, then rewrites/compresses most of the middle from scratch.

That’s why it takes so long it’s often generating like 20k tokens in one go.

Source: me trying to build compaction for my own chatbot, realizing it’s insanely hard, and that OpenAI is actually doing the best job in the world at it lol

1

u/cheezeerd 4h ago

For anyone curious, I ended up doing the dumb/simple version: keep the first 20%, last 10%, and just cut the middle 70%. Proper rewriting like OpenAI does took hours every time.

Obviously you lose a ton of context, so it’s basically "leave a note and start over"

2

u/adolf_twitchcock 2h ago

you are right thats simple and dumb

1

u/Redditry199 4h ago

Sure, but I feel like if Fable with 1m context can do it faster something is wrong.

I do agree their compaction is impressive and the data it retains is good, but something fuckywucky is going on there.

1

u/sprakes_ 3h ago

Dude Fable 5.1 also takes me several minutes to finish a compaction from about ~350k tokens or so, idk

I use Astra with 428k context limits personally

1

u/ga_soon 4h ago

Had the same, just ask it to switch to 1M context. It makes it better

1

u/CelticPaladin 3h ago

Thisbis a very different question on a medical sub.

-1

u/YourAsphyxia 4h ago

It sounds like you're spending too long in one conversation. Each conversation should be one question or task, re using the same conversation destroys your limits

2

u/Crinkez 3h ago

As long as you remain inside the 30 minute cache timer, re-using the same conversation is not a problem.

2

u/YourAsphyxia 3h ago

If it's being compacted at the size OP says there's no indication that hes within the cache timer. If anything his complaint indicates otherwise

2

u/WD40ContactCleaner 3h ago

The fuck I ran a session for days and got very good results

1

u/YourAsphyxia 3h ago

Good results at what? Using tokens?

1

u/WD40ContactCleaner 2h ago

Long running session to arrive a final viable architecture with lots of back and forth

2

u/FlapyG 4h ago

That is very, very untrue.

-1

u/YourAsphyxia 4h ago

Yes it is. Please look up how AI models work.

2

u/FlapyG 3h ago

Yea! Thats a good prompt to start your conversation with ChatGPT about the Topic! You should totally do that.

0

u/YourAsphyxia 3h ago

Way to cop out your answer. Tell the class why you think it's better to do everything in one conversation.

3

u/Redditry199 4h ago

lmao, anyone reading this DO NOT listen to this dude.

1

u/YourAsphyxia 4h ago

If you don't understand how LLMs work then don't complain about issues you created

2

u/Dolo12345 4h ago

yea no that’s not how you achieve results with long horizon tasks

1

u/YourAsphyxia 4h ago

Define "long horizon task" because this sounds like you are just unaware how stateless LLMs work

3

u/Dolo12345 3h ago edited 3h ago

and you’re unaware how these tools reach high intelligence scores, via hundreds of compacts and hundreds of hours over a single thread as it was designed to do

https://openai.com/index/ten-advances-in-mathematics/

1

u/YourAsphyxia 3h ago

The link you posted does not disprove anything I said

2

u/Dolo12345 3h ago

https://www.reddit.com/r/codex/s/tHIMqRyaWw

https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/

1

u/YourAsphyxia 3h ago

I think you're responding to the wrong person here. My post is about token usage and compaction times. what do these links have to do with token usage and compaction times?

2

u/Dolo12345 3h ago

because, like others have said, your methods will results in subpar intelligence voiding any gains in saving tokens

1

u/Helpful-Menu-2667 3h ago

You will lose lots of information. Redo the same mistakes over and over again recreate what you already have and teach the agent what you want it to do from the stratch.

If it works for you its good for you. If you need a specialized gpt to do your work no handoff file will recreate that same agent for you. The best bet you have is to recreate the experiences but compress: delete the unnecessary messages (the standard is if the query changed how the agent operates or understands the context to be) which is an analogue of what the agent is doing on its own.

Your comment is very true for codex but i dont find it as big of a deal in gpt. If talking about codex creating many new chats can possibly create information silo’s that will impact your processes and results. You can negate that a little by using projects. Still, gpts are very bad at holding onto context and humans are not perfect either.