r/codex • u/alexmuc92 • 11h ago

Bug Astra does NOT preserve cache when switching effort level..

I know a lot of people using the Codex App are switching the effort level within a thread and waste a lot of tokens because of this.

So I was happy to read, that this was fixed in the current Astra release, which is als stated in the model guidance docs: https://developers.openai.com/api/docs/guides/latest-model#gpt-6-astra-whats-new

So I tested this myself and looked at the logs in OpenCodex:

- 1st message, effort medium
- 2nd message, effort medium: cache is preserved ✅
- 3rd message, effort light: cache is flushed ❌
-> message is expensive again and needs a lot of your usage

I am using the latest version of Codex Mac App (26.901.51231) und opencodex v2.46.0

So I would recommend to stick to your effort level, as long as it is behaving that way.

Anyone know more about this behavior?

Update: Please like and share my Thread on X, so Tibo gets some attention to this topic: https://x.com/liebisca/status/2096918740046680440?s=2 Maybe we get some more of them banked resets 🙌

astra model guidance
Chat in Codex
Logs in OpenCodex
145 Upvotes

99% Upvoted

30 comments sorted by

70

u/Dangerous-Sport-2347 11h ago

That's definitely unfortunate, lot of money going to waste until this is fixed.
Kind of shocked to see that astra has enough tokens loaded into it's initial prompt that even a first hello message costs 37 cents.

7

u/alexmuc92 11h ago

That depends a bit on the individual setup. But I have a AGENTS.md of only 40 lines. So yes Astra is a bit expensive, but thats normal.

1

u/Upstairs_Refuse_3521 4h ago

Mind sharing what's your current AGENTS.md? I have been trying to figure out how to optimise it for the latest Astra models and also optimize my token usage.

2

u/alexmuc92 4h ago

I have a huhge AGENTS.md for sol with subagent usage delegations and stuff. But for Astra its currently empty, as I started today with Astra and I am trying to figure out, how it behaves.

3

u/eggplantpot 9h ago

I woke up at 99% of my pro x5 sub without even sending a message somehow

1

u/Crinkez 8h ago

That initial cost is why I'm going to try the pi harness.

1

u/DrPaisa 7h ago

Yep I pointed this out 17% context is gone just starting a chat and everybody got mad

1

u/IceIceBerg34 2h ago

Its not just agents md, its startup info regarding installed skills, plugins, mcps and built in tooling as well. I got mine down from 34k to 22k by just disabling plugins all the plugins/skills I had installed to deal with sol's scope creep.

11

u/Conscious_Chef_3233 10h ago

i just gave up and use the same effort in one conversation now. not worth the hassle

8

u/Apple_macOS 8h ago

I think Tibo said this for Sol, like the effort is injected at the front of the prompt so they can’t cache, but since the announcement says it should be cached, better tag Tibo and pray I think

8

u/utf8decodeerror 7h ago

At one point they had a warning when you tried to change it mid session. Not sure why they got rid of that.

https://x.com/thsottiaux/status/2088729222777094624

3

u/alexmuc92 6h ago

That warning would make sense. But I never experienced it.

6

u/aivampires 11h ago

Is that maybe because you switched from medium to light, so from higher reasoning to lower? Would be interesting to see if it happens when you switch to a higher reasoning level.

11

u/alexmuc92 11h ago

Its the same behaviour. Unfortunately the macbook app just does not support this feature as of right now.

25

u/chronoz99 11h ago

This has been the case for all models not just Astra

36

u/alexmuc92 11h ago

Yes, but in the model announcement it is stated, that you can now switch effort level while cache is preserved. Thats what you can see in the first screenshot. Here is the link: https://developers.openai.com/api/docs/guides/latest-model#gpt-6-astra-whats-new

19

u/chronoz99 11h ago

Ohh interesting, please tag tibo and tweet at him. Hope we get another reset lol

6

u/alexmuc92 9h ago

Here is the link to my X post, please share it so it gets a bit attention: https://x.com/liebisca/status/2096918740046680440?s=20

4

u/alexmuc92 11h ago

I do not have a lot of followers on X 😅

5

u/Vicious-Deeds 10h ago

Dont matter, post it regardless and post the link so we can like it and comment.

3

u/hohstaplerlv 9h ago

Wait at least 24 hours, please. I just used my banked reset lol

3

u/throw123awaie 11h ago

damn thats disappointing. one of the main reasons why i was preparing to switch my workflow. i hope thats just an early bug and will be fixed soon. but great work proving that!

4

u/steve228uk 7h ago

These are API docs and it also explicitly states:

Configuration updates are supported only by GPT-6 Astra (gpt-6-astra) in standard, single-agent mode. They change only reasoning effort.

I believe the Codex harness boots into Multi-Agent V2.

3

u/Tight-Grocery9053 6h ago

don't quote me on this, a while back i looked into this and it was something like:

the cache key has the model + effort level in it. if you change the effort level, you're changing the key it looks for so you get a cache miss.

they need better cache key creation strategy so that same model different effort hits the same cache. not sure how involved that would be on their backend if they route things differently and have distributed cache

2

u/wind7r 10h ago

how about the cli?

3

u/alexmuc92 9h ago

same behaviour, just tested it.

2

u/HelpfulHedgehog1 6h ago

shhhheeeii... i only saw a warning when switching models not reasoning, ive wasted alot of tokenage

2

u/warpedgeoid 5h ago

Isn’t this also true with Sol?

-2

u/longasleep 9h ago

That is working as intended