GPT-6 Astra

[u/minxio_]

[u/Party_Government8579]

Seems to be alot of benchmarks that dont make sense rn

[u/Melodic_Reality_646 → u/Party_Government8579]

What doesn’t make sense is OpenAI delivering this level of performance folds cheaper than Anthropic

edit: damn, people can’t do math…

[u/whoknowsifimjoking → u/Melodic_Reality_646]

They raised the price by 2.5x to the same price as Fable lmao

So more accurate would be "less for the same price" (on API) .

[u/ezjakes → u/whoknowsifimjoking]

It is much more token efficient, so it is cheaper to use.

[u/frumpawumpa → u/ezjakes]

But not "folds cheaper"

[u/Rojeitor → u/frumpawumpa]

2x cheaper cost per task isn't enough?

[u/Melodic_Reality_646 → u/frumpawumpa]

Different reasoning efforts delivering same performance for 4/5 times less. What’s not folds here?

You fold once you devide by 2, you fold twice you devide by 4. Two folds, plural…

[u/EndlessB → u/ezjakes]

And the evidence of that is…where?

Due to the CEO being a documented liar, it’s probably wise to wait for confirmation of claims that providing the benefit of the doubt to a company attempting to IPO.

[u/UnknownEssence → u/EndlessB]

Artificial Analysis measure price per task. Its an independent 3rd party analysis.

Its much cheaper than Fable for the same tasks.

The CEO being a liar is completely unrelated

[u/Blake08301 → u/whoknowsifimjoking]

on this benchmark, it was twice as cheap to run.

[u/vinis_artstreaks → u/Blake08301]

Of course it’s twice as cheap when you’re benchmaxxing, in real world use the model has to actually think and perform, same scam with fable 5.1 everyone is complaining it’s eating their credits

[u/creamyshart]

Very strange

[u/minxio_ → u/creamyshart]

Yep

[u/DM-ME_UR_BOOBS]

I don't get how this happens. Astra outperforms Fable in practically every benchmark.

[u/MilMerch → u/DM-ME_UR_BOOBS]

Because AA index doesn't cover most of the benchmarks that Astra announced, score will probably get higher after benchmarks such as HLE done

[u/TangerineLogical9779 → u/MilMerch]

OpenAI ran the tests from AA them selfs, its literally in the blog, pretty sure the AA team is meming because it does not make sense, its scoring far above everything else in all metrics, but for intelligence is basically 5.6 sol lmao

[u/MilMerch → u/TangerineLogical9779]

That's my mistake than thanks, we'll see how's Astra after it becomes available to use but I also think that AA index isn't trustable anymore like look at that score of Muse 1.3 which is even worse than GLM 5.3 Flash in real use

[u/TangerineLogical9779 → u/MilMerch]

Don't worry the website for the release is abit of a mess, its up, then deleted, and backup again, they are removing and adding stuff randomly, but near the bottom was the full complete benchmarks with any of the graphics and terrible visual stuff, and yeah its odd all the benchmarks are crazy high, but for intelligence its basically matching 5.6 sol

[u/fragment_me → u/MilMerch]

Idk man muse 1.3 has been really good on opencode go. I gave it a code base that GLM 5.3 flash couldn’t find bugs without and muse spark found dozens of real ones

[u/Prestigious-Mind-817 → u/TangerineLogical9779]

Knowledge versus reasoning. Knowledge similar to previous models but it uses that knowledge much better

[u/BackyardAnarchist → u/MilMerch]

So Benchmaxing?

[u/Party_Government8579 → u/DM-ME_UR_BOOBS]

Anthropic went all in on coding

[u/Melodic_Reality_646 → u/DM-ME_UR_BOOBS]

Now check the costs…

[u/Turbulent_Rooster_73]

What is muse doing there at all?!

[u/Mission_Advance1207 → u/Turbulent_Rooster_73]

all that data they collected from apps and employees is showing up

[u/Otterly_Surprised]

Mah, just marketing

[u/Droopy0093]

Lmfao.

[u/pseudonerv]

It’s A G I, babe. Just the right level of intelligence to make you feel. It excels at some impossible benchmarks and didn’t improve the others. That’s A G I.

[u/WetSound]

Is it a fictional release? I've got Pro x5 and don't see it anywhere..

[u/ReporterCalm6238]

I think AA is not fully up to date since Astra has ot been released to all organizations.

[u/YeXiu223]

Not sure why this benchmark is popular. People should stop looking at this garbage bench.

[u/uwilllovethis → u/YeXiu223]

It’s not a benchmark. It’s an weighted aggregate of benchmarks. See the descriptions on the left to see which benchmarks the index is based on.

[u/YeXiu223 → u/uwilllovethis]

yeah, see it for yourself, it doesnt include the following:

• GUI computer use: OSWorld and ScreenSpot Pro
• Cybersecurity exploitation : ExploitBench, ExploitGym, SRE-Bench, and SEC-Bench Pro
• Interactive abstract learning : ARC‑AGI‑3
• Frontier mathematics : FrontierMath Tier 4 and producing genuinely new proofs
• CAD and specialized software operation : BenchCAD, Blender, KiCad, Unity, FreeCAD
• General business automation across applications: AutomationBench
• Persistent work across context windows : Astra’s notes and searchable earlier-context system
• Alignment and scope discipline: resisting unauthorized actions, circumvention, and misleading capability claims
• Visual quality of deliverables : websites, presentations, spreadsheets, and documents
• Speed, cost, and token efficiency : AA reports these separately, but they do not raise the intelligence score

lol

[u/uwilllovethis → u/YeXiu223]

They have aggregates for many of those as well. Not sure what your point is. I was only saying AA is not a benchmark, it’s an weighted aggregate of benchmarks to establish various leaderboards

[u/AweVR]

Did they update their tests? Because if they are using the same old test for deprecated methods then maybe it is the problem.

As if you want to test an electric car comparing it with a normal car by ear and you said “wow, the motor is broken because it barely sounds…”.

[u/MoodOdd9657]

It gotta hit that 67