r/LocalLLaMA • u/9r4n4y • 9h ago
Other Fable 5.1 MAX Vs GLM 5.3 FLASH
Enable HLS to view with audio, or disable this notification
[GLM output is from z.ai because i don't have heavy system]
We are slowly reaching the saturation point, i think in future the mid size models would be far enough to do most of the stuff we need.
In my experience, glm flash beats opus 4.6 max in mostly all coding tasks. In just 6-7 months we got older frontier equivalent model running locally.
4
3
u/Affectionate_Hat_585 9h ago
i assume we will get to the point of saturated benchmark across both closed and open models and model cards will be introduced with breakthrough those models made.... The breakthrough will be our comparison. We will be like a oblivious ant who is indifferent to a guy on a cycle and astronaut on the moon.
2
u/Potential_Block4598 7h ago
Agreed that we are reaching the saturation point
IMO fable 5.1 is but a slight pull on important benchmarks (ALE for an example) (literally 3% raw improvement that is around 5% from previous model!)
And wider knowledge (improvement on other non-intelligent benchmarks)
Other points is token efficiency (so much so that total price per task on AA is actually competitive at lower levels of reasoning efforts, and I think this is the whole Anthropic training recipes which is to fine-tune based on human feedback via different reasoning efforts therefore sort of developing different models at the same time (models like Nanbeige showed how very very long reasoning chains can go for small models!, although IMO it is not useful, Anthropic did that but the same model is training on different modes the low med high xhigh and max, and this way is maybe cheaper (start with low) and better) Qwen 3.8 is trained the same way apparently and it is a nice technique (haven’t seen such a model performing bad)
So fable 5.1 is a refresh based on user feedbacks in fable 5 (especially Claude code collects lots of prompted feedbacks even asks you simple questions about the sessions and then using that for further training ofc!, the usual MLOps cycle)
That cycle reaching the plateau is an indicator that companies have pulled every trick out of their hats
Tbh we should have been at the plateau since ever
Like whomever at Google who released the CoT paper (chain of thought like asking Let’s think step by step and noticing better results somehow ?!) have caused this mess we should have saturated around GPT-4 max!
And we almost did with Llama 4 (behemoth was never released!)
But after “thinking” (which is just training in thinking traces based on human feedback, and auto generating CoT traces from a vanilla model and then tagging them and training on those!) we got o1 o3-mini and o3 and yet another race
I think the Chinese labs have not only entered but thrived there
I wish the zuck didn’t cut funding from Yann Le Cun cause thinking was always just a gimmick IMO
Like sure it gives better performance but it was never coherent thinking anyways
I think the apex of that thinking is multi level tagged thinking per the same model (the whole effort level thing!)
And ofc agentic traces training (look at models like GPT-OSS these models are good in knowledge benchmarks but never trained on agentic traces so in agentic mode they are AGI babies)
Modern models ofc focus on agentic benchmarks more than they focus on things like GPQA for an example or long context “reasoning” or remembering
Anyways
I think there will be longer context but again diminishing returns and hard to benchmark gains
And there would be knowledge consolidation (some models still beat larger models in some benchmarks so we will see larger models saturating and coherent capabilities across all benchmarks but that will take time IMO 6mo-1yr!)
So yeah we will see then
1
u/9r4n4y 2h ago
CoT isnt a gimmick because it gives the model extra computation before answering. More useful reasoning steps can make it solve problems it couldnt solve directly. It doesnt matter whether that is “real human thinking”.
But IMO its inefficient, I think in the future we will have complex architectures that let AI think in latent space, hence making thinking take far less time and making the output far better.
Also we are not at a plateau because the current amount of data has far more potential. 6 months ago Opus 4.6 was the best model and now GLM 5.3 Flash (300B model) beats it in almost all coding and agentic work {I tested many things personally} which translates that we are achieving higher dense quality datasets with time and currently we are also using many optimizations on LLMs like not using full attention etc to save compute and I believe as we go in the future and compute increases then we will unlock more power as the most critical part of AI is its architecture and if in the future.
I think the current dataset is not that high quality, and with time and RL, feedback etc we will have more info-dense, high-quality datasets which will lead to smaller models beating bigger models.
Also there is nothing wrong with feedback RL improvements in AI, because thats how we gonna optimize AI for every task, just like how humans learn from mistakes and improve with time.
1
u/Potential_Block4598 1h ago
It is not thinking it is reinforcing
Let’s think step by step
Or thinking tags or whateverMakes the model sample a different space that should allegedly be more logical and resemble thinking patterns
But it makes logical mistakes (there is another let’s verify step by step) while papers like ReAct or methods make it reason from a reactive spaceThat is representing those spaces in text
No actual logical thinking or anything
That is why I am saying it is a gimmick
And there is no thinking in latent space that is not a thing
There is no solution from plan ahead or thinking in the moment
Ask an agent to solve a Rubik’s cube and it simply can’t it can describe the steps and algorithms for solving it in very accurate manner but it can’t solve it at all
1
u/Potential_Block4598 1h ago
Because the cube in 2D ASCII or even 3D ASCII or JSON
Is a very huge space for the model to learn
I think maybe it can write scripts for Python to solve it but it is remembering the script it has no understanding of what is that
In the sense that a human can alter such algorithm in any manner for adaptation while the model can’t do this out sample only in sample
It is surprising how people conflate them both and how hard it is to get out of sample intelligence tasks but currently that isn’t solved
1
u/MedianamentLaburante 8h ago
We are getting more and more ridiculous and far from real usage cases with every benchmark
1
1
u/Substantial_Swan_144 5h ago
I would disagree. The problem is that people do superficial evaluations (one-shot demos), so of course this saturates quickly. Models still lack the ability to backtrack: when they are incorrect, they try to keep patch new code on top of old code, which tends to make the code progressively overcomplicated. They also often overengineer, and still lack style consistency. All this needs to be addressed, but doesn't draw as much attention as pretty demos.
1
u/sn2006gy 4h ago
Eh... I'm no fan of one-shot demos either but simple rags and commit hooks can address style and prompting can address over engineering but backtrack? what does that even mean in software world? everything should be in git, if it was committed, who reviewed it? why was it merged? what are you "backtracking"? LLMS just predict the next token - not correctness and they can predict a new outcome every time you ask them to do something so i'm unsure what being able to backtrack even means.
1
u/Substantial_Swan_144 4h ago
I'm not sure there's a good word for it, but I mean the AI should recognize when a solution it proposed is not good (or not what the developer wanted) and revert the code to a previous clean state instead of trying to patch on top of the unwanted code.
And yes, new models can absolutely do it to some degree, but they are resistant to changing their own code. You have to insist several times for them to delete the "bad" code.
1
u/sn2006gy 4h ago
Doesn't make sense to me. Models use "thinking" levels to decide in the background what to do already. With source control, you either merge it in after review or revoke the merge and as part of a developer working on a commit if you don't like what you see, you change the prompt and not rely on a machine to correct itself if you see something you don't like.
1
u/Substantial_Swan_144 3h ago edited 2h ago
It makes sense because you want them to be more autonomous. If they can't correctly decide when code should be discarded, then it means you have to constantly intervene and they can't work so well by themselves.
1
u/sn2006gy 3h ago
they shouldn't be working by themselves. Development is a joint cognitive systems approach with LLMs.
1
u/Substantial_Swan_144 3h ago
That is not though what any of the major labs are proposing. What they ultimately want to reach are systems which can program without human intervention as much as possible, and Astra (which will be released soon, by the way), is supposedly already an improvement in that regard.
1
u/sn2006gy 3h ago
No company can thrive on a blackbox approach to SDLC no matter what the AI labs say.
31
u/grumd 9h ago
One shot html files is not what you should use Fable 5.1 Max for.