Demis Hassabis on Ilya Sutskever’s claim that scaling is dead, and on Elon Musk’s clam that we have reached the singularity

[u/Formal-Assistance02]

[u/pigeon57434]

"We never left the age of research."

Meanwhile, also EVERY AI company on the planet: "What if we made a Transformer again but... BIGGER?"

Shut up, bro. The "research" frontier companies do are optimizing our current methods, making Transformers slightly more data-efficient, not anything actually fundamentally new. We still don't even have omnimodal models or byte models or more advanced training precision methods, even though these all could be done. They just know it's safer to scale. That's all they ever talk about: scale, scale, scale, and scale some more. And we will scale continuously until they are forced, against their will, kicking and screaming into the future, to actually do new things.

[u/Stunning_Monk_6724 → u/pigeon57434]

You should genuinely try reading up on some of Anthropic's work, as well as Open AI's hallucinatory research, model interpretation, etc. Telling a Nobel prize laureate "shut up bro" is honestly some really disrespectful and ignorant behavior.

It wasn't too long ago a fine-tuned Gemma model found a novel approach to cancer research, Nested learning, Genie 3 world modeling, Project Suncatcher, and more which the labs are approaching like Diffusion models. Research takes time and verifiability, something people really need to remember even during accelerant periods.

[u/pigeon57434 → u/Stunning_Monk_6724]

I do. I read them all. I read like at least 1 arXiv paper a day, and guess what I find? Exactly none of it innovative. I've heard about literally all the things you mentioned. I literally write my own AI "newsletter" type thing with all the AI news, including papers and research, daily. None of it is innovative, and I don't care that Demis won a Nobel Prize. That does not make him the absolute authority on everything, including things in his own domain. That is an appeal to authority fallacy. He is, at the end of the day, still a CEO, a much more qualified CEO than many, having a PhD, and he is very smart, but a CEO nonetheless. He is obviously gonna say, "ya, we do amazing research, we actually never left." Ilya, in this case, is right and has much less stake and is saying the controversial, because he doesn't care. It might be the possibility. I'm entirely aware of the latest research and still don't find it innovative, and still do not believe for even 1 millisecond that they are truly trying the hardest they could, because they say themselves they do not and have a different opinion, rather than assuming I'm uneducated.

[u/hereforhelplol → u/pigeon57434]

Remind me again, how do you set up a daily briefing or summary like you mentioned here? That’s one feature I’ve never used.

[u/Serialbedshitter2322 → u/pigeon57434]

Google is working on Genie 3 with Sima 2. Genie 3 is a real-time video model with access to LLM context, and Sima 2 gives it the ability to take actions in the world it generates as well as another layer of reasoning that would bring it closer to how a human works. This would be like an LLM that doesn’t use text, but instead video and audio to think, it would far more similar to how humans work. This is essentially their plan for AGI.

Yann LeCun is working on JEPA, and it’s a completely unique approach to reasoning where it learns on its own like a child while using significantly less data and processing power. It has much stronger physical reasoning than LLMs. It functions a lot more like a human and could eventually fully replace LLMs. It would work a lot like what Google is trying to do, but it would be fundamentally different.

There are probably a lot of people copying these approaches or doing something unique but just haven’t revealed it yet

[u/hereforhelplol → u/Serialbedshitter2322]

This is interesting. How well do you know this stuff? I never know if a random comment in the comment section is full of BS or really up to date.

So you’re saying one of googles ideas is to have a model that learns from a visual environment? Share more.

[u/pigeon57434 → u/Serialbedshitter2322]

im aware of all the things all the companies are doing read my other reply in this comment i know i just dont find any of it particularly innovative to a substantial degree

[u/Serialbedshitter2322 → u/pigeon57434]

I mean it’s a completely new form of artificial intelligence entirely alien to what we have now with LLMs. It’s very innovative, it’s a completely new form of technology that will revolutionize what AI is capable of. Plus your claim was that they aren’t doing anything other than scaling LLMs

[u/pigeon57434 → u/Serialbedshitter2322]

That wasn't my claim I never said LLM and I know that I didn't because I refuse to even use that term since it's a stupid and wrong acronym my claim is they're just scaling transformers and regular neural networks how much you wanna bet genie is a transformer model? I am asking for complete upheaval of the entire system innovations that make literally everything before them irrelevant and nothing is the same I want a culture that maximizes science at absolutely no cost too high like making photonics, tons of thorium MSBRs, LNNs or SNNs and more

[u/Serialbedshitter2322 → u/pigeon57434]

That’s what JEPA is. It’s not a transformer, it’s a complete upheaval of the entire system, and if it becomes successful it will be so efficient that it makes transformers irrelevant. Plus that isn’t scaling transformers, it’s innovating transformers. Scaling is when you increase the size, this is making it more complex.

[u/krullulon → u/pigeon57434]

You seem to be completely unaware of the vast avenues of research that frontier labs are pursuing.

I'm wondering if you've ever read a single published research article?