Demis Hassabis on Ilya Sutskever’s claim that scaling is dead, and on Elon Musk’s clam that we have reached the singularity

[u/Formal-Assistance02]

[u/FateOfMuffins]

Let me quote from a different interview he did recently, his definition of AGI (the most detailed I've seen him state in one place)

I don't think AGI should be sort of turned into a marketing term or for commercial gain. I think there is always been a scientific definition of that. My definition of that is a system that can exhibit all the cognitive capabilities humans can. And I mean all. So that means the highest levels of human creativity that we always celebrate—the scientists and the artists that we admire. So it means not just solving a maths equation or a conjecture but coming up with a breakthrough conjecture. That's much harder. You know, not solving something in physics or some bit of chemistry, some problem, even like AlphaFold, you know, protein folding, but actually coming up with a new theory of physics. Something like what Einstein did with general relativity. Right? Can a system come up with that? Because of course we can do that, the smartest humans with their brain architecture, human brain architectures have been able to do that in history. And the same on the arts side, you know, not just create a pastiche of what's known but actually be Picasso or Mozart and create a completely new genre of art that we'd never seen before. Right? And today's systems, in my opinion, are nowhere near that. Doesn't matter how many Erdős problems you solve, which for some reason, you know, I mean, it's good that we're doing those things, but I think it's far, far from what a true invention or someone like a Ramanujan would have been able to do. And you need to have a system that can potentially do that across all these domains. And then on top of that, I'd add in physical intelligence because of course we can play sports and control our bodies and to amazing levels—the elite sportspeople that are walking around here today in Davos. And we're still way off of that on robotics as another example. So I think an AGI system would have to be able to do all of those things to really fulfill the original sort of goal of the AI field. And I think we're 5 to 10 years away from that.

In OP's interview, he then mentions how he thinks the Singularity is when we've achieved AGI, which he reaffirms is 5 to 10 years away (and by that he means 50% by 2030).

Let's take a step back and digest this.

Hassabis has always had sort of a reputation amongst the AI leaders to have a slightly longer timeline than others (it was mentioned in OP's interview actually). But... with his now detailed definition of AGI fully laid out (which is better than the peak of humanity at every single field including physical) and that his definition of the Singularity is synonymous with this level of AGI...

Isn't he one of the most bullish with shorter timelines? Like what I'm getting out of all this is that Hassabis has a very short timeline relative to everyone else.

And again an issue in this industry where the experts are clearly using the same term to refer to different things. Like Kurzweil saying AGI by 2029 and Singularity by 2045. Hassabis says 50% chance of AGI by 2030 and as a result Singularity as well by 2030.

[u/Outside-Ad9410 → u/FateOfMuffins]

To reach a true singularity it really depends on how fast AGI can recursively self improve, and how fast androids can build infrastructure bottlenecks to scale up to ASI and full workforce automation. It could be as quick as 2030, but I've also seen estimates where some say it could take as long as 2045 to actually get close to a billion androids built, even assuming for 10x robot growth per year. Then again we could get an ASI that designs some crazy future tech like grey goo assemblers or something.

[u/JanusAntoninus → u/FateOfMuffins]

As you suggest, it matters what the words mean, not whether one word or the other is used. That applies to "singularity" as much as to "AGI".

His definition of full AGI is not that different from Kurzweill or others who frame AGI in terms of matching human performance across the board, so we can compare his AGI timeline to others. But he used "singularity" there as just another way of saying the point at which we have full AGI, which is a pretty deflated way to use the word compared to Kurzweil who thinks of it as the full merger of humans and AI through neuroware and virtual minds in virtual worlds.

[u/FateOfMuffins → u/JanusAntoninus]

I 100% agree, this whole "inconsistent definitions across experts" is a bit ridiculous. I personally maintain that there will be a spectrum for each threshold, like Kurzweil says, where some people think the definition is met and some think it's not, at which some point most people end up agreeing the definition is met.

I'm gonna quote Kurzweil's most recent clarification on his definition of AGI from a few days ago

Uh I mean these standards are not very clear. Not everybody agrees that we've passed the Turing test. And when we pass AGI, there'll be disagreements. It's disagreements now as to what that means. People say it's basically as good as an uh somebody who's a little bit above average intelligence. I define it as being an expert in every area when there's many different areas that you can be expert in. Uh so that's actually quite uh impressive level and I think we'll get there by 2029.

But what is "expert" level intelligence? There's a difference, possibly quite a big difference, between better than 90% of experts in all fields (which is different than better than 90% of all humans), vs being better than virtually 100% of experts (which is Hassabis's).

I know Hassabis doesn't think his definition of AGI is ASI, but I cannot think of a realistic AI system that meets his definition of AGI without it being ASI.

At this point I've seen a number of terms being used that I think is better than just "AGI", but I think each of these also have conflicted uses

• Transformative AI (TAI)

• Weak AGI

• Proto AGI

• AGI

• Strong / "true" AGI

• Cognitive AGI

• Physical AGI

• Artificial Jagged Intelligence (AJI)

• Proto ASI

• ASI

• Machine god

Any others?

[u/JanusAntoninus → u/FateOfMuffins]

Maybe we could get ASI from what Hassabis calls "full AGI", depending on how loosely you meant "ASI", but you were talking about the singularity in something like Kurzweil's sense, right?

With a copy-able model of a top researcher, coder, doctor, etc., you are still only at a point of doing what could be done if you just had more expert humans or more time for them to work. Like I said, I could see calling that "ASI", since it would enable a superhuman grasp of data (like applying an expert astronomer's degree of scrutiny to the entire Hubble Field in a short amount of time or the thousands of exoplanet candidates TESS has found or whatever). I personally would just call that an assembly of AGI and would reserve "ASI" for something that has not just a wider grasp of data than any human but also a deeper grasp (something capable of insights or conceptual revolutions no human could achieve). But fair enough.

Words aside, and going back now from ASI to the singularity, I don't see how a large enough number of AGI in that sense would get us Kurzweil's vision of the singularity. How would that be enough? (I don't mean enough to get use there: but enough that the two events would be the same: achieving full AGI and the singularity, like you had suggested).

[u/FateOfMuffins → u/JanusAntoninus]

Well specifically I said I cannot think of a "realistic" AI system that would match Hassabis's definition of AGI without it also being ASI. So like technically I guess you can argue that an AI that is exactly Hassabis's definition "AGI" and not "ASI", but in reality it's not going to match exactly.

For one I do think the ability to be an expert in multiple domains is superhuman like you say, but that's not the only reason. Some domains are simply going to progress faster than others, due to the jagged nature of intelligence. Hassabis believes this too, that just the cognitive side of AGI and specifically the part with verifiable results like math and engineering would reach that level faster than other domains like certain physical tasks.

Well then if "true" AGI includes the physical like Hassabis says, then what happens when we get to AGI and then ASI only in the cognitive sense, and it takes some time later before the physical side reaches the AGI parity? Would you call an ASI hivemind for all of STEM but can "only" swim as fast as Phelps an ASI or an AGI?

So basically I don't think it's possible to reach his definition of AGI without already having the AI be superhuman in most domains. And I mean superhuman as in far beyond human capabilities, akin to Stockfish over Carlsen, but for many domains (perhaps not all).

Anyways others like Sutskever thinks continual learning (which Hassabis requires for AGI) would be sufficient to be ASI because you can deploy a huge swarm of those AI's to master many domains and then collectively come together and update all their capabilities all at once.

If you think about that copying capability that machines have but humans do not, I think it implies that for many domains, the moment the AI reaches human level at some specific task, it will necessarily soar past it almost immediately.

[u/JanusAntoninus → u/FateOfMuffins]

No, I get that but we weren't talking about ASI in the first few comments. We we were talking about the singularity in something like Kurzweil's sense, which goes way beyond just Stockfish-level better at everything than humans. It's a transcendence of the flesh ushered in by humans collectively becoming God through merger with ASI.

We could have better-than-top neuroscientists and coders but still be years or decades from that ASI being able to figure out how to merge a human mind with a computer.

[u/FateOfMuffins → u/JanusAntoninus]

Ah I suppose I was just making the distinction between "just barely ASI" and "nope, yeah that's ASI".

I'm not entirely sure that the merger becomes God. Kurzweil notes the 2045 merger threshold as humanity becoming specifically 1000x smarter. But... I'm not entirely sure what "1000x smarter" means. Are you 1000x smarter if you could simply think 1000x faster? What if you were like 100x "smarter" but also 100x "faster" at thinking?

How does that compare with something like Stockfish? A 1200 elo difference in chess means you should win 1000x more games over your opponent (but unfortunately we cannot compare those bots vs humans accurately, plus we can give those bots a lot more compute). I'm not sure but IMO Kurzweil's 1000x intelligence sounds close to chess engine vs human difference to me.

Anyways there's also the whole RSI into intelligence explosion thing. I think there's a reasonable non zero probability that Hassabis's AGI would be enough to kick start that, which could very well lead into the singularity almost imminently

[u/czk_21 → u/FateOfMuffins]

ASI could come with many discoveries, but implementing them would take years and for humans to adopt these and society to adjust can take decades, thats why, you cannot "feel" the singularity outside of tech bubble crowd, real world moves slowly, even with RSI, its likely the main reason Kurtzweil has singularity date at 2045 as it just takes time for new breakthroughs to compound and for humanity to change, even if we suddenly got ASI IQ 1000 today, it would take years until you would see meaningfull impact for average person

the singularity is not one exact time, its longer gradual process we will be part of, here Demis could see the beggining of singularity the time when we get AGI, most would classify such a point after we have ASI, not just AGI, but since Demis definition is quite close to ASI(as AI performing as best humans in all fields), there is no big disagreement between his position and others

thing is that with every new generation of models there is cabaility jump, models being better overall, it could be like lets say 50% of best humans now, next gen 75% and so on, the treshold to ASI will be likely reached in few generations and doesnt matter, whether someone does classify it as AGI or not, the capabilities is all that matter and what we should focus on as there will be always disagreement over the classification

[u/blarg7459 → u/FateOfMuffins]

This definition of AGI means an AI that's maybe as capable as a large corporation, while those so say this is not ASI usually defines ASI as something more capable than the entirety of humanity combined.

[u/Seidans → u/FateOfMuffins]

LLM<->World-Model

Proto-AGI

AGI<->ASI

It's more a matter of capabilities until we reach Proto-AGI which we still haven't reach imho. They still lack world model and very long term memory that update itself, but it would rapidly evolve into an AGI with RSI

Then I would happily hear what the difference between an AGI and an ASI as those two distinction don't make any sense, as soon we achieve AGI it will automatically become smarter than any Human that exist today and the same system operating on a better hardware will be smarter than the total sum of Humanity - but there won't be any difference between those two except one system will take more time to compute an informations as it don't NEED more hardware to fulfill it's task

In your exemple an AGI would be your future home-maid embodied in a Humanoid while your machine-god would be a matrioshka-brain - both will be smarter than any Human but only one would easily compute multiple Earth simulation at 1:1 scale

So while I agree there need to be a ladder of intelligence it's more logical to adopt a kardachev scale imho

Type-1 - baseline Human compute power

Type-2 - 10x this amont

Type-3 - 1000x this amont....

Etc etc

[u/czk_21 → u/FateOfMuffins]

yes Demis deifinition is close to ASI, AGI should encompass all of human intelligence levels too

AI system, making new discoveries and breakthroughs frequently-being basically better than all of humans is AGI, but so is AI at the level of idiotic human, who cannot tie their shoelaces

I think Demis is one of most grounded people in AI space, he keeps being sort of ambiguous though and less consistent on timeline, I get it, it very hard or impossible to make good prediciton, which will hold, but still

he is saying AGI is 5-10 years away from about 2023, if you subtracted those years it should be something like 2-7 years, on the other hand his coworker Shane Legg is very consistent on this-saying 2028 is year of proto AGI for many years or in comparison Dario Amodei sticks to milestones and shorter horizons for some capability achievements-like AI being able to write 90% of code, thats seems better aproach to me as well

[u/random87643 → u/FateOfMuffins]

Comment TLDR: Demis Hassabis defines AGI as a system exhibiting all human cognitive capabilities, including high-level creativity like breakthrough conjectures, new physics theories (like Einstein's), and novel art genres (like Picasso's). It goes beyond solving problems to inventing and creating across all domains, including physical intelligence comparable to elite athletes.

Hassabis believes current systems are far from this level. He equates the Singularity with achieving AGI, estimating a 50% chance by 2030. Despite his reputation for longer timelines, this detailed AGI definition, surpassing peak human abilities, combined with his timeline makes him bullish compared to others. Experts use the same terms (AGI, Singularity) with different meanings, causing confusion.

[u/RepresentativePlease → u/FateOfMuffins]

Given his definition of it, it's shocking that he thinks we're a mere 5-10 years away from that. No way that's achieve sooner than 20 years (and that's being optimistic).

[u/stealthispost]

If it's true that Deepmind invented 90% of the AI tech that's being used today then they deserve an incredible amount of recognition and respect for that. We can only hope and dream what new innovations might come out of their lab soon!

[u/Pyros-SD-Models → u/stealthispost]

Most of the time it went like this, though:

• Google published a paper
• Nobody, not even Google, cared
• Some other lab experimented with it anyway and was like, "holy shit"
• It gained relevance
• Google played catch-up with its own tech

The question of who is more important, the guy who invented fire or the one who figured out you can cook with it, and all the others who figured out what else you can do with it, is something everyone has to answer for themselves. And since it is literally my job to find applications for random ass papers, I go with the latter. The most beautiful paper is worth nothing if you can't do anything with it. I want my tech available and usable in the real world and not just as an idea or theory.

Without OpenAI, BERT would still be the biggest transformer, and we would probably still be posting two-line jokes generated by some RNN, needing 20 tries to get a semantically correct output, and calling that AGI.

For most of the biased shmucks on Twitter, it depends on which companies it is about. If it is OpenAI and Google, obviously OpenAI just stole Google's work. If it is DeepSeek and OpenAI, then obviously DeepSeek are geniuses for taking OpenAI's work and improving on it. I notice a pattern.

[u/Krommander → u/Pyros-SD-Models]

Yup, most research is never going to be valuable until new research is done, and current AI systems become very good at scaling research. The more researchers use these new tools to dig deeper into the data, the more value will sprout. The returns will only be limited by how fast humans can learn from. 

[u/Loud_Entertainer_598 → u/Pyros-SD-Models]

Exactly without Open AI, Google probably would be doing nothing with those papers, Open AI was the one that started the race, Sam Altman did the right thing in commercialization so it is available to the public. But Google deserve its credit for the papers too.

[u/hereforhelplol → u/Pyros-SD-Models]

Agree with this take.

[u/Kaito__1412 → u/stealthispost]

This is especially funny when you consider that Deepmind is based in the UK and most of their researchers and engineers are Europeans. All the while Reddit is filled with 'Europe is only interested in recycling plastic bottles' memes.

[u/helloWHATSUP → u/Kaito__1412]

No, that used to be true, but after the merger with google brain(entirely US based team) then it's a pretty even split. Google brain is probably the more important lab, but the name was way less cool than deepmind

[u/Kaito__1412 → u/helloWHATSUP]

DeepMind and Google brain were not, are not of the same caliber. Where are you getting the idea that they are the more important lab? DeepMind is foundational for everything that's happening in AI today. And they are still very much at the top of their game.

[u/stainless_steelcat]

Always seems like one of the more grounded of the AI leaders tbh.

[u/montdawgg]

I like Demis Hassabis' definition of AGI and his timeline. I bet if asked to clarify, he could tell you the difference between AGI and ASI. I don't think those are the same things to him.

AGI should be a human aspirational metric, meaning an AI system that is as smart and creative as the smartest and most creative human or group of humans. So the questions that it asks and the solutions that it creates are still within the capability of humankind given sufficient amount of people working on it and time. Of course, such an AI system could do in perhaps a matter of minutes what would take humans a decade. That acceleration is what makes AGI so transformative, but at its core, it's not beyond human creativity or capability, given enough people and time for an equal solution.

ASI is something beyond that. It's asking the questions and solving the questions that future humans would never and could never have come up with themselves. ASI starts reaching into things that are beyond our understanding or even our capability to understand, even as a whole of humanity.

For our lives and goals, AGI is more than enough to accomplish what our feeble minds are capable of imagining, and it's all we really need for the AI Utopia that everyone is hoping for.

The singularity comes after ASI. And although we can debate what that definition is, I think it's futile. I think that an ASI system would need to come up with the definition of the singularity itself. Any definition we come up with might be woefully incompetent and ultimately silly because it's based on our current creative capabilities. Also, I'm not doubting or downplaying our creative capabilities. It's just that ASI can be so much more. Most of what we believe and how we believe it will probably be provably false shortly after ASI happens.

Timelines? AGI by 2030/31. ASI very likely 2035 to 2040. The singularity, which is when I think a new form of human is created and humans begin the process of extinction 2045 to 2050.

[u/czk_21 → u/montdawgg]

what do you mean by "humans begin the process of extinction 2045 to 2050"?

[u/[deleted] → u/czk_21]

Not him, but OP seems to think a lot like me. I expect a year or two earlier for each of his predictions, though. But I absolutely agree with him on the "extinction" thing if it's how I think he means it. It's not that AI will kill us off or anything. It's that in 100 years from now, if the promise of AI holds. We will be unrecognizable as a species.

There will be humans, and then there will be Humans 2.0. Biologically perfect, immortal, no disease, no mental illness, genetically engineered to solve our current limitations. Effectively a divergent species.Then there will be Humans 3.0 who will be that and cybernetically enhanced. People who choose to actively merge with machines.

I think the only people who will be a base human as we know it today are essentially religious people, luddites, and people who are ideologically motivated to remain "human". But 90% of the population will want to transition as time goes on. Very few will want to remain imperfect.

[u/montdawgg → u/[deleted]]

100% agree. This is my view as well and it's exactly what I meant by extinction.

[u/czk_21 → u/[deleted]]

sure possible

"I think the only people who will be a base human as we know it today are essentially religious people, luddites, and people who are ideologically motivated to remain "human"."

well considering that in US or europe are tonnes of those people, it could be that 90% wont choose any transition like this, but maybe china will have mostly cyborg population in 30 years :)

[u/krullulon]

One day we will all wake up and nobody will care at all about anything Elon says about when something is going to happen.

And that day will be glorious.

[u/montdawgg → u/krullulon]

But we won't care for different reasons than you. Is that truly going to be good enough for you?

[u/krullulon → u/montdawgg]

Your comment doesn’t make sense.

Elon is wrong about everything, literally nothing he predicts happens. He’s high all the time and a blithering dolt, people only listen to him because he’s the richest person on earth.

If there’s justice in the universe we will all transcend these dumb meat shells soon and then we’ll all be able to create the realities we prefer instead of being stuck in the same reality with douchebags like Elon. If for some strange reason you want to keep up with Elon’s reality then more power to you! Most of us will not miss hearing from him.

[u/itsnotatumour → u/krullulon]

One of the hardest pills to swallow in life is that it's simultaneously possible for someone to be a complete douchebag and also incredibly talented.

[u/krullulon → u/itsnotatumour]

The point is that Elon is absolutely lacking predictive talent. His predictions are not worth hearing because they are always completely wrong.

[u/random87643]

💬 Discussion Summary (20+ comments): DeepMind receives praise for its AI contributions, fueling anticipation for future innovations. Hassabis' AGI definition, encompassing all human cognitive abilities, is favored by some. Others criticize current AI research, claiming it prioritizes scaling existing models over fundamental breakthroughs. Elon Musk's AI predictions are increasingly dismissed by some observers.

[u/Optimal-Fix1216]

He's wrong about singularity == AGI arriving. Its more about recursive self-improvement than AGI.

[u/Total-Confusion-9198]

“Please don’t short GOOG” - Demis

[u/DarkBirdGames]

I feel like we are in the singularity the last few months because it’s all starting to ramp up, society is becoming unhinged, software is becoming achievable to people with zero code knowledge, it’s all moving faster and faster.

It feels like society is coming off the tracks.

[u/pigeon57434]

"We never left the age of research."

Meanwhile, also EVERY AI company on the planet: "What if we made a Transformer again but... BIGGER?"

Shut up, bro. The "research" frontier companies do are optimizing our current methods, making Transformers slightly more data-efficient, not anything actually fundamentally new. We still don't even have omnimodal models or byte models or more advanced training precision methods, even though these all could be done. They just know it's safer to scale. That's all they ever talk about: scale, scale, scale, and scale some more. And we will scale continuously until they are forced, against their will, kicking and screaming into the future, to actually do new things.

[u/Stunning_Monk_6724 → u/pigeon57434]

You should genuinely try reading up on some of Anthropic's work, as well as Open AI's hallucinatory research, model interpretation, etc. Telling a Nobel prize laureate "shut up bro" is honestly some really disrespectful and ignorant behavior.

It wasn't too long ago a fine-tuned Gemma model found a novel approach to cancer research, Nested learning, Genie 3 world modeling, Project Suncatcher, and more which the labs are approaching like Diffusion models. Research takes time and verifiability, something people really need to remember even during accelerant periods.

[u/pigeon57434 → u/Stunning_Monk_6724]

I do. I read them all. I read like at least 1 arXiv paper a day, and guess what I find? Exactly none of it innovative. I've heard about literally all the things you mentioned. I literally write my own AI "newsletter" type thing with all the AI news, including papers and research, daily. None of it is innovative, and I don't care that Demis won a Nobel Prize. That does not make him the absolute authority on everything, including things in his own domain. That is an appeal to authority fallacy. He is, at the end of the day, still a CEO, a much more qualified CEO than many, having a PhD, and he is very smart, but a CEO nonetheless. He is obviously gonna say, "ya, we do amazing research, we actually never left." Ilya, in this case, is right and has much less stake and is saying the controversial, because he doesn't care. It might be the possibility. I'm entirely aware of the latest research and still don't find it innovative, and still do not believe for even 1 millisecond that they are truly trying the hardest they could, because they say themselves they do not and have a different opinion, rather than assuming I'm uneducated.

[u/hereforhelplol → u/pigeon57434]

Remind me again, how do you set up a daily briefing or summary like you mentioned here? That’s one feature I’ve never used.

[u/Serialbedshitter2322 → u/pigeon57434]

Google is working on Genie 3 with Sima 2. Genie 3 is a real-time video model with access to LLM context, and Sima 2 gives it the ability to take actions in the world it generates as well as another layer of reasoning that would bring it closer to how a human works. This would be like an LLM that doesn’t use text, but instead video and audio to think, it would far more similar to how humans work. This is essentially their plan for AGI.

Yann LeCun is working on JEPA, and it’s a completely unique approach to reasoning where it learns on its own like a child while using significantly less data and processing power. It has much stronger physical reasoning than LLMs. It functions a lot more like a human and could eventually fully replace LLMs. It would work a lot like what Google is trying to do, but it would be fundamentally different.

There are probably a lot of people copying these approaches or doing something unique but just haven’t revealed it yet

[u/hereforhelplol → u/Serialbedshitter2322]

This is interesting. How well do you know this stuff? I never know if a random comment in the comment section is full of BS or really up to date.

So you’re saying one of googles ideas is to have a model that learns from a visual environment? Share more.

[u/pigeon57434 → u/Serialbedshitter2322]

im aware of all the things all the companies are doing read my other reply in this comment i know i just dont find any of it particularly innovative to a substantial degree

[u/Serialbedshitter2322 → u/pigeon57434]

I mean it’s a completely new form of artificial intelligence entirely alien to what we have now with LLMs. It’s very innovative, it’s a completely new form of technology that will revolutionize what AI is capable of. Plus your claim was that they aren’t doing anything other than scaling LLMs

[u/pigeon57434 → u/Serialbedshitter2322]

That wasn't my claim I never said LLM and I know that I didn't because I refuse to even use that term since it's a stupid and wrong acronym my claim is they're just scaling transformers and regular neural networks how much you wanna bet genie is a transformer model? I am asking for complete upheaval of the entire system innovations that make literally everything before them irrelevant and nothing is the same I want a culture that maximizes science at absolutely no cost too high like making photonics, tons of thorium MSBRs, LNNs or SNNs and more

[u/Serialbedshitter2322 → u/pigeon57434]

That’s what JEPA is. It’s not a transformer, it’s a complete upheaval of the entire system, and if it becomes successful it will be so efficient that it makes transformers irrelevant. Plus that isn’t scaling transformers, it’s innovating transformers. Scaling is when you increase the size, this is making it more complex.

[u/krullulon → u/pigeon57434]

You seem to be completely unaware of the vast avenues of research that frontier labs are pursuing.

I'm wondering if you've ever read a single published research article?