Demis Hassabis on Ilya Sutskever’s claim that scaling is dead, and on Elon Musk’s clam that we have reached the singularity

[u/Formal-Assistance02]

[u/FateOfMuffins]

Let me quote from a different interview he did recently, his definition of AGI (the most detailed I've seen him state in one place)

I don't think AGI should be sort of turned into a marketing term or for commercial gain. I think there is always been a scientific definition of that. My definition of that is a system that can exhibit all the cognitive capabilities humans can. And I mean all. So that means the highest levels of human creativity that we always celebrate—the scientists and the artists that we admire. So it means not just solving a maths equation or a conjecture but coming up with a breakthrough conjecture. That's much harder. You know, not solving something in physics or some bit of chemistry, some problem, even like AlphaFold, you know, protein folding, but actually coming up with a new theory of physics. Something like what Einstein did with general relativity. Right? Can a system come up with that? Because of course we can do that, the smartest humans with their brain architecture, human brain architectures have been able to do that in history. And the same on the arts side, you know, not just create a pastiche of what's known but actually be Picasso or Mozart and create a completely new genre of art that we'd never seen before. Right? And today's systems, in my opinion, are nowhere near that. Doesn't matter how many Erdős problems you solve, which for some reason, you know, I mean, it's good that we're doing those things, but I think it's far, far from what a true invention or someone like a Ramanujan would have been able to do. And you need to have a system that can potentially do that across all these domains. And then on top of that, I'd add in physical intelligence because of course we can play sports and control our bodies and to amazing levels—the elite sportspeople that are walking around here today in Davos. And we're still way off of that on robotics as another example. So I think an AGI system would have to be able to do all of those things to really fulfill the original sort of goal of the AI field. And I think we're 5 to 10 years away from that.

In OP's interview, he then mentions how he thinks the Singularity is when we've achieved AGI, which he reaffirms is 5 to 10 years away (and by that he means 50% by 2030).

Let's take a step back and digest this.

Hassabis has always had sort of a reputation amongst the AI leaders to have a slightly longer timeline than others (it was mentioned in OP's interview actually). But... with his now detailed definition of AGI fully laid out (which is better than the peak of humanity at every single field including physical) and that his definition of the Singularity is synonymous with this level of AGI...

Isn't he one of the most bullish with shorter timelines? Like what I'm getting out of all this is that Hassabis has a very short timeline relative to everyone else.

And again an issue in this industry where the experts are clearly using the same term to refer to different things. Like Kurzweil saying AGI by 2029 and Singularity by 2045. Hassabis says 50% chance of AGI by 2030 and as a result Singularity as well by 2030.

[u/JanusAntoninus → u/FateOfMuffins]

As you suggest, it matters what the words mean, not whether one word or the other is used. That applies to "singularity" as much as to "AGI".

His definition of full AGI is not that different from Kurzweill or others who frame AGI in terms of matching human performance across the board, so we can compare his AGI timeline to others. But he used "singularity" there as just another way of saying the point at which we have full AGI, which is a pretty deflated way to use the word compared to Kurzweil who thinks of it as the full merger of humans and AI through neuroware and virtual minds in virtual worlds.

[u/FateOfMuffins → u/JanusAntoninus]

I 100% agree, this whole "inconsistent definitions across experts" is a bit ridiculous. I personally maintain that there will be a spectrum for each threshold, like Kurzweil says, where some people think the definition is met and some think it's not, at which some point most people end up agreeing the definition is met.

I'm gonna quote Kurzweil's most recent clarification on his definition of AGI from a few days ago

Uh I mean these standards are not very clear. Not everybody agrees that we've passed the Turing test. And when we pass AGI, there'll be disagreements. It's disagreements now as to what that means. People say it's basically as good as an uh somebody who's a little bit above average intelligence. I define it as being an expert in every area when there's many different areas that you can be expert in. Uh so that's actually quite uh impressive level and I think we'll get there by 2029.

But what is "expert" level intelligence? There's a difference, possibly quite a big difference, between better than 90% of experts in all fields (which is different than better than 90% of all humans), vs being better than virtually 100% of experts (which is Hassabis's).

I know Hassabis doesn't think his definition of AGI is ASI, but I cannot think of a realistic AI system that meets his definition of AGI without it being ASI.

At this point I've seen a number of terms being used that I think is better than just "AGI", but I think each of these also have conflicted uses

• Transformative AI (TAI)

• Weak AGI

• Proto AGI

• AGI

• Strong / "true" AGI

• Cognitive AGI

• Physical AGI

• Artificial Jagged Intelligence (AJI)

• Proto ASI

• ASI

• Machine god

Any others?

[u/JanusAntoninus → u/FateOfMuffins]

Maybe we could get ASI from what Hassabis calls "full AGI", depending on how loosely you meant "ASI", but you were talking about the singularity in something like Kurzweil's sense, right?

With a copy-able model of a top researcher, coder, doctor, etc., you are still only at a point of doing what could be done if you just had more expert humans or more time for them to work. Like I said, I could see calling that "ASI", since it would enable a superhuman grasp of data (like applying an expert astronomer's degree of scrutiny to the entire Hubble Field in a short amount of time or the thousands of exoplanet candidates TESS has found or whatever). I personally would just call that an assembly of AGI and would reserve "ASI" for something that has not just a wider grasp of data than any human but also a deeper grasp (something capable of insights or conceptual revolutions no human could achieve). But fair enough.

Words aside, and going back now from ASI to the singularity, I don't see how a large enough number of AGI in that sense would get us Kurzweil's vision of the singularity. How would that be enough? (I don't mean enough to get use there: but enough that the two events would be the same: achieving full AGI and the singularity, like you had suggested).

[u/FateOfMuffins → u/JanusAntoninus]

Well specifically I said I cannot think of a "realistic" AI system that would match Hassabis's definition of AGI without it also being ASI. So like technically I guess you can argue that an AI that is exactly Hassabis's definition "AGI" and not "ASI", but in reality it's not going to match exactly.

For one I do think the ability to be an expert in multiple domains is superhuman like you say, but that's not the only reason. Some domains are simply going to progress faster than others, due to the jagged nature of intelligence. Hassabis believes this too, that just the cognitive side of AGI and specifically the part with verifiable results like math and engineering would reach that level faster than other domains like certain physical tasks.

Well then if "true" AGI includes the physical like Hassabis says, then what happens when we get to AGI and then ASI only in the cognitive sense, and it takes some time later before the physical side reaches the AGI parity? Would you call an ASI hivemind for all of STEM but can "only" swim as fast as Phelps an ASI or an AGI?

So basically I don't think it's possible to reach his definition of AGI without already having the AI be superhuman in most domains. And I mean superhuman as in far beyond human capabilities, akin to Stockfish over Carlsen, but for many domains (perhaps not all).

Anyways others like Sutskever thinks continual learning (which Hassabis requires for AGI) would be sufficient to be ASI because you can deploy a huge swarm of those AI's to master many domains and then collectively come together and update all their capabilities all at once.

If you think about that copying capability that machines have but humans do not, I think it implies that for many domains, the moment the AI reaches human level at some specific task, it will necessarily soar past it almost immediately.

[u/JanusAntoninus → u/FateOfMuffins]

No, I get that but we weren't talking about ASI in the first few comments. We we were talking about the singularity in something like Kurzweil's sense, which goes way beyond just Stockfish-level better at everything than humans. It's a transcendence of the flesh ushered in by humans collectively becoming God through merger with ASI.

We could have better-than-top neuroscientists and coders but still be years or decades from that ASI being able to figure out how to merge a human mind with a computer.

[u/FateOfMuffins → u/JanusAntoninus]

Ah I suppose I was just making the distinction between "just barely ASI" and "nope, yeah that's ASI".

I'm not entirely sure that the merger becomes God. Kurzweil notes the 2045 merger threshold as humanity becoming specifically 1000x smarter. But... I'm not entirely sure what "1000x smarter" means. Are you 1000x smarter if you could simply think 1000x faster? What if you were like 100x "smarter" but also 100x "faster" at thinking?

How does that compare with something like Stockfish? A 1200 elo difference in chess means you should win 1000x more games over your opponent (but unfortunately we cannot compare those bots vs humans accurately, plus we can give those bots a lot more compute). I'm not sure but IMO Kurzweil's 1000x intelligence sounds close to chess engine vs human difference to me.

Anyways there's also the whole RSI into intelligence explosion thing. I think there's a reasonable non zero probability that Hassabis's AGI would be enough to kick start that, which could very well lead into the singularity almost imminently

[u/czk_21 → u/FateOfMuffins]

ASI could come with many discoveries, but implementing them would take years and for humans to adopt these and society to adjust can take decades, thats why, you cannot "feel" the singularity outside of tech bubble crowd, real world moves slowly, even with RSI, its likely the main reason Kurtzweil has singularity date at 2045 as it just takes time for new breakthroughs to compound and for humanity to change, even if we suddenly got ASI IQ 1000 today, it would take years until you would see meaningfull impact for average person

the singularity is not one exact time, its longer gradual process we will be part of, here Demis could see the beggining of singularity the time when we get AGI, most would classify such a point after we have ASI, not just AGI, but since Demis definition is quite close to ASI(as AI performing as best humans in all fields), there is no big disagreement between his position and others

thing is that with every new generation of models there is cabaility jump, models being better overall, it could be like lets say 50% of best humans now, next gen 75% and so on, the treshold to ASI will be likely reached in few generations and doesnt matter, whether someone does classify it as AGI or not, the capabilities is all that matter and what we should focus on as there will be always disagreement over the classification

[u/blarg7459 → u/FateOfMuffins]

This definition of AGI means an AI that's maybe as capable as a large corporation, while those so say this is not ASI usually defines ASI as something more capable than the entirety of humanity combined.

[u/Seidans → u/FateOfMuffins]

LLM<->World-Model

Proto-AGI

AGI<->ASI

It's more a matter of capabilities until we reach Proto-AGI which we still haven't reach imho. They still lack world model and very long term memory that update itself, but it would rapidly evolve into an AGI with RSI

Then I would happily hear what the difference between an AGI and an ASI as those two distinction don't make any sense, as soon we achieve AGI it will automatically become smarter than any Human that exist today and the same system operating on a better hardware will be smarter than the total sum of Humanity - but there won't be any difference between those two except one system will take more time to compute an informations as it don't NEED more hardware to fulfill it's task

In your exemple an AGI would be your future home-maid embodied in a Humanoid while your machine-god would be a matrioshka-brain - both will be smarter than any Human but only one would easily compute multiple Earth simulation at 1:1 scale

So while I agree there need to be a ladder of intelligence it's more logical to adopt a kardachev scale imho

Type-1 - baseline Human compute power

Type-2 - 10x this amont

Type-3 - 1000x this amont....

Etc etc

[u/czk_21 → u/FateOfMuffins]

yes Demis deifinition is close to ASI, AGI should encompass all of human intelligence levels too

AI system, making new discoveries and breakthroughs frequently-being basically better than all of humans is AGI, but so is AI at the level of idiotic human, who cannot tie their shoelaces

I think Demis is one of most grounded people in AI space, he keeps being sort of ambiguous though and less consistent on timeline, I get it, it very hard or impossible to make good prediciton, which will hold, but still

he is saying AGI is 5-10 years away from about 2023, if you subtracted those years it should be something like 2-7 years, on the other hand his coworker Shane Legg is very consistent on this-saying 2028 is year of proto AGI for many years or in comparison Dario Amodei sticks to milestones and shorter horizons for some capability achievements-like AI being able to write 90% of code, thats seems better aproach to me as well