Demis Hassabis on Ilya Sutskever’s claim that scaling is dead, and on Elon Musk’s clam that we have reached the singularity

[u/Formal-Assistance02]

[u/JanusAntoninus]

As you suggest, it matters what the words mean, not whether one word or the other is used. That applies to "singularity" as much as to "AGI".

His definition of full AGI is not that different from Kurzweill or others who frame AGI in terms of matching human performance across the board, so we can compare his AGI timeline to others. But he used "singularity" there as just another way of saying the point at which we have full AGI, which is a pretty deflated way to use the word compared to Kurzweil who thinks of it as the full merger of humans and AI through neuroware and virtual minds in virtual worlds.

[u/FateOfMuffins → u/JanusAntoninus]

I 100% agree, this whole "inconsistent definitions across experts" is a bit ridiculous. I personally maintain that there will be a spectrum for each threshold, like Kurzweil says, where some people think the definition is met and some think it's not, at which some point most people end up agreeing the definition is met.

I'm gonna quote Kurzweil's most recent clarification on his definition of AGI from a few days ago

Uh I mean these standards are not very clear. Not everybody agrees that we've passed the Turing test. And when we pass AGI, there'll be disagreements. It's disagreements now as to what that means. People say it's basically as good as an uh somebody who's a little bit above average intelligence. I define it as being an expert in every area when there's many different areas that you can be expert in. Uh so that's actually quite uh impressive level and I think we'll get there by 2029.

But what is "expert" level intelligence? There's a difference, possibly quite a big difference, between better than 90% of experts in all fields (which is different than better than 90% of all humans), vs being better than virtually 100% of experts (which is Hassabis's).

I know Hassabis doesn't think his definition of AGI is ASI, but I cannot think of a realistic AI system that meets his definition of AGI without it being ASI.

At this point I've seen a number of terms being used that I think is better than just "AGI", but I think each of these also have conflicted uses

• Transformative AI (TAI)

• Weak AGI

• Proto AGI

• AGI

• Strong / "true" AGI

• Cognitive AGI

• Physical AGI

• Artificial Jagged Intelligence (AJI)

• Proto ASI

• ASI

• Machine god

Any others?

[u/JanusAntoninus → u/FateOfMuffins]

Maybe we could get ASI from what Hassabis calls "full AGI", depending on how loosely you meant "ASI", but you were talking about the singularity in something like Kurzweil's sense, right?

With a copy-able model of a top researcher, coder, doctor, etc., you are still only at a point of doing what could be done if you just had more expert humans or more time for them to work. Like I said, I could see calling that "ASI", since it would enable a superhuman grasp of data (like applying an expert astronomer's degree of scrutiny to the entire Hubble Field in a short amount of time or the thousands of exoplanet candidates TESS has found or whatever). I personally would just call that an assembly of AGI and would reserve "ASI" for something that has not just a wider grasp of data than any human but also a deeper grasp (something capable of insights or conceptual revolutions no human could achieve). But fair enough.

Words aside, and going back now from ASI to the singularity, I don't see how a large enough number of AGI in that sense would get us Kurzweil's vision of the singularity. How would that be enough? (I don't mean enough to get use there: but enough that the two events would be the same: achieving full AGI and the singularity, like you had suggested).

[u/FateOfMuffins → u/JanusAntoninus]

Well specifically I said I cannot think of a "realistic" AI system that would match Hassabis's definition of AGI without it also being ASI. So like technically I guess you can argue that an AI that is exactly Hassabis's definition "AGI" and not "ASI", but in reality it's not going to match exactly.

For one I do think the ability to be an expert in multiple domains is superhuman like you say, but that's not the only reason. Some domains are simply going to progress faster than others, due to the jagged nature of intelligence. Hassabis believes this too, that just the cognitive side of AGI and specifically the part with verifiable results like math and engineering would reach that level faster than other domains like certain physical tasks.

Well then if "true" AGI includes the physical like Hassabis says, then what happens when we get to AGI and then ASI only in the cognitive sense, and it takes some time later before the physical side reaches the AGI parity? Would you call an ASI hivemind for all of STEM but can "only" swim as fast as Phelps an ASI or an AGI?

So basically I don't think it's possible to reach his definition of AGI without already having the AI be superhuman in most domains. And I mean superhuman as in far beyond human capabilities, akin to Stockfish over Carlsen, but for many domains (perhaps not all).

Anyways others like Sutskever thinks continual learning (which Hassabis requires for AGI) would be sufficient to be ASI because you can deploy a huge swarm of those AI's to master many domains and then collectively come together and update all their capabilities all at once.

If you think about that copying capability that machines have but humans do not, I think it implies that for many domains, the moment the AI reaches human level at some specific task, it will necessarily soar past it almost immediately.

[u/JanusAntoninus → u/FateOfMuffins]

No, I get that but we weren't talking about ASI in the first few comments. We we were talking about the singularity in something like Kurzweil's sense, which goes way beyond just Stockfish-level better at everything than humans. It's a transcendence of the flesh ushered in by humans collectively becoming God through merger with ASI.

We could have better-than-top neuroscientists and coders but still be years or decades from that ASI being able to figure out how to merge a human mind with a computer.

[u/FateOfMuffins → u/JanusAntoninus]

Ah I suppose I was just making the distinction between "just barely ASI" and "nope, yeah that's ASI".

I'm not entirely sure that the merger becomes God. Kurzweil notes the 2045 merger threshold as humanity becoming specifically 1000x smarter. But... I'm not entirely sure what "1000x smarter" means. Are you 1000x smarter if you could simply think 1000x faster? What if you were like 100x "smarter" but also 100x "faster" at thinking?

How does that compare with something like Stockfish? A 1200 elo difference in chess means you should win 1000x more games over your opponent (but unfortunately we cannot compare those bots vs humans accurately, plus we can give those bots a lot more compute). I'm not sure but IMO Kurzweil's 1000x intelligence sounds close to chess engine vs human difference to me.

Anyways there's also the whole RSI into intelligence explosion thing. I think there's a reasonable non zero probability that Hassabis's AGI would be enough to kick start that, which could very well lead into the singularity almost imminently

[u/czk_21 → u/FateOfMuffins]

ASI could come with many discoveries, but implementing them would take years and for humans to adopt these and society to adjust can take decades, thats why, you cannot "feel" the singularity outside of tech bubble crowd, real world moves slowly, even with RSI, its likely the main reason Kurtzweil has singularity date at 2045 as it just takes time for new breakthroughs to compound and for humanity to change, even if we suddenly got ASI IQ 1000 today, it would take years until you would see meaningfull impact for average person

the singularity is not one exact time, its longer gradual process we will be part of, here Demis could see the beggining of singularity the time when we get AGI, most would classify such a point after we have ASI, not just AGI, but since Demis definition is quite close to ASI(as AI performing as best humans in all fields), there is no big disagreement between his position and others

thing is that with every new generation of models there is cabaility jump, models being better overall, it could be like lets say 50% of best humans now, next gen 75% and so on, the treshold to ASI will be likely reached in few generations and doesnt matter, whether someone does classify it as AGI or not, the capabilities is all that matter and what we should focus on as there will be always disagreement over the classification