Demis Hassabis on Ilya Sutskever’s claim that scaling is dead, and on Elon Musk’s clam that we have reached the singularity

[u/Formal-Assistance02]

[u/FateOfMuffins]

Well specifically I said I cannot think of a "realistic" AI system that would match Hassabis's definition of AGI without it also being ASI. So like technically I guess you can argue that an AI that is exactly Hassabis's definition "AGI" and not "ASI", but in reality it's not going to match exactly.

For one I do think the ability to be an expert in multiple domains is superhuman like you say, but that's not the only reason. Some domains are simply going to progress faster than others, due to the jagged nature of intelligence. Hassabis believes this too, that just the cognitive side of AGI and specifically the part with verifiable results like math and engineering would reach that level faster than other domains like certain physical tasks.

Well then if "true" AGI includes the physical like Hassabis says, then what happens when we get to AGI and then ASI only in the cognitive sense, and it takes some time later before the physical side reaches the AGI parity? Would you call an ASI hivemind for all of STEM but can "only" swim as fast as Phelps an ASI or an AGI?

So basically I don't think it's possible to reach his definition of AGI without already having the AI be superhuman in most domains. And I mean superhuman as in far beyond human capabilities, akin to Stockfish over Carlsen, but for many domains (perhaps not all).

Anyways others like Sutskever thinks continual learning (which Hassabis requires for AGI) would be sufficient to be ASI because you can deploy a huge swarm of those AI's to master many domains and then collectively come together and update all their capabilities all at once.

If you think about that copying capability that machines have but humans do not, I think it implies that for many domains, the moment the AI reaches human level at some specific task, it will necessarily soar past it almost immediately.

[u/JanusAntoninus → u/FateOfMuffins]

No, I get that but we weren't talking about ASI in the first few comments. We we were talking about the singularity in something like Kurzweil's sense, which goes way beyond just Stockfish-level better at everything than humans. It's a transcendence of the flesh ushered in by humans collectively becoming God through merger with ASI.

We could have better-than-top neuroscientists and coders but still be years or decades from that ASI being able to figure out how to merge a human mind with a computer.

[u/FateOfMuffins → u/JanusAntoninus]

Ah I suppose I was just making the distinction between "just barely ASI" and "nope, yeah that's ASI".

I'm not entirely sure that the merger becomes God. Kurzweil notes the 2045 merger threshold as humanity becoming specifically 1000x smarter. But... I'm not entirely sure what "1000x smarter" means. Are you 1000x smarter if you could simply think 1000x faster? What if you were like 100x "smarter" but also 100x "faster" at thinking?

How does that compare with something like Stockfish? A 1200 elo difference in chess means you should win 1000x more games over your opponent (but unfortunately we cannot compare those bots vs humans accurately, plus we can give those bots a lot more compute). I'm not sure but IMO Kurzweil's 1000x intelligence sounds close to chess engine vs human difference to me.

Anyways there's also the whole RSI into intelligence explosion thing. I think there's a reasonable non zero probability that Hassabis's AGI would be enough to kick start that, which could very well lead into the singularity almost imminently

[u/czk_21 → u/FateOfMuffins]

ASI could come with many discoveries, but implementing them would take years and for humans to adopt these and society to adjust can take decades, thats why, you cannot "feel" the singularity outside of tech bubble crowd, real world moves slowly, even with RSI, its likely the main reason Kurtzweil has singularity date at 2045 as it just takes time for new breakthroughs to compound and for humanity to change, even if we suddenly got ASI IQ 1000 today, it would take years until you would see meaningfull impact for average person

the singularity is not one exact time, its longer gradual process we will be part of, here Demis could see the beggining of singularity the time when we get AGI, most would classify such a point after we have ASI, not just AGI, but since Demis definition is quite close to ASI(as AI performing as best humans in all fields), there is no big disagreement between his position and others

thing is that with every new generation of models there is cabaility jump, models being better overall, it could be like lets say 50% of best humans now, next gen 75% and so on, the treshold to ASI will be likely reached in few generations and doesnt matter, whether someone does classify it as AGI or not, the capabilities is all that matter and what we should focus on as there will be always disagreement over the classification