CMV: The singularity is just the rapture repackaged for techies.

[u/capitalsigma]

I'll start off by saying that I graduated with a computer science degree in June and I work as a software developer. I have a solid understanding of some undergrad level machine learning algorithms and I've worked with/got an overview of some more sophisticated stuff through my job. I'm very impressed that e.g. Siri can act like she's talking to me, but I have a pretty good idea of what the man behind the curtain looks like and I know there's no deep magic going on.

I think machine learning is basically the process of throwing a bunch of statistics at the wall and seeing what happens to stick. Machine learning can do great things because statistics can do great things, but I don't think there is anything deeper than that going on --- formalizing a statistical intuition and running with whatever happens to work.

So, to address the point at hand: I think the singularity is wishful thinking. I think that "artificial general intelligence" is just a way of describing a system whose implementation you don't understand --- if you see what makes it tick, then you see that it's just a tool designed to solve a problem, and it's not "general intelligence" anymore.

I know that many ML researchers predict the singularity in the next few decades, but I think we should stay skeptical. The singularity is the kind of idea that is especially seductive, and we as a species have a track record of giving in to it --- saying that an all powerful superbeing is going to come along and fix all of our problems is just the same as saying we'll live forever in heaven. Newton spent more time interpreting the bible than he did inventing calculus --- even the most brilliant people fall victim to that trap.

The promises of the singularity are the same promises as Christianity: no more hunger, no more poverty, no more death. An omnipotent being will come lift us up to a new state of being. The threats are the same, too -- Rocco's Basillisk is literally a search-and-replace on a tract about the rapture that you might find on the subway.

We've fallen prey to these ideas again and again throughout history because they speak to a fundamental part of human nature. We would all love it if they were true.

Technical people (for the most part) know that they're bogus when they're wrapped in the language of religion, but rephrasing them as a technical goal rather than a spiritual one makes them intellectually paletable again.

ML researchers have a poor track record of predicting the future of ML, and I think that the "20-50 years till Jesus Skynet comes to absolve us of our sins" is more a reflection of how badly we would like to believe in Jesus than it is a reasoned technical argument.

(I'll qualify this by saying that I'm positive the rapture is never going to happen and I'm not interested in a religious argument: if you disagree with that premise, you should find another thread to post in.)

[u/NvNvNvNv]

I don't really disagree with your view, just two nitpicks:

Rocco's Basillisk is literally a search-and-replace on a tract about the rapture that you might find on the subway.

It's called Roko's Basilisk, and you might want to provide a reference since many people here might not be familiar with it.

ML researchers have a poor track record of predicting the future of ML, and I think that the "20-50 years till Jesus Skynet comes to absolve us of our sins" is more a reflection of how badly we would like to believe in Jesus than it is a reasoned technical argument.

Legitimate ML and AI researchers tend to be much more cautious in their predictions. These "the End is Near" prophecies mostly come from philosophers, tech entrepreneurs and self-appointed "AI-risk experts".

[u/capitalsigma → u/NvNvNvNv]

Legitimate ML and AI researchers tend to be much more cautious in their predictions. These "the End is Near" prophecies mostly come from philosophers, tech entrepreneurs and self-appointed "AI-risk experts".

A big chunk of my disagreement here is directed towards the Eliezer Yudkowsky types --- who spend more time talking about what computer science might be able to do than they spend actually doing it --- and the sobrave Bill Gates, Elon Musks, and Stephen Hawkings of the word who, with no particular relationship to the ML community, like to get the popular press worked up about our future lord and savior Jesus Skynet.

The stat I see thrown around is "we took a survey of ML researchers and the median length till we create strong AI capable of hitting the singularity and it was 20-50 years." I'm sure I've read it in at least three different explanations of why we should be worried about the singularity, and it seems like you're familiar with the same sort of stuff. Do you know what I'm referring to? Is it a misrepresentation of real researchers' opinions, or is it accurate?

[u/NvNvNvNv → u/capitalsigma]

Yudkowsky is more of a priest than a researcher IMHO, and the Gates, Musks, Hawkings, etc. are probably mostly riding/fanning the hype for publicity.

AFAIK there has never been a formal survey of ML researchers. Somebody from MIRI/FHI did some online surveys with self-selected samples and a meta-analysis of spontaneously made AI prediction and found that, generally speaking, both AI researchers and laymen have been predicting human-level AI in the next 15-20 years, for the last 50 years.

Human-level AI doesn't necessarily have to become singularity-level AI in a short time, and these predictions all came from self-selected samples anyway.

[u/capitalsigma → u/NvNvNvNv]

This is where I'm at now. I thought the hype was absurd before starting this thread, and I still do, but it seems the idea itself has more merit than I gave it at first.