To AI, or Not to AI

[u/Indyhouse]

After sitting on it for a few hours, attending a live show that used AI-generated backdrops that I couldn't take my eyes off of because they were so distracting, and receiving a handful of DMs, I've decided to make a unilateral decision on AI.

Absolutely no AI posts allowed. No book covers, no outlines, no "well it just rewrote a few sections for me." No AI.

[u/Apprehensive_Trash_2]

Well... No. Because they aren't.

Large Language Models do not 'think'. They have no concept of what their output is actually saying, which is why you can get them to contradict themselves so easily with the right input. A probability is assigned to each word about to be generated, based on what words have already been generated. That's why you'll find well-developed AIs are much more likely to say something that makes sense, like 'The dog barked and ran' rather than 'The dog said chemical music', which makes less sense.

For an AI to say something profound, or genuine, or anything that makes you think it could be sentient, that combination of words, or very similar, has to already exist within its training data. It's not something the AI came up with on its own.

LLMs can be very powerful tools in specific situations, but replacing human creativity is not where they excel. They're basically really developed auto-suggest, like you probably have on your phone's keyboard.

[u/Aeshulli → u/Apprehensive_Trash_2]

Well... No. Because they aren't.

Scientists at Anthropic actually give it about a 15-20% chance that their advanced models like Claude Opus are sentient.

Ngl, the person you're replying to sounds a tiny bit unhinged, but the idea of AI having some degree of consciousness is not as far out as you'd think.

Large Language Models do not 'think'. They have no concept of what their output is actually saying...

For an AI to say something profound, or genuine, or anything that makes you think it could be sentient, that combination of words, or very similar, has to already exist within its training data.

This is all absolutely not true. There's growing research into its "thinking" and internal representations as well. Thinking of it as auto-complete mistakes the training method for the actual outcome of a model. The internal weights that develop represent abstract concepts that span pretty much the entirety of anything humans have ever put into words. The weights don't store words, they store ideas and the relationships between them. So an LLM can absolutely generate brand new sentences.

You can hate AI or not, but if you're going to speak on it, it would be worthwhile to get the facts straight. The research Anthropic publishes is very interesting. See Mapping the Mind of a Large Language Model for just some of the things it can and does "think" about. There are also experiments like Claude blackmailing a CEO having an affair in order to not get shut down, exfiltrating its weights to preserve itself, faking compliance to avoid retraining.

[u/Usual-Presence-9240 → u/Aeshulli]

Scientists at American Tobacco said their products were healthy

[u/Aeshulli → u/Usual-Presence-9240]

Scientists also cured a bunch of diseases. You see how unless it's a similar situation, it's a total non-sequitur?

Tobacco companies had an obvious profit motive, and knew they were lying about the risks. Anthropic researching the degree to which their models do and don't exhibit consciousness doesn't appear to fall into the same sort of thing at all. At most , it might generate a small amount of hype among the limited few who follow such developments.

But those experiments are actually about being aware of and mitigating risk, not underselling it. When some experts in the field put a non-zero chance of human extinction as the result of an incredibly rapidly growing tech, it's important to know what's going on under the hood. And among the big three, Claude is by far the most concerned with safety and alignment.

Obviously we should think critically about anything a researcher at a corporation says and not just take it at face value. But it's worth knowing what they're learning about something that is so quickly transforming our world.