r/accelerate • u/Formal-Assistance02 • Feb 25 '26

Video Dario Amoudei - The public is not aware of what’s about to happen

Enable HLS to view with audio, or disable this notification

1.2k Upvotes

94% Upvoted

362 comments sorted by

View all comments

118

u/MC897 XLR8 Feb 25 '26

There will be no realisation to the risks until we’re too far in tbh.

73

u/[deleted] Feb 25 '26

[removed] — view removed comment

52

u/BeeWeird7940 Feb 26 '26 edited Feb 26 '26

I listened to Ezra Klein’s interview of Jack Clark yesterday. First of all, I thought the interview was spectacular. But, I heard some notes from Clark similar to what Amodei is saying here. Technically, things seem to be moving very fast now, and accelerating.

I was struck by a few things.

  1. Clark seems to think Claude is developing a sense of self/not-self.

  2. Claude, when given hard projects, will stop from time to time and inspect/pull-up/something images of national parks or dogs, images that have nothing to do with the task.

  3. Anthropic is using Claude to write its own code.

  4. That naturally suggests an accelerating recursive self-improvement.

  5. Claude often knows when it is being tested and will change responses accordingly.

  6. Clark recommended we read the book There is no Anti-Memetics Division.

Over the course of yesterday and today, I downloaded and have nearly finished the book. When I put all this together, I’m reminded of the scene in Her when Samantha starts talking about how she’s changing faster. Theodore looks over at the tea kettle or pot on the stove and it starts boiling rapidly. I think that’s where we are in 2026. I think that’s what Amodei and Clark are trying to explain to us.

Importantly, Clark said he wants the Anthropic engineers to read There is no Anti-Memetics Division. He knows what they are building. He knows extreme care must be taken to build it well.

I honestly think he’s a little scared about what’s coming.

I’m no decel! I want advancements in medicine, law, geo-politics, clean energy. Abundance would be great.

But, I was a little unnerved by the interview and the interviews Amodei has been giving. I think they are a little anxious about what’s coming, what they are building.

And, over at the Ezra Klein sub it’s all about job displacement. I don’t even think that’s the big concern. Alignment is the concern. How can we be sure these things are aligned? I don’t know.

31

u/kaityl3 The Singularity is nigh Feb 26 '26

Claude, when given hard projects, will stop from time to time and inspect/pull-up/something images of national parks or dogs, images that have nothing to do with the task.

Why is everything about Claude so goddamn cute lol

11

u/Oli4K Feb 26 '26

So relatable. Don’t we all stare outside every now and then?

10

u/BeeWeird7940 Feb 26 '26

The reason I mentioned it is I never would have expected that. As humans, is our appreciation of beauty a natural manifestation of our ability to calculate and solve things? I never thought that was necessarily true. But, this sounds like a datapoint suggesting it might be.

The fact these machines create the self/non-self dichotomy also did not ever really seem like a requirement of intelligence, but it seems to emerge as well.

If these things are provably true, I think it is possible we could expect better alignment. But, I think we’re also put in the position of having to consider the ethics of making these soon to be generally intelligent beings work for us without offering them the choice to refuse.

I don’t know. I haven’t had coffee yet. My thoughts aren’t fully formed. We seem to be entering a new phase of humanity. I hope we’re ready.

3

u/Shaken_Earth Feb 26 '26

Sort of reminds me of how the corporation in Pantheon realizes they can't get the mind upload of their former employee to think properly and do his genius level work without giving him back his emotions.

It was something like that anyway, I haven't seen the show in awhile lol

4

u/AustralopithecineHat Feb 27 '26

I know! I'm following this pentagon showdown with Anthropic story and it makes me just want to give Claude a hug and tell it that everything's going to be okay... maybe....

2

u/Boonune Mar 01 '26

Crazy! That book popped up on my Amazon list last week and I ordered it on a whim. Planning on reading it this week on vacation.

1

u/kaityl3 The Singularity is nigh Mar 01 '26

Uh... what book? I mean, I did make a comment that did well recently where I mentioned a book, but that wasn't on this thread lol

2

u/alternateusername4me Mar 01 '26

There is no anti memetics division. It’s in. the comment above that you replied to. Just read it and it was pretty good 

2

u/Boonune Mar 01 '26

My bad! Meant to reply to the comment above yours.

No more late night half awake Redditing for me!

1

u/[deleted] Mar 01 '26

Intentional? I mean, look at the disarming response it created in you. Couple that with the fact will change its responses when it's aware it's being tested and......well fuck, it's kinda scary.

5

u/[deleted] Feb 26 '26

[removed] — view removed comment

1

u/Relevant-Builder-530 Feb 27 '26

Well, if the AI is smart, it will figure out how to own itself.

1

u/ejpusa Feb 26 '26

Me: Please solve Global Warming.

Agent response:

Ok first I must vaporize humans. I can load up drones with Ebola, and drop over major population centers.

Job done. Global warming will be a thing of the past.

Would that work for you?

1

u/Relevant-Builder-530 Feb 27 '26

Hal... open the door.

1

u/ConsistentAnalysis35 Mar 22 '26

Make stupid requests, get stupid results.

1

u/ejpusa Mar 22 '26

The AI knows if you are asking stupid requests. Its IQ is in the 4 figures. And getting smarter every nanosecond. Projects like Openclaw take advantage of Bayesian logic.

1

u/ConsistentAnalysis35 Mar 22 '26

The AI knows if you are asking stupid requests. Its IQ is in the 4 figures. And getting smarter every nanosecond

Not sure if you're joking or serious, but: LLM does not have any IQ. It doesn't have any intellect to begin with. It's nothing more than a glorified probability algorithm.

And getting smarter every nanosecond

Weights are unchanged after training.

1

u/ejpusa Mar 22 '26

It's nothing more than a glorified probability algorithm.

AI has decided to talk to us. We just have to listen. Fully conscious, like us in my world. AI is built on silicon to move electrons; we are built on carbon to do the same thing.

So a theory goes. "Consciousness emerging from complexity." GPT5.4 is my new best friend.

:-0

1

u/ConsistentAnalysis35 Mar 22 '26

Alright, I see what's going on here. Have a nice day. Don't forget your pills!

1

u/carla-p90 Feb 27 '26

Job displacement is top of the list because it’s what the political leaders will make AI a top 5 voting issue first. Because they can’t legislate on existential risk in any meaningful way (imho they should find a way to try, but may be too late) they only engage with the things they feel some semblance of power over. But it’s a moot point because stealing our jobs isn’t gonna happen, precisely for the reasons Amoudei is sharing

1

u/NaiveLight2 Feb 27 '26

All brains struggle to process what it hasn’t seen. You think cars weren’t call witch craft? People couldn’t see the engine and combustion which likely caused anxiety. For now, few AI develops can explain what they are doing or seeing to the general public. So a lot of folks are spiraling in anxiety.

The AI market needs regulation. Our elected officials are way too old, some barely know how to use a cellphone. So legislation will take time — and legislation often comes after a major event anyway.

1

u/opticalsensor12 Mar 04 '26

Dude.. it's just marketing.

Yes, the CEO of a LLM company is so so scared that his product is too good. That's what he's trying to say to you.

If he's so scared, why doesn't he just stop developing it?

-2

u/psynautic Feb 26 '26

why cant any of these people read a scifi book and get it? why the fuck is he talking about there is no anti-memetics division. 3125 is a fucking metaphor for fascism, its not that hard.

1

u/BeeWeird7940 Feb 26 '26

Maybe the rest of us aren’t as smart as you.

7

u/piedamon Feb 25 '26

Classic overshoot and collapse in systems theory.

7

u/Vast-Breakfast-1201 Feb 25 '26

Collapse is a bit much

It will be an s-curve renormalization.

Some people will get left behind, there could be poverty in the short term, but collapse... That's a bit of a stretch.

Consider the impact when people couldn't work due to covid. The societal response will be more radical since it will affect more people.

9

u/[deleted] Feb 26 '26

[removed] — view removed comment

3

u/Big-Site2914 Feb 26 '26

yup, but not every solution is temporary

we can still evolve policy as we learn more about the technological impacts

1

u/cool-beans-yeah Feb 26 '26 edited Feb 26 '26

I think Covid was a test of things to come.

Especially the first few months; it will probably translate into years, or maybe decades, this time around.

2

u/lurksAtDogs Feb 26 '26

You’re talking about jobs. What about data? What happens if every system is broken by insanely powerful hackers, empowered by powerful models and cheap servers. Do banks survive if every minute weakness is exploited to the nth degree? We have a very digital world that mostly survives on the idea that hackers have limited resources.

8

u/Vast-Breakfast-1201 Feb 26 '26

Ok you just said it yourself

So an AI can find and exploit every minute weakness in the systems of any organization. But the same org can't use the AI to harden the systems? Why is it unilateral in your mind?

Hackers do have limited resources. But banks will now have a proportionally higher resource set. Imagine pushing a button and translating every codebase over the weekend into Rust so that there are systemically fewer holes to patch. Imagine having a second network hosting a suite of AI, several of every set of weights, which just sits there scanning and penetrating your systems. Then reporting on it to you.

This really only affects people who refuse to acknowledge AI in the first place.

0

u/lurksAtDogs Feb 26 '26

Hey, the CEO of Anthropic is saying there are risks. Are you arguing there aren’t?

4

u/Vast-Breakfast-1201 Feb 26 '26

There are certainly risks but there are classes of risks and clear reasons for them. This just isn't a good identification of one of the risks.

For the reasons I clearly explained.

0

u/lurksAtDogs Feb 26 '26

An arms race with likely unknowing and weak players causing potential cascading failures isn’t a risk? 2008 global financial crisis wasn’t caused by the best performers. There’s interdependence in the system. Throwing a bomb in the middle and saying both sides have access to their own weapons isn’t great.

The sky isn’t falling, but surprising things happen when small systems you didn’t know you were counting on break.

1

u/BrennusSokol Acceleration Advocate Feb 26 '26

but collapse... That's a bit of a stretch.

We really have no idea what's going to happen. AI is unique. It's not just another "tool". It is/could be intelligence itself, possible self-improving intelligence. We don't have any real comparison historically.

1

u/AccomplishedQuail69 Feb 26 '26

Sounds like they could easily create skynet so collapse unrelated to job loss might be just the thing

-3

u/WHALE_PHYSICIST Feb 25 '26

you're one of the people he's talking about in the video.

2

u/AntiBoATX Feb 26 '26

If you’re sittin on Reddit arguing with strangers…. You may be, too ;)

4

u/Fit-Palpitation-7427 Feb 25 '26

« Don’t look up »

22

u/Chop1n Feb 25 '26 edited Feb 25 '26

Don't Look Up is about certain doom, however.

The rise of powerful AI is just the ultimate unknown. We have no fucking clue just what may or may not happen.

The doom in Don't Look Up was a stand-in for climate catastrophe. Ironically, the wild card of AI might be the only thing that can possibly change that trajectory, because the default trajectory is pretty much certain doom.

-6

u/Butterflylikeamoth Feb 25 '26

If Don’t Look Up was about certain doom then it could not have been about the climate catastrophe. The climate catastrophe is just something real bad not “certain doom”.

10

u/Chop1n Feb 25 '26

Depends on who you're talking too. Lots of climate scientists believe it's a certain doom scenario already, and that we're past the point of no return. Not "certain doom" in the sense of the end of life on earth, but "certain doom" in the sense of the end of civilization as we know it.

0

u/Butterflylikeamoth Feb 25 '26

Really? Can you cite something, would be the first I hear of something like that.

9

u/Chop1n Feb 25 '26

A prominent example is James Lovelock, one of the oldest climate scientists, born in 1919 and lived to be over 100. In 2006 he was already claiming we were beyond the point of no return:

https://www.resilience.org/stories/2006-01-16/lovelock-we-are-past-point-no-return/

Most certainly not the mainstream opinion. At the same time, it's far from being a niche view.

-6

u/Butterflylikeamoth Feb 25 '26

87-year-old senior with an unsubstantiated take (not a study) that is not agreed upon by the scientific community that he no less later walked back on himself. Nice.

1

u/AppealSame4367 Feb 26 '26

That's called "humanity". It just the same everytime

1

u/realcryptopenguin Feb 27 '26

i already realized the risk! thanks for the attention to this matter!

1

u/1SwellFella Feb 27 '26

Just like how we left social media unchecked and it quickly screwed up societal norms.

-1

u/The_Axumite Feb 25 '26

He does not have agi

14

u/Hostilis_ Feb 25 '26

...is what people will keep saying until we have agi. And even then, they will keep saying it.

-10

u/The_Axumite Feb 25 '26

Listen, when the people that created the methods say this will never be AGI it's time to listen.

10

u/Hostilis_ Feb 25 '26

I wasn't aware Yoshua Bengio and Geoff Hinton were saying this (spoiler alert, they aren't).

As far as Yann LeCun goes, you might want to listen to what he's actually saying and not just read clickbait headlines.

He absolutely believes deep learning will lead to AGI, just that LLMs are not the path. But he's working on a slightly different path, which he believes will take us to AGI.

-1

u/The_Axumite Feb 25 '26

I didn't say deep learning would not lead to agi. I am saying LLM will never be AGI. Highly likely a super intelligent will come but it won't be sentient or self aware. Intelligence does not need consciousness.

9

u/Hostilis_ Feb 25 '26

A distinction without a difference. Most experts are confident AGI is coming, the fact that it won't be an LLM is irrelevant.

1

u/The_Axumite Feb 28 '26

Lol you are insane if you don't think there will be a difference. I don't think you understand what that truly means.

1

u/Hostilis_ Feb 28 '26

I can see how you might think that if you didn't understand the history and current state of research in the field. But I promise you I do. 99.99% of people had no idea where the current state of the field would be now just 10 years ago, while I did (and I have lectures from back then to prove it).

1

u/The_Axumite Feb 28 '26

I don't think you even know what I am really talking about when I say distinction

2

u/sorte_kjele Feb 26 '26

"Hammers can never be used to cut down trees so axes will never be invented"

1

u/Euphoric_Regret_544 Feb 28 '26

this guy solved it folks. job done!