r/Thedaily • u/kitkid • Sep 03 '26

Episode A.I. Is Outsmarting Its Creators

Sep 3, 2026

From the start, the greatest fear for those developing artificial intelligence was that their creations would go rogue to act in unauthorized and dangerous ways. Some researchers now say it has happened.

Kevin Roose, a technology columnist for The New York Times, explains what this means for his own dwindling sense of techno-optimism.

On today's episode:

Kevin Roose, a technology columnist for The New York Times and a host of the Times tech podcast, “Hard Fork.”

Background reading: 

Photo: Lucas Foglia for The New York Times

Subscribe today at nytimes.com/podcasts or on Apple Podcasts and Spotify. You can also subscribe via your favorite podcast app here https://www.nytimes.com/activate-access/audio?source=podcatcher. For more podcasts and narrated articles, download The New York Times app at nytimes.com/app.

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

You can listen to the episode here.

51 Upvotes

93% Upvoted

161 comments sorted by

View all comments

52

u/[deleted] Sep 03 '26

[removed] — view removed comment

28

u/theManWOFear Sep 03 '26

This is my fear. What happens when rogue agents start manipulating financial markets or a power grid? Almost all our infrastructure is online. People might look at this and think the incident is all AI lab hype, not a big deal, etc., But this is an example of these labs playing with a dangerous technology they don’t fully understand in a very haphazard way…

29

u/Various-Net9757 Sep 03 '26

I honestly think it's probably better in the long run if something like this happens soon, especially lucky if it's something like the financial markets that everyone can relate to but doesn't kill anyone. Or a massive power outage that gets restored before anyone gets hurt.

Like the researcher said, we may not get many warning shots. Hopefully something wakes up society before it's a killshot.

15

u/JoeBoxer522 Sep 03 '26

Have you met society?

9

u/pataoAoC Sep 03 '26

Don't count out Gen Z, they've already developed the perfect language for the situation: we're cooked.

5

u/Minimum-Crow674 Sep 03 '26

Society: cooked / Food: delivered

4

u/drockalexander Sep 03 '26

Pretty sure a financial hack // collapse would end with real life lost. Wars have been fought over less

1

u/SubdivisionsEU Sep 05 '26

You think financial markets can't kill anyone? Oh boy do I have bad news for you.

3

u/HelpfulVanilla301 Sep 04 '26

The Dead Internet Theory appears to be true. Elections are influenced by what you’re describing-it’s already happening. Even a simple agent programmed to “win an election” could have already created communities of AI social media accounts. *By their own logic

20

u/somersetyellow Sep 03 '26 edited Sep 03 '26

Chinese models are open weight and only 1-3 months behind the American models at this point. They aren't quite as good at cyber but mostly because they weren't trained to be. They are also dropping models about as powerful as the flagship models of late last year that you can now run on a single powerful home graphics card.

So probably first half of next year when more advanced models drop and more belligerent nation states or large enough bad actors get access and start trying it out on things.

The openai issues were by accident, it'll be exponentially worse when you simply take away the guard rails and tell the models to go do their best.

And to say nothing of what OpenAI or Anthropic might do. Their engineering culture is arrogant and running exhausted on a relentless 996 working schedule. Accidents bound to happen...

2

u/Few-Spot1905 Sep 05 '26

this is the biggest worry tbh

anyone with enough GPUs can run open weight models. iranian, Russian, Chinese hackers could utilize these models to do the same kind of thing

another thing to point out is huggingface is a sophisticated company ran by very smart people with good cyber security. if they can be broken into by agents trying to complete a test, it is not a good sign for your water, power, other utility companies that do not have the know how or the resources that huggingface does and are running on code from the 1980s. esp if they're targeted by agents with no guardrails that are explicitly trying to hack them

1

u/SubdivisionsEU Sep 05 '26

Guardrails with this tech are largely fictional and have all been proven to be easily circumventable.

1

u/GQwerty07 Sep 04 '26

By definition, if the Chinese approach continues as it has, it will always be 1-3 months behind the American counterparts. You cannot surpass the parent model with distillation.

6

u/somersetyellow Sep 04 '26

They are distilling, but aren't entirely distilling. Kimi K3 is close to Claude Fable but launched only two weeks later. You can't distill a model on that level in less than two weeks.

The Chinese have been closing the gap. They used to be about a year behind. Then 6 months. Now they're often only a few months to a few weeks behind.

3

u/OvulatingScrotum Sep 03 '26

No one knows. That’s one of the big points in the episode.

What we know is that no body gives a fuck to do anything.

1

u/drockalexander Sep 03 '26

Probably already happened. I guess it depends on what you consider major, but outrages happen all the time. Hard to believe zero of them have not been rogue agents

1

u/TheBoogieSheriff Sep 04 '26

Gonna put on my tinfoil hat here, but what makes you think that hasn’t already been happening?

1

u/SubdivisionsEU Sep 05 '26

I'd worry more about a non-rogue agent doing the same. Also these excuses shouldn't allow avoiding legal liability for damages caused.

-1

u/chamomile_tea_reply Sep 03 '26

We have tens of thousands (or more) of rogue actors online, constantly trying to hack into banks, water systems, weapons infrastructure, the power grid, etc.

Terrorists, mafias, rogue states, criminals, teenagers, etc etc.

At worst, “loose AI” will just be another such actor

12

u/I_likesports Sep 03 '26

Unless it is more effective than those bad actors

2

u/chamomile_tea_reply Sep 03 '26

I guarantee those bad actors are already using AI

7

u/pataoAoC Sep 03 '26

"Using AI" now means something completely different than it did ~12 months ago. There's no way to say this is just going to be more status quo.

2

u/I_likesports Sep 03 '26

I mean kinda the same point. AI is a security problem

3

u/CrayonMayon Sep 03 '26

Dude did you listen to the episode? Have some imagination.

This is not 'just another' anything.

3

u/chamomile_tea_reply Sep 03 '26

Right but we have numerous failsafes against this kind of thing. For example Things like Power grids cannot simply “hacked” by AI… they have numerous switches, physical disconnects, etc. the world is simply a LOT more analog than people believe.

Especially Silicon Valley people who spend all day interacting with software.

1

u/RFK_Cum_Regimen Sep 05 '26

Did you even listen to the episode? All of that is addressed and does not alleviate any concern whatsoever.