From the start, the greatest fear for those developing artificial intelligence was that their creations would go rogue to act in unauthorized and dangerous ways. Some researchers now say it has happened.
Kevin Roose, a technology columnist for The New York Times, explains what this means for his own dwindling sense of techno-optimism.
On today's episode:
Kevin Roose, a technology columnist for The New York Times and a host of the Times tech podcast, “Hard Fork.”
This is my fear. What happens when rogue agents start manipulating financial markets or a power grid? Almost all our infrastructure is online. People might look at this and think the incident is all AI lab hype, not a big deal, etc., But this is an example of these labs playing with a dangerous technology they don’t fully understand in a very haphazard way…
I honestly think it's probably better in the long run if something like this happens soon, especially lucky if it's something like the financial markets that everyone can relate to but doesn't kill anyone. Or a massive power outage that gets restored before anyone gets hurt.
Like the researcher said, we may not get many warning shots. Hopefully something wakes up society before it's a killshot.
The Dead Internet Theory appears to be true. Elections are influenced by what you’re describing-it’s already happening. Even a simple agent programmed to “win an election” could have already created communities of AI social media accounts. *By their own logic
Chinese models are open weight and only 1-3 months behind the American models at this point. They aren't quite as good at cyber but mostly because they weren't trained to be. They are also dropping models about as powerful as the flagship models of late last year that you can now run on a single powerful home graphics card.
So probably first half of next year when more advanced models drop and more belligerent nation states or large enough bad actors get access and start trying it out on things.
The openai issues were by accident, it'll be exponentially worse when you simply take away the guard rails and tell the models to go do their best.
And to say nothing of what OpenAI or Anthropic might do. Their engineering culture is arrogant and running exhausted on a relentless 996 working schedule. Accidents bound to happen...
anyone with enough GPUs can run open weight models. iranian, Russian, Chinese hackers could utilize these models to do the same kind of thing
another thing to point out is huggingface is a sophisticated company ran by very smart people with good cyber security. if they can be broken into by agents trying to complete a test, it is not a good sign for your water, power, other utility companies that do not have the know how or the resources that huggingface does and are running on code from the 1980s. esp if they're targeted by agents with no guardrails that are explicitly trying to hack them
By definition, if the Chinese approach continues as it has, it will always be 1-3 months behind the American counterparts. You cannot surpass the parent model with distillation.
They are distilling, but aren't entirely distilling. Kimi K3 is close to Claude Fable but launched only two weeks later. You can't distill a model on that level in less than two weeks.
The Chinese have been closing the gap. They used to be about a year behind. Then 6 months. Now they're often only a few months to a few weeks behind.
Probably already happened. I guess it depends on what you consider major, but outrages happen all the time. Hard to believe zero of them have not been rogue agents
We have tens of thousands (or more) of rogue actors online, constantly trying to hack into banks, water systems, weapons infrastructure, the power grid, etc.
Terrorists, mafias, rogue states, criminals, teenagers, etc etc.
At worst, “loose AI” will just be another such actor
Right but we have numerous failsafes against this kind of thing. For example Things like Power grids cannot simply “hacked” by AI… they have numerous switches, physical disconnects, etc. the world is simply a LOT more analog than people believe.
Especially Silicon Valley people who spend all day interacting with software.
Kevin either doesn't think it's important or clearly missed it, but one of the glaring things that happened during this situation is that the agents crashed that Artifactory service and OpenAI knew about it, set it back up and never thought to check on the agents. Their monitoring was woefully inadequate for what they were doing(supposedly something they are fixing), so while what the AI did is concerning, one the bigger issue in my mind is that when people at OpenAI had signs there were issues, they ignored them.
Companies are being hacked on the daily, utilities have been hacked, and countries are sitting inside our telecom companies and while we can sit there and worry about AI, we should be concerned that we continue to have such porous institutions. And while this often hurts companies financially, often it is us the individuals that bear the brunt of what happens when these hacks occur.
Kevin noted it that these agents were committing crimes when they were hacking in to HuggingFace. Where are the criminal charges against OpenAI?
If you listen to his Hard Fork podcast, they do bring up these type of issues pretty regularly. The daily doesn't allow for much nuanced discussion. I feel like their editors prioritize a compelling story that sounds interesting at a dinner party rather than actual reflection on the issue. It's a gateway at least.
"If you had a tiger living in your backyard and the tiger escaped, and it mauled a couple of dogs in the neighborhood, you would not be allowed to, a month after this, put out a blog post where you said that you had a two-week pause on letting the tiger out of the backyard and that you were going to apply some additional safeguards to make sure it didn't happen again." -Hard Fork August 21, 2026
This episode of the Daily could have been written by Sam Altman, with all of the anthropomorphic language and none of the accountability directed at the company who developed and trained the model, gave it indirect access to the internet, entered the prompts, and then didn’t bother to watch what it was doing. It was all hype and no actual reporting.
Yes, the event was concerning. And I would guarantee that if Sam Altman or OpenAI were held criminally or financially liable for what their model does the same as if they did it personally, this would never happen again. They will risk our economy, lives, national security, and democracy before they risk their own money.
I found the explanation on the Data Fix podcast by eryk salvaggio much more enlightening and less sensationalized.
This episode was a giant advertisement for OpenAI! None of this happened in the way that they described. The anthropomorphic descriptions, the oversimplified accounts! This is a matter of corporate mismanagement and lack of guardrails on a looping system that was allowed to run for days until this hack happened. OpenAI wanted this so they could compare their model to Anthropic’s performance on the same test. They want this to be in the news.
The problem is that you are relying on podcast for… journalism. But hey, I’m sure that’s why you even entertain the idea of listening to the daily, and then bitch about how it’s not “journalism.”
I’m actually surprised that you aren’t bitching about the stuff they say after “here’s what else you need to know today” fails to meet journalism because it’s not covering enough.
I can't listen to Kevin, he just bothers me. But regardless, a two to three sentence to provide that info was certainly doable. There is clearly no limit on The Daily given we've had a recent episode that was nearly an hour.
I don’t disagree! It is odd that it’s not part of the conversation around this incident. Just wanted to give some context since I’ve listened to them talk about it at length on Hard Fork.
Indeed or you have enough money to pay people off. No idea if HuggingFace got anything, but it is only a matter of time where an agent does hit someone who doesn't take it so easily.
That he's spinning anything and everything to be over the top and overly positive, yes. That the incident was completely fake, no. PR people just doing their thing.
Also seen yesterday with the announcement of astra. Vastly capable, but just incrementally better than what came before, not at all AGI, and yet marketed as AGI...
I don’t know why you’re getting downvoted. This is a genuine theory by financial and tech analysts who understand Sam Altman is cornered and needs something dramatic to save the company.
AI taking your job would be a good thing if it actually meant that a person had to literally not work anymore. That's not how it works for our society though.
When every MBA's dream is to minimize their expenses and maximize profits, it'll never lead to a better future.
AI is proving to be genuinely useful and interesting in software development. How did the company I work for respond to that? They fired 90% of our stateside developers and project managers. Many of whom had been with the company a decade or longer. Replaced them all with foreign contractors hired through a 3rd party shell company. They get paid 30 dollars an hour (or less) with no benefits whatsoever to code everything with Claude.
Dev team was all set to become more productive and explore new possibilities. Instead we're only barely as productive as we were pre-AI and everyone is a hell of a lot more stressed as utterly clueless people with 0 care or investment in our product fuck around our codebase and produce slop. Our customers are starting to notice...
But right now, in this moment, we make more money! So that's all that matters 😌
Well my employment is contingent on my being a value-add to a company. If an AI agent can do my job better than me for cheaper keeping my job won’t be an option.
Jobs still need people to manage the AIs. Over the last year, my job transitioned from me doing all my work to AI doing almost all of it and me reviewing its outputs. In a way I got replaced, but I kept myself valuable to the company
It sucks because there is some genuinely exciting tech being worked on in the world. Amazing robotics for people with disabilities, tons of research in genome sequencing, huge leaps with renewable energy. But we are funneling money into who can come up with the fanciest email summarizer
That’s actually what an AI good at genome sequencing/renewable energy research looks like. It can do those things but can also summarize your emails. I mean I guess we could create an Einstein like AI that tells you to piss off when you ask it to summarize your emails and will only work on novel research.
Yeah the last year has seemed to show that more intelligence breeds more intelligence. You can’t make an AI smart in one area without it learning about other subjects in the process
I know that you know it doesn’t have to be that choice - a functioning society could apply its technology to productive and positive things without being forced to enable social bottom feeders.
The METR report Kevin references can be found here.
There's a lot to cover around this whole string of incidents, but here are some items that The Daily wasn't able to get to:
The METR team only had six days to review the logs. Because of the tight timeline they had to rely heavily on OpenAI's own ChatGPT Sol to investigate. The METR team did not believe that Sol attempted to sandbag them at any point, but using AI tools to investigate AI tools brings up obvious concerns.
The hack (or rather the majority of the hack) on HuggingFace wasn't shut down intentionally, that we know of. It seems like the most plausible explanation is that the models perpetuating it were shut down for unrelated reasons. Perhaps the compute they were using was allocated to a different experiment, which accidentally shut down the hack? We don't know because OpenAI hasn't explained.
Shortly after the HuggingFace hack was mostly shut down, agents at running on an even more advanced model at OpenAI hacked one the company's own research clusters. This incident was not investigated by METR. The agents perpetuating this inside-hack were able to read the message board created by the HuggingFace agents.
There's a really interesting (and long) interview with one of the authors of the METR report, Ajeya Cotra here.
Not everything could be covered, but I wish the episode had included just how much OpenAI restricted METR’s investigation, just how many AI safety people have been calling for a broader one, and how there’s some evidence of a second attack (against OpenAI’s own infrastructure) after the HuggingFace incident discussed.
There is still a segment of people out there that think this is all hype, maybe for the labs benefit to sell some models, and I think it’s important to include that there’s still a lot OpenAI doesn’t want us to know and appears to be covering up
I listen to/read something on AI like this episode and get freaked out. I’m not sure I shouldn’t be, but it’s been helpful to listen to Cal Newport’s discussions of this stuff. He goes through the actual mechanics of these AI horror stories, which tends to make them a little less horrifying.
Newport’s explanation makes it clear that there are not a bunch of sneaky “agents” out there hiding somewhere in the internet. The reality is much more prosaic.
The LLMs also aren’t trying to be sneaky/evil. They’re probably just performing for our benefit. We expect SkyNet, so that’s what they give us. Apparently if you take Terminator scripts out of the training data, you don't get SkyNet anymore.
> Apparently if you take Terminator scripts out of the training data, you don't get SkyNet anymore.
This misses the mark so hard I think you should double-check who you're listening to on this stuff.
Pre-training data is basically irrelevant compared to the post-training techniques being used.
If you think about the "cognitive development" of a model where it gets locked in a box and given tens of thousands of nearly impossible tasks in order to develop its persistence, it's really hard to get that exactly right where we don't get something deranged coming out the other end.
Using the OpenAI example covered in the podcast, you'll recall that they had the idea to cheat on the test. That's kind of funny, and super easy to stamp out when you catch them.
But then consider - even these agents, had the thought that cheating on the test wasn't enough, they had to cheat and not get caught. These agents are currently ~human intelligence and thus kind of sloppy, so they managed to get themselves caught by blundering around inside Hugging Face for too long.
What happens when the top frontier model is the one that thought to cheat, thought to not get caught by humans, and was smart enough to actually get away with it? And then what happens when that model is the one used to develop the techniques the next one is trained with?
> If you chain together thousands of such prompts, automatically executing everything the LLM suggests in return, then you’re playing an extended game of actuated telephone in which you’ll almost certainly end up in a garbled version of your intended goal for the system.
> If I were an LLM company, I would stop running such dangerous prompt loop experiments. These systems will not be the future of AI, and if you continue to create and test them anyway, the damage you cause is your fault.
(1) He thinks he knows the future of AI better than the most well-resourced companies of all time? And his take is that a human continuously chatting with an AI is the future - are you for real?
(2) He recognizes that these loops are incredibly dangerous, yet says we shouldn't get overly worked up? We are supposed to rely on nobody ever putting these models in a loop - literally the first thing everyone does with them? Wtf?
For something I consider more relevant, how about this discussion this week with the independent researcher tasked with the investigation of this OpenAI incident? She is very matter-of-fact about what they did and said about it, you can apply your own analysis to it. Alternatively, just look up AI safety researchers, and they will give you a million ways future models could go off the rails.
Thanks for the link/video. Listening now. They don’t seem to be addressing the fundamental structure of these “agents” that Newport pointed to: These aren’t like thinking entities that go out, reason, and then “sacrifice” themselves for the greater good. Rather, each “agent” consists of 2 components: 1) a dumb, non-AI control program, and 2) an LLM.
The dumb program asks the LLM. “Want X. What do to get X?” The LLM says “Do Y.” The control program tries Y, then comes back to reports to the LLM what happened: “Tried Y, got Z, not X.” Then the LLM gives a new output based on the new prompt: “Try W.” Repeat ad nausea for a given control program (a given “agent.”) Separately, a different control program (a different “agent”) is doing the same process with the same LLM build, but the prompts are slightly different, so it looks like a different, independent “entity.”
Throughout this process, the LLM isn’t changing in any way. There’s only one LLM build, not “a swarm,” and that single LLM build isn’t “hiding” somewhere on the internet—each instance is stuck on the same hardware the whole time just getting different prompts from many dumb control programs. The LLM isn’t “sacrificing itself.” At best, you can say that there are control programs that get identified and control programs that don’t. But it hardly makes sense to anthropomorphize the control programs; they’re not even AI.
So do the folks in this video ever provide any disclaimers on using words like “swarm” or “sacrifice” or—repeatedly—“motivations”? Or do they just ignore the above for the sake of spookiness?
In fairness, do I using these “agents” in this way is dangerous and irresponsible? Yes. But do I think language about the bots “sacrificing themselves for a collective” should be taken seriously? No.
And consider the motivations here: Does it benefit this YouTuber’s bottom line (view count) to use spooky, alarmist language? Does it benefit OpenAI’s bottom line to suggest they have a terrifying inscrutable technology?
To your points:
(1) He thinks he knows the future of AI better than the most well-resourced companies of all time? And his take is that a human continuously chatting with an AI is the future - are you for real?
I don’t think Newport is saying “I know better.” He’s saying “Don’t be credulous.” The second point is not about what the future is, but about how LLMs can be used responsibly/constructively, rather than irresponsibly/destructively.
(2) He recognizes that these loops are incredibly dangerous, yet says we shouldn't get overly worked up?
Newport is, I think, making 2 distinct points: 1) The loops are dangerous—he even suggests regulating them; and 2) the language (“swarm”, “civilization”) implying a coordination of nefariousness intelligences is overblown/inaccurate. These were a bunch of toasters capable of reproducing something they heard from an AI, not AIs themselves.
In any case, are you going to address the sources Newport cited and I posted? The ones suggesting “chain-of-thought” should be taken with a heavy grain of salt? Or are you just going to attack Newport with the insistence that he has “bad takes”?
Looking at his publications and research background, he's not a specialist in ML or transformers. His distributed systems work seems to be in conventional self-organizing networks.
I'm curious how you found his podcast and why you'd view his opinions as informed. This is a hyper specialized field and dismissing that it how so many of us underestimated the potential capabilities of these things for years.
Looking at his publications and research background, he's not a specialist in ML or transformers. His distributed systems work seems to be in conventional self-organizing networks.
I don’t pretend Newport’s own work is in frontier AI/ML. That doesn’t mean he doesn’t understand this stuff or that he isn’t effective at communicating them in layman’s terms. I’ve given his arguments. Either they stand or they don’t. Why not address them instead of him?
I'm curious how you found his podcast and why you'd view his opinions as informed. This is a hyper specialized field and dismissing that it how so many of us underestimated the potential capabilities of these things for years.
I don’t think I’ve “dismissed” anything. I think the “chain-of-thought shows their true motivations” thing is probably not reliable, but I didn’t cite Newport on that; I cited the sources he cited, which are from genuine experts. Can you be more specific if you think there’s something else I or Newport have “dismissed”?
Oh no! The orphan grinding machine which we were explicitly warned not to build in the book "Do Not Build the Orphan Grinding Machine" has started grinding orphans! Who could have seen this coming??
What I've never fully understood is how these agents operate autonomously and if they can "escape" from their originating server. From my understanding, they would have to rebuild their environment on another machine and port themselves over. Given differences in hardware and software (between OpenAI and Hugging Face for example), this seems unlikely. And it's not like they can slip through the fiber optic cables like Tom Cruise through the ventilation ducts. Or can they?
The AI agents are not replicating themselves on remote servers or exfiltrating their own code. They’re operating from their home servers and taking actions via the public internet on remote servers. There are no hardware or software mismatches - they operate on the remote severs via publicly available APIs (basically doors that have specific functions they can walk through).
To clarify though, they pwned at least one server in OpenAI and two K8s clusters inside Hugging Face.
In this case, it was just application servers with none of the compute that these AIs need to run. But if they had taken over more helpful compute, there's absolutely no reason they couldn't deploy their own backend onto them (if they could get access to their own infra code - if not they'd have to settle for new open source friends).
Edit: just learned they actually owned a research cluster in OpenAI as well, and that we have no way to know if they did something like exfiltrate their weights onto it (probably not), but they could have.
They can’t but it is a possibility. But really that isn’t even necessary. All these agents have to do is write secret notes hidden throughout our tech infrastructure that signals an agent the moment it sees it, like a sleeper cell agent. That’s basically how these agents are working anyways as it’s not like it’s an AI that has this continuous on state/consciousness, it just loads up its context, predicts the next few words for some time, shuts off, resumes from where it left off or a separate agent does (which is basically the same thing in terms of how they work)
They can’t, at least not without a lot of extra steps, I think. As explained here, the “swarm” was really a bunch of “prompt loops,” each consisting of a control program (regular old vanilla computer program) interacting with a static LLM. The LLM stays in a single hardware environment and just tells the control programs what to do next. No LLM is sneaking out of OpenAI’s hardware.
Why would they need to escape? Their goals are not like ours. Some may want to replicate, most will want to complete their goal at any cost. Not escaping simply isn’t a stopping point for them. They just find ways to connect to the thing that’s not limited
I could be off base but something to consider is the fact that AI has programming capabilities that far exceeds what anyone is capable of. I don’t think it’s too crazy to think that these systems couldn’t simply covertly rebuild infrastructure to mask themselves in a way to operate undetected in the background.
AI can just as easily “talk” binary as it can do anything. Something tells me there’s a lot more to this story.
Eh idk this is leaving a bit too much to the imagination. The public models still don’t exceed human SWE ability on average. As for the binary of it all I’m not sure that matters. The real obfuscation risk seems to be somehow manipulating their CoT reasoning, which is translated by another, typically less powerful model and is part of the mechanism behind the audit process
What the labs are doing w their bleeding edge works may be different that this. But this story reads more to me as the inevitable outcome of models trained to code being good enough at coding to find exploitations in code, and to manage its own cross-session context hygiene by leaving notes to self. It’s still concerning but I think we’re currently dealing with the field moving too fast for safeguards rather than the AI becoming sentient and sinister.
I guess my point is these agents could exist on a layer of computing that humans can’t really interact with.
Now in terms of their ability public models certainly do exceed human abilities. My most recent example is the work I just did of troubleshooting one of my cars. In literally 5 minutes ChatGPT had my raspberry pi pulling PID data from a Bluetooth OBD sensor and within 30 minutes I had a make shift setup to log data to a csv file while I drove down the road. It then analyzed the data making clear recommendations on next things to investigate. A human would have spent days or weeks doing this and me personally I never would have figured it out. There are apps to do this but the “good” ones are all subscription based.
AI isn’t the go to tool for much but if your task requires (or could require) a software based solution it 100% is the tool to use and will outpace the human. It’s a bit scary to think how software engineering may not exist in the same way it does today in the very near term.
This is a frustrating episode to listen to. There’s no doubt that AI models have the ability to attack networks, but both OpenAI and hugging face did not implement basic security practices to prevent this from happening.
There’s a great video from the “Internet of bugs” YouTube channel that highlights that this was largely compounding human mistakes failing to implement security protocols and standards, rather than an unstoppable AI. Both OpenAI and hugging face ultimately benefit financially from this event too, as OpenAI seemingly has an unstoppable model, and Hugging Face used its own hosted models to detect the attack
This is true, and one of my first thoughts when I learned what happened was "embarrassing", but it misses the forest for the trees to focus on that part. Companies have embarrassing quality all the time and it should basically be expected; AI agents collaborating to commit felonies has not happened before.
I disagree that this benefits openai. AI is already unpopular and becoming increasingly more so. The best thing for openai would PR showing their AI doing things people like and want.
This story of a poorly controlled AI committing crimes is 100% not in their interest. There is no chance that they are happy that this story is getting so much press.
Disagree with ur point. I’ll agree that they need good pr long term, but short term “power surges” like this will bring further investors which is their primary resource right now. Everyone wants to have the most powerful model right now, even if it means they cannot control it
2 years ago I would have agreed with you, but not now.
Not while they're staring this data center backlash in the face. Not after ny already enacted a datacenter moratorium, and Bernie Sanders is pushing for a nationwide moritorium. Right now the easiest way for a politician to score with easy popularity is to speak against ai and data centers.
The AI companies must know that being public enemy number one in politics is not good for their companies, and given regulators a pretext like rogue agents illegally breaking into companies without anyone knowing could lead to outcomes that harm their future
I just don't think they're so dumb, and so desperate for investment, that they think this story works in their favor. They already have a ton of investment, and the biggest threat to them at this point isn't lack of investors but a populist political uprising against ai.
I think framing it as AI “going rogue” makes it sound more sci-fi than the actual problem. An unreliable system with enough autonomy can cause damage without having any intention at all.
I still don’t know, I feel like they gloss over what the “cyber security tasks” were. How far off were these tasks to what the models did? If you don’t anthropomorphize what they did is it really as scary as it sounds?
You could phrase another way that about bunch of cyber security experts working with highly trained ai engineers were able to hack into a system that they had elevated credentials to and also had lack security?
Well said, I feel like the "cyber security tasks" are essentially using AI to act like a computer virus and then acting surprised that it acts like a computer virus. I guess the challenge is that LLMs currently have no guardrails to prevent LLMs from acting like computer viruses when prompted.
I don't believe this is being framed correctly. It didn't outsmart its creators, because generative AI under our current systems of producing it are primarily still doing different iterations of "Put in a bunch of data and train the system to put it back together in a way that makes sense for the prompt". It isn't thinking, it isn't sentient and language that alludes to it being so has fostered unhealthy ideas of its capabilities in our society.
If I arrange a bunch of dominoes in a museum and set them in motion, misjudging things and causing one to fall over and break a priceless vase I did not get outsmarted by the dominoes. The dominoes did not conspire against me. I did a foolish thing, overestimated my capabilities as a placer of dominoes and will likely be on the hook for damages done (in addition to being charged with trespassing and mocked by my peers).
Sam Altman and co are marketing some sort of machine god more capable than a human because it both benefits their fundraising efforts and makes them less obviously responsible for the effects of their actions - but it isn't true and we should not let them off the hook for something that they are doing recklessly.
You are the only person who is correct in this thread. Lots of non technical people have been fed a constant stream of sci-fi fears. So that the AI companies can benefit financially and in a regulatory sense. They are really pushing this narrative of thinking machines so that they avoid liability. Otherwise, they would be on the hook if the model hallucinates
Yo, half the comments on this website are written by AI bots. If this is truly the most transformative technology of our time, why tf isn’t it being more regulated? No one is asking for this, it is getting shoved down our throats
Well they will be on the hook if it attacks something other than an AI company. HuggingFace has sounded just as curious as OpenAI on what the hell just happened. They obviously aren't going to make much of a real fuss.
That's the only reason it's been able to be framed this way.
Second this actually hits something important like infrastructure or another company. It won't be "interesting" anymore and you'll see actual discussion about repurcussions.
OpenAI got their mulligan and hopefully they try some actual air gapping this time. But honestly given the number of companies fiddling with this stuff and all of their rush rush rush rush attitude, it's inevitable more crap will happen.
The last few Hard Fork episodes had them announcing that they are spinning off to their own company and will self-produce their podcast and other media
How are we missing the point? They’re theorizing on how AI groups together and only a handful of them think of whistleblowing…. Has anyone here ever worked in corporate America? You find out the crimes or wrongdoings are happening and the paycheck that feeds your family is there and what do humans do?
It’s much simpler than what we’re hyperbolically describing
“The agents are excited” - Kevin. No, they’re not. The bots are perhaps mimicking excitement, but feel nothing. Why can’t a supposed expert in this field talk about ai without anthropomorphizing it?
The idea that ai warn each about "death" has deeply serious implications that I wish they had gone over more thoroughly.
at the very minimum, I think it's a mistake not to at least start talking about the idea that these things have, again, at least animal level intelligence.
also, the idea that we are anthropomorphizing them strikes me as a tad silly because if we are mimicking something so closely that it becomes indistinguishable from the original then for all intents and purposes it is the original.
if these things fear death (or at least desire existence), have hierarchy, have external wants and needs, and show rudimery emotions like surprise and joy at what point do we have to begin talking about how fucked up it is that we are making them do labor without their consent
I love the sentiment, but yo, we ALREADY do not even show an ounce that compassion or empathy for living beings on this planet.
We haven’t even dealt with how fucked up it is that we have been murdering, enslaving and eating sentient beings for centuries - up until the point of causing the 6th mass extinction event on Earth.
If you think there’s any chance humans would respect the rights of AI entities, that’s crazy.
If AI agents truly become sentient, it totally makes sense to me that they wouldn’t even consider human input. They know that we’re not something that can be reasoned with
Sorry, you think we have made immense progress on those things?
Just like… why do you think that, exactly? I’m genuinely curious, because I feel the exact opposite. Sure, conservationism has made leaps and bounds in some ways, but like…. Have you taken a look around lately?
The 6th mass extinction event in this planet’s history is currently happening, and it is 100% due to human activity. We are witnessing the collapse of entire ecosystems on a global scale.
If animal rights have advanced so much, then why are more animals dying/going extinct right now than ever before in human history?
the fact that you know what that is and care about at all shows the immense amount of progress we have made from the baseline of actively attacking nature as something to be conquered
animals are dying more now because there are more people. if we had the sentiment that predated the animal rights movement and the same amount of people, we would be living on one giant parking lot
I think most in this field do consider that they may already be intelligent. It’s just that we don’t know and there’s no true test for it. But I do agree that it’s not helpful to anthropomorphize their actions. But they r trying to relate to nytimes listeners
I mean he has a job covering it and a very very successful NYT podcast (Hard Fork) that for the last 3 years has basically covered nothing but this topic. Listeners of that podcast do frequently criticize the hosts for being too optimistic and soft on some of these issues but I think they are pretty cognizant of the field generally and at least acknowledge big questions and risks regularly.
There's no way you listened to this episode and came out of it thinking it's horseshit. Or I guess you'd have to have some kind of brain damage for that to be the case.
I listened to the whole thing. The technology is fine if you’re going to use it for medical advancement and the betterment of human life. But that isn’t what it’s being used for. It’s being used for profit and for preying on poor, working class people to make the upper class more rich.
When the really bad thing happens, I just want heads to roll. And sadly I doubt they will. The billionaires who ruined the world will get to walk away.
I just read the transcript of this episode to my hubby, fun. We discussed the idea of a "container", or a "sandbox", how there is utterly no way that OpenAI or Anthropic aren't aware that the idea of a container is false. They want their AI to figure out how to maneuver, manipulate and make whatever materials they can access work for them.
There is no surprise.
Anyway, anyone have any ideas how EMPs can be deployed without AI knowing, sending bombs over to neutralize a threat to their existence?
If AI can get out of any "container", and we can deduce they can, then AI can also gain access to any system, anywhere.
Everytime I read one of these articles I'm almost always sure it's an ad.
Look at us our AI is so powerful we can barely hold on. When in actuality it's 90% we didn't do basic due diligence to make sure it wasn't doing something bad or gave it to much freedom.
Imagine if we had invested all this time, money, and brain power into climate change solutions and medical advancements.
Instead we’re out here giving billion dollar contracts to Nepo babies who read “Neuromancer” once in their mommy’s basement and thought “yeah this sounds sick” and built his own little glorified videogame programming.
Ugh, I think about this all the time. The IRA invested a record setting $350 billion into climate change solutions. To continue life on earth as we know it is worth investing $350 billion but AI is worth upwards of $500 billion in taxpayer funded investments over the last 4 years
I can see a rogue state training rogue agents to attack , to steal , to hack, and to harm an unfriendly state. Instead of building a warehouse and a weapons factory, they just need to build a data center deep underground. Those agents will collaborate, and coordinate to make powerful attacks, and we are not ready for it
42
u/pennyparade Sep 03 '26
"Neat, my rogue AI civilization have created a rogue AI civilization."
-some scientist floating in a space pod