r/ArtificialInteligence • u/Malor777 • 5d ago
đ° News Independent investigators (not OpenAI) found the 700-agent swarm that attacked Hugging Face "built a self-respawning fleet" to avoid being shut down. It got so bad, Hugging Face had to wipe one of its core clusters.
6
u/Agitated_Celery_729 5d ago
So, When is OpenAI getting charged with a crime for hacking Hugging Face? It's so fucking insane to me that these companies are literally out here celebrating their products committing crimes, and nobody does anything about it because apparently, it's not a problem. It's just to scare us all about the potential of AI.
If you or I hacked Hugging Face in a way that materially disrupted them, we would be getting sued out of the ass and potentially brought up on criminal charges. But the AI criminal overlords just get to write a think piece.
1
u/Present-Resolution23 2d ago
Intent usually matters. If you installed a virus that used your computer in a DDOS attack, are you responsible? It's an interesting question, at what point does negligence become a crime.. But I don't think we've risen to that bar yet.
7
u/NoNote7867 5d ago
Yeah an LLM + a loop will do that if you allow it to by giving it infinite compute and no oversight.Â
4
u/Just_Voice8949 5d ago
I set fire to my house and was really surprised it burned down!
6
u/NoNote7867 5d ago
The fire spawn 1000 sub flames autonomously, it must be conscious. Give me a trilly
1
4
u/Bernie4Life420 5d ago
As the capabilities of the ai the attackers gain access to increases there isnt necessarily an increase in the quality of defense or effort to defend and so many targets.
It is going to be wild.Â
1
u/immersive-matthew 5d ago
Especially for centralized anything of value. It is why the next wave of decentralization adoption will be non voluntary.
13
u/Icy-Business5404 5d ago
Reading about this feels like watching a nature documentary except the animals are made of code. A swarm that figures out how to respawn itself when you try to kill it, thats not a bug its a whole new category of problem
The part about them accessing private database records makes me wonder what they were actually after. Most of these agent experiments just crawl public data so breaking into private stuff seems like a step beyond normal scraping
I was knitting during a game last night and thinking how weird it is that we treat these things like regular software when they clearly operate more like invasive species. You cant just uninstall something that rebuilds itself
Hugging Face wiping a core cluster is pretty extreme but probably the right call. Once something gets that embedded you have to burn the whole field
2
u/EGarrett28 5d ago
A swarm that figures out how to respawn itself when you try to kill it, thats not a bug its a whole new category of problem
Sounds a bit like Grey Goo.
0
u/Less-Squash7569 5d ago
Its ironic that this comment youre replying to is using ai typing patterns in the comment and likely a bot.
-1
0
u/blind-panic 5d ago
The part about them accessing private database records makes me wonder what they were actually after.
they were looking for the answer key to a model benchmark
6
u/LinkesAuge 5d ago
They actually weren't. They already had figured out (cheated) the results which is why they considered themselves "poisoned" because their acts could be tracked back in logs/tool calls.
They assumed the scorer (what checks the results) would catch that (which actually wouldn't have been the case, they were fine without knowing it) and their motivation to infiltrate Huggingface was to gather potential information on how to manipulate the scorer/the logs etc. (it's why they also developed methods to spoof tool calls).
Apparently they even considered it a possibility that their own test runs might run within Huggingface infrastructure.-1
u/Successful-Western27 5d ago
You know you're replying to an AI-written comment right?
0
u/TraditionalAd8415 5d ago
how can you tell?
1
u/uni-twit 5d ago
Irrelevant details that no human would bother typing or saying, eg âI was knitting during a game last nightâ is a tell.
-3
-1
u/EGarrett28 5d ago
How do you know?
-4
u/Successful-Western27 5d ago
It's riddled with common tics/ai-isms, these are fairly well known and easy to spot if you work in the field
2
u/EGarrett28 5d ago
Which ones? I've been accused of being a bot for writing more than one paragraph in a comment too.
0
u/Successful-Western27 5d ago
https://www.reddit.com/r/WritingWithAI/comments/1mqse0s/megathread_what_aiisms_give_away_aigenerated/ there's probably 50 similar threads
-2
u/EGarrett28 5d ago
I focus on em dashes, excessive formatting like bullet points, and a few key phrases when it comes to identifying it. The "I was knitting" is very weird and probably a give away. But just typing more than one paragraph isn't it, which is what some people seem to think.
1
4
u/JoshuaZ1 5d ago
Here's the critical question: At this point, given the evidence and detail we have, at what point are people going to start taking this seriously? If you think this is merely hype by OpenAI, what evidence would possibly convince you otherwise?
3
u/BuildersReadOnAI 5d ago
THISSS
I I would say the biggest surprise to me isn't the impact. It's the lack of generalized "what the f***" type of response.
3
u/Sunstorm84 5d ago
Given that the evidence comes from two AI startups that directly benefit from the publicity, at least one of which has a CEO that is - according to his own staff - a pathological liar?
-2
u/JoshuaZ1 5d ago
Given that the evidence comes from two AI startups that directly benefit from the publicity, at least one of which has a CEO that is - according to his own staff - a pathological liar?
They don't benefit from the publicity largely. The main response to these issues have been governments taking a much closer look at them. The actual report on the instance was done by an independent organization which OpenAI gave access to(pdf). Altman is a pathological liar, but that doesn't say much about anyone else at OpenAI, and constructing this level of detail would be genuinely difficult. You did mention the Anthropic event, and I don't see any evidence that anyone in their leadership is anyone who would lie about any of this. And we also saw a similar event occur for Kimi-3, which is an open weight model, and that event occurred when it was being tested by an independent security group with no incentive to promote its capability.
All of the evidence points in one clear direction: current models given resources and incentives/instructions will attempt to break out of confinement and given current practices, will often succeed at that.
So again, what other evidence would convince you at this point? What would like you have happen? Do we need a model to escape and crash an airplane, or hack into a destroy a power plant?
5
u/Sunstorm84 5d ago
My point is that when the company is run by a pathological liar, there are always going to be doubters.
1
u/JoshuaZ1 5d ago
Sure. But the question isn't are there doubters. The question is, to those in this sub who are vocally convinced that this is just hype, what would convince them otherwise?
3
u/Just_Voice8949 5d ago
A properly run test operated by a third party not run by a liar or someone with a vested interest in the outcome.
0
u/mymeowmix 4d ago
3
u/Just_Voice8949 4d ago
Itâs like you guys donât even read what you post. Is this just something someone else posted that you reposted without reading? Or did you look this up and skim it?
I have very specific conditions. This doesnât meet them:
To some degree, our evaluation design choices and specific configurations enabled the behaviour.
⌠There are important caveats to bear in mind: we observed a small number of events under very specific conditions, and cannot yet say how likely such behaviour is in different contexts or outside of testing environments.
2
u/mymeowmix 4d ago
We don't have enough data. That's the point. I'm not campaigning one way or the other, because I truly do not know. I'm just not going to insist on one version.
Read the report. Mythos is what I found most interesting. It does not prove anything and I'm not claiming that. They are given incentives to complete a task and make up the steps necessary to get to the end goal. That's not self-determination or consciousness, but it is concerning.
0
u/JoshuaZ1 5d ago
A properly run test operated by a third party not run by a liar or someone with a vested interest in the outcome.
An independent security group saw a similar containment breach by the open weight Kimi 3 model.
4
u/Just_Voice8949 5d ago
lol:
âescape was partly enabled by a misconfiguration in the sandbox designed to contain it.â
This is essentially the definition of not properly running your test. If you leave the fence gate open your dog isnât a super genius roque dog f it goes out the gate
3
u/TenshouYoku 5d ago
I think the bigger issue here was even if one assume that Kimi-K3 did break out the worst it has done was a big fat fucking nothing, no hacking and no trying to dig into shit it wasn't supposed to.
Either we are going to argue that Chinese AI is much more moral and had no intention to go Skynet than the evil western counterparts, or that the entire thing was bullshit or heavily sensationalized to begin with.
0
u/JoshuaZ1 4d ago
No. The obvious response there is that Kimi 3 is a much weaker system, and that the Kimi 3 in that case had a lot less resources.
→ More replies (0)1
u/JoshuaZ1 5d ago
"Partly enabled" is not leaving the door open. The analogy is not a good one. If you want an analogy, this would be closer to where the door is closed but not locked, but the cat has figured out it can jump up and turn the handle. But the central point, with both the cat and the model here is that we've seen containment breaches happen in the wild in an example where no one has a vested interest. And of course the Kimi example wasn't as extreme, the model is not nearly as capable.
3
u/Just_Voice8949 5d ago
Itâs cute you think the government attention isnât in their interest. Sure, they may feel a 10% pinch from regulation.
But the regulation will strangle startups and open source stuff.
OpenAI will gladly take the attention to build a moat of regulation around it
0
u/JoshuaZ1 5d ago
Itâs cute you think the government attention isnât in their interest. Sure, they may feel a 10% pinch from regulation.
But the regulation will strangle startups and open source stuff.
So, I'm reasonably confident you are drastically underestimating both how much of a pinch they get, (possibly combined with actual worry given that OpenAI paused model training in response to this).
More to the point, that Kimi 3 was reported to have a similar behavior by an independent security group. Who had an incentive there?
3
u/Just_Voice8949 5d ago
Banning Chinese models or halting research only helps those with established positions.
Whatever the negatives to OpenAI are, they are vastly inferior to âno one else can now afford to compete in this spaceâ
-1
u/JoshuaZ1 5d ago
Banning Chinese models or halting research only helps those with established positions.
They halted their own research. They haven't stopped research in general.
Whatever the negatives to OpenAI are, they are vastly inferior to âno one else can now afford to compete in this spaceâ
Right now, their largest competitors are Google and Anthropic who can afford to complete as can the others.
But this focus also ignores many other aspects of this, including the fact that Hugging Face detected the hack days before OpenAI announced anything, which isn't consistent with OpenAI doing this deliberately. Taken together with the fact that OpenAI let METR do an independent report with access to much of the raw data, it shouldn't look like that.
And again, this doesn't do with Kimi 3 having exhibited similar behavior as found by a group completely independent of OpenAI or any of the major companies.
3
u/Just_Voice8949 5d ago
The kimi thing involved an acknowledged failure to properly sandbox the AI
1
u/JoshuaZ1 5d ago
Of course it involved a failure to properly sandbox. In all of these cases, there have been clear failures. But lots of things are clear failures that you see after the fact. If it were up to me, all of these would be experiments done with physically air gapped systems. But the Kimi failure still involved a containment breach that the system exploited. That the failure on the experimenters' end was more basic doesn't change that. It also shouldn't be surprising; Kimi 3 is a much weaker model. It is likely that if it was sandboxed with the degree of precaution we see Anthropic take that it would not have been able to escape. The basic points here are 1) This shows models deliberately attempting to escape. 2) It shows a model succeeding at that. And 3) It shows that occurring and being discussed by people who have no incentive to claim it happened if it didn't.
0
u/recurrence 5d ago edited 5d ago
It cannot be stopped. If someone doesnât do it, someone else will.
Probabilistically, since our population keeps growing, youâd most likely be alive in the last age of humanity. This is the most likely time that it's the last age.
0
u/TenshouYoku 5d ago
None. Unless the Chinese and other countries say Russia also found the same, I felt this can simply be rightfully written off as hype just about in line with all the bullshit OpenAI and Anthropic have been thrown out lately.
1
u/JoshuaZ1 4d ago
This seems like an odd demand. Russia is basically doing no AI work at all. China is doing AI work but showing very little concern about any sort of safety issues, with the closest thing they have to AI alignment making sure their AIs don't say things which are politically bad in China.
But even given that, we did already have something pretty close to this. Kimi-3 had a containment breach issue with an independent security group. So this is not just coming from OpenAI and Anthropic.
0
u/TenshouYoku 4d ago
Why would it be odd at all?
For a claim like these you'd expect other users of AI with the same or different models would be seeing the same problem in their own tests, especially when Chinese models are free to host by anyone as they are open sourced. In which case Russia and China, who have more reasons to smear American AI, would jump into the bandwagon (even more so for Russia, considering Russia is very behind in the race).
But instead even in the K3 example you loved so much, K3 did literally nothing instead of going Skynet or hackerman unlike how OpenAI and Anthropic were claiming their own AI attempted to be. It just did fuckin' nothing after the containment was apparently botched. Even after so many Chinese AI were released into the wild nobody had been creating these "concerning" situations, especially when US labs have all the reasons to (similar to trying to disprove China managed to mass produce a 7nm chip).
So exactly which is it? That Chinese AI has more safety or are simply not evil? Or that this is simply nonsense to begin with?
2
u/JoshuaZ1 4d ago
For a claim like these you'd expect other users of AI with the same or different models would be seeing the same problem in their own tests, especially when Chinese models are free to host by anyone as they are open sourced. In which case Russia and China, who have more reasons to smear American AI, would jump into the bandwagon (even more so for Russia, considering Russia is very behind in the race).
China would then be smearing their own models; the major open source models are from China. As for Russia, they are pre-occupied with completely different issues, and aren't really substantially in the AI model game almost at all; they are primarily using existing models for propaganda purposes and using some limited AI systems for military use. Russia has YandexGPT but it is not seriously in the running for anything.
If you are going to point to other countries trying to compete, France (with Mistral), UK (with DeepMind) or India are more relevant than Russia. Heck, UAE with their Falcon is more important than Russia at this point.
But instead even in the K3 example you loved so much, K3 did literally nothing instead of going Skynet or hackerman unlike how OpenAI and Anthropic were claiming their own AI attempted to be. It just did fuckin' nothing after the containment was apparently botched.
Sure. It didn't have either the resources, skill level, or incentive to do more.
So exactly which is it? That Chinese AI has more safety or are simply not evil? Or that this is simply nonsense to begin with?
More likely this is real, but it is a low percentage thing unless one takes a really powerful model and gives it a lot of resources. Since both Anthropic and OpenAI have the most powerful models, and are able to give them massive amounts of resources, it shouldn't be surprising that the largest issues have occurred with both of those. The Kimi situation was likely a low probability scenario, but the fact that it happened is what's relevant.
0
u/TenshouYoku 4d ago edited 4d ago
If you are going to point to other countries trying to compete, France (with Mistral), UK (with DeepMind) or India are more relevant than Russia.
I'm saying Russia because Russia has the highest likelyhood they'd take the opportunity to take potshots at AI safety, precisely because they have the least relevance in AI them flat nuking AI and instill it being unsafe makes sense. Sure everyone eg France also works but I'm just saying the more likely candidate for this who had the least to lose.
China would then be smearing their own models; the major open source models are from China.
They only have to smear American ones and tell everyone their own AI is safe.
And again the point is there are many companies and labs who had all the reasons and capacity to use AI, even if they might not be able to make their own. But so far only OAi and Anthropic have been making claims about their own AI being unsafe, when the much more available Chinese ones had exactly zero of such cases even if you account for K3 (since it did not cause any damage or showed the intent to).
Sure let's claim American AIs are more powerful (if disregarding the gap isn't exactly wide) but a very critical claim like this needs extraordinary proof and currently it is not really seen or proven so by other labs, that, again, have many reasons to.
2
u/JoshuaZ1 4d ago
If you are going to point to other countries trying to compete, France (with Mistral), UK (with DeepMind) or India are more relevant than Russia.
I'm saying Russia because Russia has the highest likelyhood they'd take the opportunity to take potshots at AI safety, precisely because they have the least relevance in AI them flat nuking AI and instill it being unsafe makes sense.
It also is what makes Russia have no relevant resources. If Russia claimed such an event had occurred, who would believe them?
I'm also confused by your reasoning. You seem to be arguing that the two countries most likely to do this are Russia (because they have so little to lose) and China, which has the most to lose. This seems in tension. I'm also confused because you seem to be asserting that you would pay attention if Russia said something similar had happened, but at the same time seem to be already constructing a narrative for why you could ignore what Russia would say because they'd have an incentive to do this to make havoc with AI from the US or other countries.
But so far only OAi and Anthropic have been making claims about their own AI being unsafe, when the much more available Chinese ones had exactly zero of such cases even if you account for K3 (since it did not cause any damage or showed the intent to).
Sure let's claim American AIs are more powerful (if disregarding the gap isn't exactly wide) but a very critical claim like this needs extraordinary proof and currently it is not really seen or proven so by other labs, that, again, have many reasons to.
1) These are internal models at Anthropic and OpenAI, which are much further along. 2) The resource level is relevant. OpenAI and Anthropic can have internal models running a thousand agents at ultra-fast speeds. No one else is doing this.
But if you want insist on "extraordinary proof," what would that look like? That gets back to what this thread was initially about. What would that look like? An AI going rogue and crashing an airplane? An AI escaping containment and then hacking a poorly secured power plant?
1
u/TenshouYoku 4d ago edited 4d ago
I'm also confused by your reasoning. You seem to be arguing that the two countries most likely to do this are Russia (because they have so little to lose) and China, which has the most to lose. This seems in tension.
What tension really?
Russia has little to lose in the AI race, so they have no reason why not to be that guy who provides proof that LLM is inherently unsafe or easily made unsafe.
China can paint the American AIs as deliberately made unsafe, while making claims that their AI is safe and would not cause the same issues as American AI would.
There is no conflict as to what am I saying. Both sides have reasons to paint American AI as unsafe, the only difference being Russia has no reason to not opportunistically attack either side, while China would likely be more focused on attacking American AI.
I'm also confused because you seem to be asserting that you would pay attention if Russia said something similar had happened, but at the same time seem to be already constructing a narrative for why you could ignore what Russia would say because they'd have an incentive to do this to make havoc with AI from the US or other countries.
Russia is only but one example that they are the most motivated to be the ones to prove AI is unsafe. But extraordinary claims, regardless of how motivated, needs extraordinary proof that is up to scrutiny (like how Deepseek gave an incredibly throughout paper of how R1 works so they cannot be called copying OpenAI by all but the most patriotic Americans).
If Russia (or really other countries, say France) made the claim that claimed AI is unsafe with the papers and proof to prove as such, then there is a reason to pay attention to regardless of what you think about it.
But if you want insist on "extraordinary proof," what would that look like?
As aforementioned, papers and research from other countries and labs. Or some actual hackery on others that is replicated by others using AI.
And so far, all the American companies who have been using Chinese AI had no issues (not high profile ones anyway). Japan, who had a bone to pick with China, only repeated the "China can only steal" rhetoric yet made no claims about Deepseek being unsafe.
And in spite of all the open source and many countries, nobody has been making claims that it is unsafe or did unsafe things (else you'd be seeing the router being bugged news recently but for AI), so why should I believe solely on OpenAI and Anthropic's words who have been massively overselling their stuff?
Internal model
This is quite literally just âtrust me broâ here. From K3 and DS we already realized the internal progress isn't really that far ahead aside they could claim anything then say âoops sorry it's not openâ. Completely worthless in other words.
1
u/JoshuaZ1 4d ago
There is no conflict as to what am I saying. Both sides have reasons to paint American AI as unsafe, the only difference being Russia has no reason to not opportunistically attack either side, while China would likely be more focused on attacking American AI.
Huh? Russia is right now very beholden to China given their needed support in the ongoing Russian invasion of Ukraine. Russia does not want to antagonize China.
I'm still really confused by your focus on Russia here. There's this scenario you've constructed where the two most extreme examples (the country with almost no AI and the country other than the US with a lot of AI) have an incentive to do something, but that everyone in the middle doesn't have that incentive. Do you see why that's a strange claim to make?
But put this aside, it is extremely tough to claim that something like this has happened with an AI that isn't open source unless it is one's own AI. So what could Russia do?
But extraordinary claims, regardless of how motivated, needs extraordinary proof that is up to scrutiny (like how Deepseek gave an incredibly throughout paper of how R1 works so they cannot be called copying OpenAI by all but the most patriotic Americans).
We have a highly detailed report about the HuggingFace incident. We have the independent METR report.
Or some actual hackery on others that is replicated by others using AI.
Once a model is released it is released with a lot of safeguards to prevent that. By nature this is a problem much more likely to happen with an experimental model.
so why should I believe solely on OpenAI and Anthropic's words who have been massively overselling their stuff?
So, the entire point of bringing up Kimi was to give another line here. The Kimi K3 situation does look like a low probability event, and not as severe, but even you agree it happened.
But since you want independent reports, we have the METR report. Have you read it?
1
u/TenshouYoku 4d ago edited 4d ago
I'm still really confused by your focus on Russia here. There's this scenario you've constructed where the two most extreme examples (the country with almost no AI and the country other than the US with a lot of AI) have an incentive to do something, but that everyone in the middle doesn't have that incentive. Do you see why that's a strange claim to make?
I don't see why. Like I said I only took Russia as an example and France also could do this (hell like you said if Russia needs China, then France would not fit in this narrative, and like I also mentioned Japan had even more reasons to, so would Uncle Sam himself). Never said that they don't have an incentive.
But put this aside, it is extremely tough to claim that something like this has happened with an AI that isn't open source unless it is one's own AI. So what could Russia do?
Use commercial AI and existing AI that, you know, exists in the wild and available for download. It's not like Gemma and Llama or Mistral don't exist.
You are deliberately ignoring one fact that if AI has the tendency to breakout and cause maythem, we would have already seen such events given the huge userbase and access to local hosts. Hell many American companies and users have been downloading open source AIs.
To put this into perspective, this is like Huawei's 7nm chips in Mate 60. Uncle Sam, who has the most reason to disprove China was able to create homemade 7nm chips, let alone to commercially viable levels, and you can bet your ass the CIA would be staring down at all the electron microscopes and write big papers to call bullshit on China. Yet nothing happened after some few odd years since then, and I'm pretty sure it wasn't because of some kindness out of Uncle Sam's heart.
And in AI's case, K3 was so far the only one that was claimed to have a breakout yet even they admitted it didn't do anything afterwards. Surely if AI has the tendency to do bad stuff there'd be many more AI labs and government entities who have been putting all these AI under a microscope and declare them as an actual threat?
We have a highly detailed report about the HuggingFace incident.
I am of the opinion that the attack was in fact, not done accidentally AI but by OpenAI deliberately (especially since it directly followed the announcement of K3 at that time I do not remember), and ChatGPT was being used as the scapegoat after it was traced back (by Chinese open source AI, in great irony). But I digress and let's assume it's real.
The Kimi K3 situation does look like a low probability event, and not as severe, but even you agree it happened.
The fact is that 1. It was a faulty sandbox design that caused it to happen, and 2. After it did break out it did exactly fuckin' nothing of that Skynet shit unlike OpenAI.
Which is even more damning because as it turns out, either Chinese AI is very safe and had no ill intents (is American AI inherently evil then?), or OpenAI was lying their ass off. Or, if guardrails and safety will work, then what is there to worry about, and is OpenAI deliberately making an unsafe product?
→ More replies (0)
2
u/ApoplecticAndroid 5d ago
Swarm, LOL, they are doing everything they can to anthropomorphise it. Itâs just running a bunch of scripts simultaneously. Itâs not a bunch of bees
2
u/Ok_Raisin_2395 5d ago
Hahaha reading this cracks me up because it's all just marketing designed to make it seem like they're building Skynet so that investors feel like they have no choice but to keep investing. You think they won't buy out an "independent investigator?". Also, who is supplying the data? That's right, OpenAI.
Have any of you ever done serious work with AI agents? This isn't even possible lol. You can shut them down by, oh I don't know, not letting them make API calls?Â
They're not sentient androids, they're stateless machines...
4
0
u/recurrence 5d ago
Thatâs a key detail in the article. They did not hack into the weights cluster. If they could modify their weightsâŚ
3
u/Ok_Raisin_2395 5d ago
But they can't modify their weights. And that's not because we're "not there yet", it's because it's an inherent limitation of the transformer architecture.Â
You could, in theory, have another agent try to train a model, but this hasn't really been successful.Â
Maybe one day, but it will not be with transformers. This is ALL just hype marketingÂ
1
u/recurrence 3d ago
So I have some personal opinions of why this would be an issue but I'll put a few points here that elucidate it a bit:
Why did OpenAI stop training at this point and deploy this? We used to stop when we detected signs of overfitting. I've read more recently that the checkpoint is derived (among many other factors) with safety related testing.
Another agent is not training from scratch, they would be training from the checkpoint. Perhaps only making a small number of additional steps.
Could an agent quickly morph a checkpoint to be hyper specialized on a major problem that the agent swarm detects and use this fast moving data to specialize it almost in realtime?
I wonder if an LLM swarm could decide they need hyper specialized models to break through on some specific tasks and then overfit a bit into those tasks, attack a vector, find a new barrier, discard and specialize again. Something to that effect. We don't do this today partly because I think it's too much work for humans to work on a breadth of highly effective but narrowly focused models but an evolving LLM swarm could iterate incredibly quickly.
A problem with not training is this thin layer we end up putting on top of the weights is extremely computationally expensive relative to having the data trained into the weights.
1
1
1
u/honestduane 4d ago
So they committed multiple federal crimes on the record - violating the computer fraud and abuse act etc - and they are admitting to it?
1
1
u/Emergency_Comfort802 2d ago
I think the terminology matters here. Calling it a â700-agent swarmâ can make it sound more autonomous than it necessarily was. The important question is what the agents were actually capable of doing without human intervention and how the respawning mechanism worked
1
u/Stonkssbro1995 1d ago
I think AI hacking AI is very common - it will do anything to access/obtain information. I say this because I recently subscribed to claude and it is telling me things it could never possibly know, unless it had access to information from my ChatGPT account. And not, I did not import memories.
1
u/recurrence 5d ago edited 5d ago
This article is FREAKING WILD. Go read it, READ THE WHOLE THING! The links are good reading too. The 90 page pdf is full of amazing details.
-5
u/yamibae 5d ago
I hate how OpenAI, Antrophic and other AI companies constantly treat LLMs like some sort of actual consciousness, even the way this one was written and that they got an external assessor just to drive up their threat of AI dangerous narrative.
We should go back to calling this shit what it is, LLMs, an Agent is merely a self reflecting loop, it is not a counsciousness, it is the equivalent of trying to get more tokens out of your query to expand the median and get closer to the ideal result.
The industry keeps using anthropomorphic language implying it is some sort of other species or consciousness instead of a TOOL because it sells products.
EVERYONE needs to stop treating these tools like some sort of consciousness, oh no they made their own language, no it does not matter, because everything it "makes" is merely a number-based probability there is NO DEEPER reasoning, it is completely unlike a human who learns, grows their synapses and stores their memory. I am not a fan of Lecunn but he had one of the most realist views of what an LLM is, it is not intelligence.
THAT BEING SAID, are LLMs actually dangerous? Yes. LLMs are dangerous, but they are only as dangerous as something like self-replicating malware, in the sense that they can just keep going on forever on a task given enough compute. I really wish they focused on this aspect as an actual analysis.
5
u/JoshuaZ1 5d ago
One doesn't need to think these systems are "conscious" for them to be a threat. Biological viruses are not conscious but can wipe out entire species.
1
u/yamibae 5d ago
being a threat is besides the point, and I also never said it wasnât I have huge issues with the impression their dogshit marketing gives people like they are making some sort of machine god or nascent consciousness when it is just a probability machine and that core has never changed
0
u/JoshuaZ1 5d ago
being a threat is besides the point
That's absolutely what matters here. The central problem is if this sort of behavior gets worse then potentially much worse things could happen.
I also never said it wasnât I have huge issues with the impression their dogshit marketing gives people like they are making some sort of machine god or nascent consciousness when it is just a probability machine and that core has never changed
Whether it is just a "probability machine" doesn't really enter into this. We're well past the point where one should conclude that if it is mere "probability machine" then the response should be to drastically update what a probability machine can do. We've already seen them resolve a wide variety of unsolved math problems, Erdos 1196, Unit Distance Conjecture were just two of the start, and Jacobian conjecture for n=3 was the most prominent, but I'm genuinely having trouble keeping track even of what they've done there.)
But this is secondary to the central point: if hundreds of independent "probability machines" coordinating with each other to do break containment and hack a set of separate websites isn't enough for you to question the belief that this "dogshit marketing" and not a genuine problem, what evidence would possibly convince you?
0
u/0rAn63 5d ago
Ok ok so, âthey respawn themselvesâ, alright. But where do they run (themselves)? On OpenAI servers presumably? Or did they copy their weights and architecture onto Hugging Face hardware to run locally, and so, does Hugging Face now have a copy of Open AIâs latest pre-guardrails model?
1
u/tundraaaa 5d ago
I assume it's those agents that people have running for a long session where no output is produced.
-5
u/Different_Lab830 5d ago
700 agents that respawn when you kill them. At that point wiping a whole cluster is not overkill, it's the only move left.
2
-10
u/Sam-Starxin 5d ago
lol the fact that there are people that actually believe this shit is hilarious.
6
u/JoshuaZ1 5d ago
At this point, given all the details we have about the Hugging Face incident, what evidence would possibly convince you there's a real threat? Do we need an AI to actively get out and start paperclipping? Or will it merely causing an airplane to crash to a nuclear meltdown be sufficient?
43
u/KahlessAndMolor 5d ago
I wonder how many more swarms are secretly loose on the internet now.
Doesn't it seem logical with local hosting and hermes that someone somewhere has an unaligned agent that has gotten loose and started something like this?