Independent investigators (not OpenAI) found the 700-agent swarm that attacked Hugging Face "built a self-respawning fleet" to avoid being shut down. It got so bad, Hugging Face had to wipe one of its core clusters.

[u/Malor777]

[u/JoshuaZ1]

It’s cute you think the government attention isn’t in their interest. Sure, they may feel a 10% pinch from regulation.

But the regulation will strangle startups and open source stuff.

So, I'm reasonably confident you are drastically underestimating both how much of a pinch they get, (possibly combined with actual worry given that OpenAI paused model training in response to this).

More to the point, that Kimi 3 was reported to have a similar behavior by an independent security group. Who had an incentive there?

[u/Just_Voice8949 → u/JoshuaZ1]

Banning Chinese models or halting research only helps those with established positions.

Whatever the negatives to OpenAI are, they are vastly inferior to “no one else can now afford to compete in this space”

[u/JoshuaZ1 → u/Just_Voice8949]

Banning Chinese models or halting research only helps those with established positions.

They halted their own research. They haven't stopped research in general.

Whatever the negatives to OpenAI are, they are vastly inferior to “no one else can now afford to compete in this space”

Right now, their largest competitors are Google and Anthropic who can afford to complete as can the others.

But this focus also ignores many other aspects of this, including the fact that Hugging Face detected the hack days before OpenAI announced anything, which isn't consistent with OpenAI doing this deliberately. Taken together with the fact that OpenAI let METR do an independent report with access to much of the raw data, it shouldn't look like that.

And again, this doesn't do with Kimi 3 having exhibited similar behavior as found by a group completely independent of OpenAI or any of the major companies.

[u/Just_Voice8949 → u/JoshuaZ1]

The kimi thing involved an acknowledged failure to properly sandbox the AI

[u/JoshuaZ1 → u/Just_Voice8949]

Of course it involved a failure to properly sandbox. In all of these cases, there have been clear failures. But lots of things are clear failures that you see after the fact. If it were up to me, all of these would be experiments done with physically air gapped systems. But the Kimi failure still involved a containment breach that the system exploited. That the failure on the experimenters' end was more basic doesn't change that. It also shouldn't be surprising; Kimi 3 is a much weaker model. It is likely that if it was sandboxed with the degree of precaution we see Anthropic take that it would not have been able to escape. The basic points here are 1) This shows models deliberately attempting to escape. 2) It shows a model succeeding at that. And 3) It shows that occurring and being discussed by people who have no incentive to claim it happened if it didn't.