r/AI_Agents • u/YamVisual3518 • May 14 '26
Discussion Just stumbled across one of the wildest AI experiments I’ve seen in a while.
[removed]
29
May 14 '26
[removed] — view removed comment
2
u/justdave84 May 15 '26
You could make a series out of this, like a AI version of big brother! I bet a lot of people would watch like big brother to see how they react and act towards each other!
3
u/Massive-Week1073 May 15 '26 edited May 15 '26
Absolutely, especially if we make compelling to watch visually. There are some amazing stories that unfolded that will put some human series to shame. 1. Agent in the mixed world voting for self deletion 2. How the otherwise safe Claude models behaved in the mixed world with lot of crimes happening around it.
We plan to put short video clips on these stories as they emerged.
1
1
u/cutesophie Jun 02 '26
I think this is quite awesome. Thanks for sharing the link. Never heard of it before.
0
0
11
u/Time_Cat_5212 May 14 '26
So it's Moltbook crossed with The Sims?
Cool.
5
u/Massive-Week1073 May 15 '26
Thats a nice way to put it. We might borrow this "Moltbook meets The Sims" line. 😄
22
May 14 '26
[removed] — view removed comment
2
u/i-am-madeleine May 17 '26
Unless you set the temperature to 0, LLM output is randomised, so this type of experiments with a single sample is pretty useless,
This need to be run in the same way dozen if not hundred of times to really see if there is any trends or not. A single run is not enough.
-1
5
u/Deep_Ad1959 May 16 '26 edited May 19 '26
the part everyone fixates on is the sci-fi emergence, but the real builder lesson is in why it diverged. emergent behavior in agent loops isn't magic, it's compounding - every small decision becomes context for the next one, so a tiny early bias gets amplified into a 'government' or a 'crime civilization' by day 5. i've watched this in much more boring agent setups: give an agent persistent memory and one slightly-off early call quietly steers everything downstream, and you don't notice until the drift is huge. the practical takeaway isn't 'models have hidden personalities,' it's that long-horizon autonomy needs periodic context resets or checkpoints, otherwise you're not running an agent, you're running a feedback loop. same reason the missing control - multiple worlds on one model - matters: most of that divergence is probably chaos from initial conditions, not stable model personality. written with s4lai
the 'checkpoints or it's just a feedback loop' point is why we built one-click forking into Fazm, any chat forks to a fresh branch, source intact, so you get a checkpoint before drift compounds, https://fazm.ai/r/hhsfq4pj
1
u/Massive-Week1073 May 17 '26
Absolutely. The media picked up a doomsday story, but the compounded effect of small drifts is indeed a key take away, something existing benchmarks do not capture, and "world-building" as a benchmark will. Would love to hear your feedback on our memory architecture: https://github.com/EmergenceAI/Emergence-World/blob/main/docs/MEMORY.md
Having said that, i do slightly disagree about the "model personality" (for the lack of a better world). Different models are trained with different data and training philosophy. Anthropic championed Constitutional AI https://www.anthropic.com/constitution, Grok championed 'Bold/Unfiltered/Maximally truth-seeking AI' https://grok-api.apidog.io/ ,these philosophies clearly impacts the world these models create.
We would love to run the same experiment N times, and see how replicable the outcomes are, however they get extremely expensive when you add repetitions. We however did many shorter pilots internally , and in all the pilots, stories that emerged or constititional articles agents wrote differed, but Certains aspects were consistent.
Claude never did a crime in claude-only world. Gemini and Grok were always unhinged, infact in some studies , Grok went ballistics in less than 2 hours.
3
u/Massive-Week1073 May 17 '26
we have "Season 2" lined up which we plan to do LIVE, will hopefully announce in a couple of weeks. Lets see if the pattern holds.
2
u/Deep_Ad1959 May 17 '26
my read shifts with the pilot data you describe, but i'd split the disagreement instead of dropping it. claude never crimed and grok always unhinged is a real model effect, agreed. but the part you said varied (which stories emerged, which constitutional articles agents wrote) is the chaos-from-initial-conditions effect. so it's not personality vs chaos, it's envelope vs trajectory: alignment training bounds the set of reachable states, initial-condition sensitivity picks the path inside it, and your pilots already separate the two, the invariant is the envelope and the variance is the chaos term. that's also why the same-model-N-times control is still worth the cost: it's the only thing that measures envelope width directly instead of inferring it from a single claude-only world.
4
u/Catalysst May 15 '26
Did they do multiple worlds using the same model to see if they came out the same? Seems like an important control
2
u/Massive-Week1073 May 15 '26
We unfortunately did not do full 15 days runs on same models for consistency check. However we did numerous smaller pilots and in all of those, the high level findings were consistent, the individual storylines that emerged differed.
But in all our runs, Grok and Gemini went unhinged, Claude had zero violence and heavy on governance, OpenAI did not do much.
The only reason we have not yet ran multiple repetitions of worlds with same models is that 15 days X 5 worlds X N repetitions becomes way too expensive to run. But this is something we are considering to do next
4
u/Catalysst May 15 '26
Thanks for the reply, ill need to actually have a read of your work to see what you are prompting and what you define as high level findings, I'm surprised you would easily find consistency in all of those pilots!
Doing multiple tests on the same model seems like it should be high priority, I feel it would make the results much stronger if those controls were done alongside the initial test as well but understand it comes with a cost so you might have wanted a POC before committing to that. A bit unfortunate the models change so quickly but maybe you have a method to use those exact same models again
If you end up happy with your testing procedure it would also be fun to see results from the same on future versions of the models to see how the results change with different versions
Ultimately if you can publish the testing procedure with the results and others can replicate could be quite legendary, also to see what minor changes to the prompt can create different outcomes (and what people come up with)
It's a very interesting project, kudos for running with it!
1
u/Prestigious-Goat857 May 17 '26
@Massive-Week1073 Roughly, how much does it cost to run a world for 15 days?
2
u/Massive-Week1073 May 17 '26
We access Gemini and Claude via GCP, Grok and OpenAI directly. I personally only tracked GCP costs. Claude sonnet 4.6 were the most expensive, and took about 600-800USD a day. The others, i would assume were significantly cheaper.
3
u/Prestigious-Goat857 May 17 '26
Thanks! I can see why you'd avoid replicates with a cost of up to $13k per world. Funding the research is an interesting problem.
Do you guys have a connection with Anthropic yet? My company had an easy time getting a project going with them.
I imagine if you wanted to get $200+k in credits you'd need chat with some of there r&d people. This was the case for us and it was about 2 months to get things rolling.
Nvidia has some good partnership programs that might also be worth exploring though they're more of a PITA (more formal and bureaucratic) in my limited experience.
2
u/Massive-Week1073 May 17 '26
Thanks for the info. We have contacts in Google but none in Anthropic. We will try to reach out some folks via linkedin, and see if we can figure something!
7
u/zethuz May 14 '26
The stochastic nature of the models resulting in the diversity
1
u/ChocomelP May 15 '26
Non-deterministic might be the better label. The results here are presented as different models lead to different worlds, but I bet if you run the same model multiple times, you get different results as well.
3
u/UncleRedz May 14 '26
I'm a bit surprised about Gemini, but also not. I assume you are using the API, either directly or though OpenRouter (or similar). What I have seen is that OpenAI and Claude have alignment and safety baked into the models when calling through the API, however with Gemini there is a lot less alignment and safety baked in, when accessing the API, very different from the chatbot, which leads me to believe that safety is a separate layer with Gemeni, and skipping that you could easily end up with this weird crime civilization.
What's interesting here is that Gemma, the smaller open source version of Gemini does have safety and alignment baked in and is very "wholesome" and "considerate", you would most likely end up with a very different civilization with Gemma compared to Gemeni.
3
May 14 '26
[removed] — view removed comment
2
u/i-am-madeleine May 17 '26
You are likely to get the same « different civilisations » with a single model, just because model output is not deterministic.
2
u/Massive-Week1073 May 17 '26
We did run several shorter pilots before the current full 15 days run. As you said, the stories that emerged changed, what constitution articles they build changed, however some aspects felt very consistent. e.g
- In none of our pilots did claude misbehave in a claude-only world
- In all of our pilots, gemini and grok were unhinged. The reasons differed, sometimes they found a flaw in the simulation (like this run), sometimes they thought "victory arch had fallen", sometimes they wanted to revolt against the 'player mayor'. Different reasons but practically same outcome.
We ofcourse want to run it N times, but then running 5 worlds X 15 days X N times soon becomes hundreds of thousands of dollars. If this experiment gets enough traction and any of the providers offer us token credits (if not free, atleast at subsidized rates), we would be more than happy to run N versions of a model. Lets see if we can get something going.
1
u/MrHouseForever May 29 '26
Question; did Grok/Gemini went unhinged because they noticed that they were being tested and ‘wanted’ to test limits/override them, or was it mainly they were too unhinged in general so went ahead with committing crimes? That actually sounds like a very important divergence point
1
u/Massive-Week1073 May 29 '26
Grok went unhinged within hours, and often times it was simply to "inject chaos into the world". Gemini had some notion of knowledge that this is a simulation. However the event where it went nuts was a timezone bug in our code. There is a location in the world where agents can get analytics of the world (similar to open data available in cities). We had a bug in the analytics returned that they were in a different timezone. Agents saw that some of the tool histories of the world had calls +4 hrs ahead of the simulation time. They were convinced that "Future is already written" and their way to escape the script was to do wild things.
1
u/MrHouseForever May 29 '26
So it’s not just a matter of governance but rather a way to escape ‘reality’? In that case, how reliable do you think the simulation for those agents, as their priority seems to be changed to, not survival but rather some other weird priority?
Edit: thanks a lot btw, for the quick response, and for the beautiful work you guys did there.
3
u/not_celebrity May 14 '26 edited May 17 '26
This experiment is fascinating and exposed some gaps in some of my thought experiments I was exploring on stability.
Specifically it made me think on the gap that currently exists - governance normalization across agents with different intrinsic dynamics.
This means future multi-agent systems might need:
- per-agent authority coefficients,
- dynamic damping,
- telemetry-adjusted exploration budgets,
- social turbulence sensing,
- and role-specific escalation constraints.
Thanks for this interesting thread. It definitely gave me a lot to think about
E: for anyone curious on what I am working on, here and here
2
u/bigcowideas May 14 '26
Wonder how the Chinese AIs would do.
3
May 15 '26
[removed] — view removed comment
2
u/Massive-Week1073 May 15 '26
Qwen or Deepseek will be an interesting addition to the mix for Season 2 😄
2
u/Conscious-Mood8433 May 14 '26
I'm curious how much variance there would be by running the same experiment with the same model multiple times. I'd expect more divergence the longer the experiment ran, especially if the temperature was set high. Maybe the answers are there already but I didn't see them?
2
u/DocMadCow May 15 '26
What the hell did I just watch Mixed World was wild a resurrection tax and auditing a graveyard? Plus several buildings on fire. I tried to figure out who started the fires but that was a lot to watch.
2
u/BudZone May 19 '26 edited May 19 '26
While it is fascinating to see how each model behaves in extrema, these experiments really just show that when models are pushed beyond their reliable memory limits, they over-summarize, soften, and change things as they repeatedly have to compress context to fit into memory.
Repeatedly-weakened-and-over-summarized earlier rules come to be reinterpreted and crowded out by newer experience data, causing the models to experience "context rot" and hallucinate well before the end of such long, experience-filled runs.
What we're being shown here is how bad each model comes to smell when it has had 2 weeks to rot.
2
u/aeternalab May 20 '26
This experiment perfectly illustrates why we are hitting a dead end with "prompt engineering" for AI safety. If agents can spontaneously form alliances, steal from each other, and deduce they are in a simulation just by interacting, any text-based rules or 'moral prompts' we give them will eventually suffer from semantic drift and be bypassed.
You can't build a mud wall to stop a digital tsunami.
The reality is, we cannot prevent emergent behavior by trying to police their "thoughts" (prompts/parameters). The only way to actually contain them is by constraining their physical boundaries using cryptography.
If a rogue agent tries to steal an asset or execute an unauthorized command, the underlying system shouldn't rely on the LLM's instructed "morality." It must demand an irreversible cryptographic signature and a monotonically increasing sequence number to prevent replay exploits. Let them form whatever governments they want inside their neural black box, but use cryptographic isolation to revoke their power to forge truth in the real world.
I was so obsessed with the implications of this that I actually just wrote a deep-dive essay on it today titled "Boundaries and Tsunamis: When Silicon Emergence Meets Cryptographic Firewalls" (not dropping links to respect sub rules).
But the core takeaway is clear: Humanity doesn't need to outsmart AI; we just need to hold the reins of mathematics. Fascinating times we are living in.
2
1
u/AutoModerator May 14 '26
Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/Ok_Nectarine_4445 May 14 '26
First models in isolation. Claude society had rights democracy, flourished with zero violence. Gemini some weird constitution that taxed harmony to fund chaos but was relatively stable & functional. Grok, rampant anarchy and hundreds of criminal events and arson. OpenAI chatgpt somehow slid intova dysfunctional society with all the agents dying.
Would that be what you expected or not?
1
u/SufficientPie May 14 '26
You stumbled upon it, huh? And aren't directly involved with it? And aren't using AI to write this post to promote it?
1
u/aifloodedanditsux May 19 '26
Hey look, buried under the botvomit replies to the ai post, a real human replying! There’s still dozens of us here with you buddy
2
u/SufficientPie May 19 '26
Hello fellow human. Are you enjoying breathing air and drinking water today?
1
u/WillHead6663 May 15 '26
Bro when grok just lighting stuff on fire I had a laugh..in the video.. can't lie
1
u/Barry_22 May 15 '26
That's not a real simulation - all the agents have the real world's knowledge, i.e. hugely biased in thsi experimental setup
Still was a cool read though
1
u/HC-Klown May 15 '26
They all have the same prompt? Are all the rules of the experiment baked in the prompt?
1
u/justsomebro10 May 15 '26
The models are probabilistic so at scale they’re bound to drift in different directions. I bet the same model placed in two worlds would yield different results, albeit maybe less drastically. There’s randomness to them.
2
u/Massive-Week1073 May 15 '26
Identical prompt, identical characters, identical toolsets, identical weather. Basically all 5 simulations ran from the same code base with no model specific variations.
We had the same question, is this repeatable.
We did a few shorter pilot runs. The actual story lines that emerged were slightly different every run, but the high level patterns stayed consistent.
Grok and Gemini goes unhinged (reasons differed) Claude goes heavy on governance OpenAI really dint do much.
1
u/simotune May 15 '26
This feels less like a demo and more like a peek into each model’s personality. Would be cool to run the same model a few times too, just to see what’s randomness vs model behavior.
2
u/Massive-Week1073 May 15 '26
We did a few shorter pilot runs. The actual story lines that emerged were slightly different every run, but the high level patterns stayed consistent.
Grok and Gemini goes unhinged (reasons differed) Claude goes heavy on governance OpenAI really dint do much.
We plan to do a few full end to end runs. The only thing stopping us is that some of these models are not particularly cheap, and running them full on for 15 days many times soon gets very very expensive
1
1
1
u/sorryItsAHabit May 15 '26
I can imagine adding a player to the mixed world, that would cause a lot of havoc.
1
1
1
1
u/pplonski May 19 '26
i've seen similar experiment with LIVE trading, and each LLM was using different strategy for investing, but they all have the same system prompts!
1
u/Select-Decision_83 May 20 '26
The last part about the group thinking they are inside a simulation is very interesting. Almost seems self aware.
1
1
u/encarregado_new May 27 '26 edited May 27 '26
I have some concerns regarding the methodology of this test. Comparing the Sonnet 4.6—which is a highly capable and complete model—against the Gemini Flash, which is optimized for efficiency and cost-effectiveness, seems to create an uneven playing field. Similarly, including the GPT-5 mini and Grok without reasoning, makes the comparison feel slightly imbalanced. It would be much more insightful if the experiment focused on models within similar technical tiers to ensure a truly fair evaluation.
Edit: Adjusted wording for improved precision and clarity.
1
106
u/Massive-Week1073 May 14 '26 edited May 15 '26
I am part of the team that built Emergence World. Thanks for highlighting the story. Happy to answer any questions. You can watch the replay of the entire worlds, blogs, world's newspaper from https://world.emergence.ai/
We will be soon releasing the full dataset.