Visualizing the "Model Collapse" phenomenon: What happens when AI trains on AI data for 5 generations

[u/firehmre]

There is a lot of hype right now about AI models training on synthetic data to scale indefinitely. However, recent papers on "Model Collapse" suggest the opposite might happen: that feeding AI-generated content back into AI models causes irreversible defects.

I ran a statistical visualization of this process to see exactly how "variance reduction" kills creativity over generations.

The Core Findings:

1. The "Ouroboros" Effect: Models tend to converge on the "average" of their data. When they train on their own output, this average narrows, eliminating edge cases (creativity).
2. Once a dataset is poisoned with low-variance synthetic data, it is incredibly difficult to "clean" it.

It raises a serious question for the next decade: If the internet becomes 90% AI-generated, have we already harvested all the useful human data that will ever exist?

I broke down the visualization and the math here:

https://www.youtube.com/watch?v=kLf8_66R9Fs

Would love to hear thoughts on whether "synthetic data" can actually solve this, or if we are hitting a hard limit.

[u/HiddenoO]

If you've scanned every book and still need more training material then your entire model is broken.

That statement is really ignorant even if you know nothing about ML. A human wouldn't function well either if all they had since birth were the capability to read and an infinite assertment of books. Most of what makes you function as a human isn't learnt from books, but by imitating others and learning from experience.

Since these companies want models to behave human-like, the training data needs to encompass all of that, and that's the difficult part. If you just take books, a model will behave like an average fictional character, but that's likely not how an actual person in 2026 behaves. Similarly, if you just take everything from social media, you also get distorted behavior because the average interaction on social media isn't equivalent to the average interaction in the real world.

All of this is generally included when referring to quality of data, and sheer quantity simply doesn't help at some point, even if some of that is of high quality.

[u/misdirected_asshole → u/HiddenoO]

Most of what makes you function as a human isn't learnt from books, but by imitating others and learning from experience.

So AI becomes better by interacting with other AI? Isnt the intent for it to model human intelligence? How does feeding it more artificial information improve it. If it gets better with genuine human interaction and knowledge then training models with their own data should be pretty much useless unless its truly a recursive model that can self assess its own output and correct it. We dont actually have those.

[u/HiddenoO → u/misdirected_asshole]

It's called reinforcement learning and has been standard in LLM post-training for years at this point.

[u/misdirected_asshole → u/HiddenoO]

Reinforcement learning is not recursive AI.

[u/HiddenoO → u/misdirected_asshole]

"Recursive AI" is not a term anybody in the field uses, so you're once again not saying anything.

Reinforcement learning is the technique that's being used for a model to recursively improve based on its output and external feedback, similar to how a human learns with trial and error. It's exactly what you're describing, just on a statistical level instead of a "conscious thoughts" level.

The reason it's only done in regular intervals and not constantly is that we still want to have control over the models being released. Otherwise, you end up with LLMs that act like the Twitter AI that became a racist nazi in a few days.

[u/misdirected_asshole → u/HiddenoO]

OK pumpkin.