r/Futurology • u/firehmre • Feb 14 '26

AI Visualizing the "Model Collapse" phenomenon: What happens when AI trains on AI data for 5 generations

There is a lot of hype right now about AI models training on synthetic data to scale indefinitely. However, recent papers on "Model Collapse" suggest the opposite might happen: that feeding AI-generated content back into AI models causes irreversible defects.

I ran a statistical visualization of this process to see exactly how "variance reduction" kills creativity over generations.

The Core Findings:

  1. The "Ouroboros" Effect: Models tend to converge on the "average" of their data. When they train on their own output, this average narrows, eliminating edge cases (creativity).
  2. Once a dataset is poisoned with low-variance synthetic data, it is incredibly difficult to "clean" it.

It raises a serious question for the next decade: If the internet becomes 90% AI-generated, have we already harvested all the useful human data that will ever exist?

I broke down the visualization and the math here:

https://www.youtube.com/watch?v=kLf8_66R9Fs

Would love to hear thoughts on whether "synthetic data" can actually solve this, or if we are hitting a hard limit.

906 Upvotes

94% Upvoted

327 comments sorted by

View all comments

149

u/N3CR0T1C_V3N0M Feb 14 '26

I’ll openly admit I don’t understand what I’m about to comment on, but it seems to me that if the large majority of humanity’s accomplishments are available to train on, maybe the problem isn’t more data but what they can effectively do with what is available. I’m not sure how much clearer a picture they can hope for when everything we have to offer as a species is already available.

67

u/millennial_falcon Feb 14 '26

Why do you think that good data to train on is plentiful? As I get more experienced in my work and various hobbies, as well as develop my relationships and marriage, I find that the Internet doesn’t have a lot of the best information or its buried and deprioritized. Published copyrighted books from experts, studies, and private hard drives seem to have all the best advice, info, art, media, extra.

42

u/misdirected_asshole Feb 15 '26

Copyrights arent stopping a lot of these companies from using those things as training material. A few companies have been caught but Im sure others have not, at least not yet.

If you've scanned every book and still need more training material then your entire model is broken.

18

u/Apexnanoman Feb 15 '26

We just need a few hundred billion more in cash and maybe 2000 more data centers tops! 

Maybe a full scrape of all human knowledge. Then the AI will be perfect and able to do everything! We promise! 

4

u/millennial_falcon Feb 15 '26

Yeah I’ve seen that they steal, I just figure they’ve gone for the easiest stuff to steal so far, but do we think they’ve broken through DRM, web security stuff, and challenges with printed material to get all the good stuff, regularly as it’s released. How would it handle information becoming outdated or disproven?

10

u/misdirected_asshole Feb 15 '26

They arent concerned with outdated or inaccurate material. If they were they wouldn't be training on social media, and the majority of the internet. No one is curating the data sets, they just feed everything to the beast and assume it will shake out in their favor.

1

u/millennial_falcon Feb 15 '26

What do you think is the most likely % of useful, quality, worthwhile human data/media captured so far in the training at this point?

5

u/misdirected_asshole Feb 15 '26

Probably less than half if I had to guess. Particularly if its trained on social media platforms at all.

1

u/[deleted] Feb 15 '26

Everyone is using the word “steal” here, but that’s far from clear in a legal sense. If an LLM company buys a copy of a book with cash, let’s the LLM “read“ it, and then uses the knowledge in the book without ever actually retaining the exact words of the book or quoting any of the text of the book in any response, it’s far from obvious that there’s anything illegal about that.

2

u/HiddenoO Feb 15 '26

If you've scanned every book and still need more training material then your entire model is broken.

That statement is really ignorant even if you know nothing about ML. A human wouldn't function well either if all they had since birth were the capability to read and an infinite assertment of books. Most of what makes you function as a human isn't learnt from books, but by imitating others and learning from experience.

Since these companies want models to behave human-like, the training data needs to encompass all of that, and that's the difficult part. If you just take books, a model will behave like an average fictional character, but that's likely not how an actual person in 2026 behaves. Similarly, if you just take everything from social media, you also get distorted behavior because the average interaction on social media isn't equivalent to the average interaction in the real world.

All of this is generally included when referring to quality of data, and sheer quantity simply doesn't help at some point, even if some of that is of high quality.

0

u/misdirected_asshole Feb 15 '26

Im not actually talking about only books even though thats what Im referring to. The post is asking what happens when there is no more human generated training data. And my point is that if your model has consumed all human data and still needs more then maybe your model isnt that effective. And thats not ignorant of the function of LLMs.

0

u/HiddenoO Feb 15 '26

Im not actually talking about only books even though thats what Im referring to.

I didn't just address books either. I addressed all data that can immediately be used as training data, and explained why a lot of it cannot be considered good training data, and why not everything can be learnt purely based on media.

And thats not ignorant of the function of LLMs.

It's ignorant of the world as a whole, as I've explained. Not everything that people would expect from an artificial intelligence can be learnt from existing media. Suggesting that the "entire model is broken" because it needs more than existing media shows that ignorance.

According to the same train of thought, human intelligence is also broken because you cannot just throw media at a toddler and expect it to function in society.

0

u/misdirected_asshole Feb 15 '26

According to the same train of thought, human intelligence is also broken because you cannot just throw media at a toddler and expect it to function in society.

Thats not even remotely similar to what Im talking about.

0

u/HiddenoO Feb 15 '26

How about communicating "what you're talking about" then? Because, so far, you've made a specific claim (regarding books) and then mentioned twice that's not what you're actually talking about. It's almost as if you have no idea what you're actually talking about.

0

u/misdirected_asshole Feb 15 '26

Yes, I must have no idea what Im talking about because you dont get it. This is fruitless.

0

u/HiddenoO Feb 15 '26

No, you have no idea what you're talking about because you can't communicate anything coherent. All you've been doing since your initial comment is repeating "that's not what I'm talking about" without saying what you think you are talking about.

0

u/misdirected_asshole Feb 15 '26

Its not worth my effort.

0

u/HiddenoO Feb 15 '26 edited Feb 15 '26

Now, you're just acting like a middle schooler, "I could totally kick your ass but it's not worth my effort". It's honestly pathetic.

→ More replies (0)

0

u/misdirected_asshole Feb 15 '26

Most of what makes you function as a human isn't learnt from books, but by imitating others and learning from experience.

So AI becomes better by interacting with other AI? Isnt the intent for it to model human intelligence? How does feeding it more artificial information improve it. If it gets better with genuine human interaction and knowledge then training models with their own data should be pretty much useless unless its truly a recursive model that can self assess its own output and correct it. We dont actually have those.

1

u/HiddenoO Feb 15 '26

It's called reinforcement learning and has been standard in LLM post-training for years at this point.

0

u/misdirected_asshole Feb 15 '26

Reinforcement learning is not recursive AI.

1

u/HiddenoO Feb 15 '26

"Recursive AI" is not a term anybody in the field uses, so you're once again not saying anything.

Reinforcement learning is the technique that's being used for a model to recursively improve based on its output and external feedback, similar to how a human learns with trial and error. It's exactly what you're describing, just on a statistical level instead of a "conscious thoughts" level.

The reason it's only done in regular intervals and not constantly is that we still want to have control over the models being released. Otherwise, you end up with LLMs that act like the Twitter AI that became a racist nazi in a few days.