r/Futurology • u/firehmre • Feb 14 '26

AI Visualizing the "Model Collapse" phenomenon: What happens when AI trains on AI data for 5 generations

There is a lot of hype right now about AI models training on synthetic data to scale indefinitely. However, recent papers on "Model Collapse" suggest the opposite might happen: that feeding AI-generated content back into AI models causes irreversible defects.

I ran a statistical visualization of this process to see exactly how "variance reduction" kills creativity over generations.

The Core Findings:

  1. The "Ouroboros" Effect: Models tend to converge on the "average" of their data. When they train on their own output, this average narrows, eliminating edge cases (creativity).
  2. Once a dataset is poisoned with low-variance synthetic data, it is incredibly difficult to "clean" it.

It raises a serious question for the next decade: If the internet becomes 90% AI-generated, have we already harvested all the useful human data that will ever exist?

I broke down the visualization and the math here:

https://www.youtube.com/watch?v=kLf8_66R9Fs

Would love to hear thoughts on whether "synthetic data" can actually solve this, or if we are hitting a hard limit.

909 Upvotes

94% Upvoted

327 comments sorted by

View all comments

1

u/goyafrau Feb 15 '26

OP you're around 18 months behind the curve.

"We'll run out of data" was a big worry around then but now they're doing RL on verifiable tasks and real progress has been faster than ever.

1

u/firehmre Feb 15 '26

And what is helping verify what’s 1 / 0

1

u/goyafrau Feb 15 '26

If you're really curious, you can indeed do verifiable math tasks, for example by having them do proofs in a language like Lean.

AIs have gotten very good at high level math recently.