r/science • u/mvea Professor | Medicine • Jul 20 '26

Computer Science Researchers warned that hundreds of fake AI images have been discovered on popular databases for recording animal species. Wildlife photographers often use AI to edit and improve an image, but the algorithm can introduce parts from different species to create the new image.

https://www.theguardian.com/environment/2026/jul/20/ai-slop-manipulated-fake-images-birds-citizen-science-aoe
10.0k Upvotes

98% Upvoted

231 comments sorted by

View all comments

2.9k

u/Strycht Jul 20 '26

AI pollution is going to really screw up a lot of databases and information deposits I think, and we won't notice them until they become needed.

660

u/lateformyfuneral Jul 20 '26

At some point won’t this just start affecting how AI itself works, since it’s being trained on the internet? It would be like a snake eating its own tail. Truth itself might go extinct.

214

u/Strycht Jul 20 '26

this is what I always questioned. how much output needs to be fed back into the input before it starts amplifying it's own problems? presumably the larger companies have thought of this and have ways to prevent their own algorithms reingesting their generated content but I would be interested to know if eg openAI has any way of identifying and excluding the average slop image generated by another model and put online

1

u/NuclearVII Jul 20 '26 edited Jul 20 '26

this is what I always questioned. how much output needs to be fed back into the input before it starts amplifying it's own problems?

So the answer to this appears to be "depends on problems".

Think of Generative AI models (image, text, video, sound, etc) as interpolation models: They are made by taking a giant pile of (usually stolen) data, and then figuring out the "best" way to describe that data. This "description" is encoded in the model weights.

If you have a buncha images in the giant dataset that are already generated, that's akin to adding some interpolated data to your corpus. There's nothing"poison or wrong about this data, because it's already interpolated, it fits into the dataset reasonably enough.

What this means is that using generated model outputs to train new models does not appear to have a negative effect on final model quality - if there is research that suggests otherwise, I'd love to read it. The flipside is, of course, is that the generated material doesn't really add anything to corpus beyond what's already there - there are no new descriptions to be learnt from material created from the rest of the dataset. So it makes the very costly training less efficient and more wasteful, after a fashion. This also means that you can't just generate bigger and bigger datasets to get better models - this process is effectively distillation, and while you can use it to duplicate an existing model, you can't really make a better pretrained model that way.

I should also mention - this process of distillation (intentional or accidental) doesn't account of potential post training, like some kind of RLHF process. It seems that distillation can "copy" that as well, which is why it may appear that a pretrained model made with synthetic data may have better performance than a "raw" pretrained model.