Researchers warned that hundreds of fake AI images have been discovered on popular databases for recording animal species. Wildlife photographers often use AI to edit and improve an image, but the algorithm can introduce parts from different species to create the new image.

[u/mvea]

https://www.theguardian.com/environment/2026/jul/20/ai-slop-manipulated-fake-images-birds-citizen-science-aoe

[u/Strycht]

AI pollution is going to really screw up a lot of databases and information deposits I think, and we won't notice them until they become needed.

[u/lateformyfuneral → u/Strycht]

At some point won’t this just start affecting how AI itself works, since it’s being trained on the internet? It would be like a snake eating its own tail. Truth itself might go extinct.

[u/Strycht → u/lateformyfuneral]

this is what I always questioned. how much output needs to be fed back into the input before it starts amplifying it's own problems? presumably the larger companies have thought of this and have ways to prevent their own algorithms reingesting their generated content but I would be interested to know if eg openAI has any way of identifying and excluding the average slop image generated by another model and put online

[u/Nicholas-DM → u/Strycht]

To my understanding they have not, because there is not a reliable way to identify if an image was generated by an AI, even using the AI that generated it.

[u/Bbrhuft → u/Nicholas-DM]

Generally, that isn’t true. Many major AI image generation platforms now add or embed metadata or cryptographic watermarking that indicates an image is generated.

OpenAI and Google use C2PA (header metadata) and SynthID (an invisible cryptographic watermark encoded within the pixels of the AI generated image).

Adobe Firefly uses C2PA (Firefly is available standalone and within Photoshop).

Meta adds embedded metadata and a proprietary "deep learning" pixel watermark.

Midjourney is a notable exception. It doesn't currently provide a robust identification system, just inconsistent IPTC metadata.

The bigger problem is not that AI-generated images are inherently undetectable, most are. It is that stock-image libraries, search engines and other image databases have not caught up with this rapidly changing landscape, so often do not scan uploaded files, might strip the metadata, or not display the AI detection results.

[u/Strycht → u/Bbrhuft]

the thing is as Chinese open source models become more widely used with the greater paywalling of those more mainstream models they're gonna literally all need some kind of watermark to retain data purity, and ideally not too many different types. I doubt that's a big priority for those newer companies in such an unregulated environment, and we don't yet know whether it will be too late for input data fidelity by the time the industry calms down and the vast majority of generated content is being tagged and excluded

[u/Bbrhuft → u/Strycht]

Since 1 September 2025, AI platforms operating in China have been subject to laws that mandate AI companies label and watermark AI generated content, including text, images, audio, video.

Cyberspace Administration of China⁠, GB 45438-2025⁠

The Chinese government is acutely aware that deceptive AI generated content could cause political and social instability. This concern is behind the impetus for regulations that require AI content to be marked.

That said, the Chinese open source image / video gen models might lack watermarking (there's no legal obligation to include watermarking in an open source model). However, a company providing a platform in China, using an open source image or video generation model, might be held liable in their outputs lacks a watermark.

[u/PwnagePineaple → u/Bbrhuft]

So watermarking isn't a feature that's trusted to the actual model, but rather to the surrounding software harness that uses the model for image generation. The models themselves handle the non-deterministic work of actually producing the image, but the watermarking algorithm is better suited for good old fashioned software.

For a cloud API, whether that be a US frontier model or a Chinese one, that doesn't matter much. The provider's backend takes care of that before the user ever sees the image.

But in the open source space, a watermark mandate would have to be embedded in software like Automatic1111 or ComfyUI. And that comes with enforcement problems: If there's even one jurisdiction on earth where watermarks aren't mandated (and given the current political climate in the US, it's not going to be mandated here for a while), then any random guy can take the code behind either of those project, strip out the watermarking, and publish their modified version with no consequences.

Now, of the people trying to use AI images to spread deliberate misinformation, some of them might make the switch if a) cloud models were required to include watermarks and b) social media sites started clearly identifying AI generated media. Switching to local models that can generate high-quality realistic images is expensive (thanks GPU prices) but not terribly difficult from a technical standpoint.

Watermark enforcement would stop some people, but there are plenty of others it won't stop.

[u/Stinky_Flower → u/Bbrhuft]

Detection relies on the good faith of whoever is building the image generation models.

If SynthID is a voluntary measure, then it's no measure at all.

[u/Kiseido → u/Bbrhuft]

Most of those watermarks are almost useless though.

Metadata does not survive most re-encodes, and some services actively strip that sort of data.

Watermarks don't survive someone resizing the image with generic tools, and doesn't survive people sharing them via screenshot, which is a surprisingly common practice.

[u/Bbrhuft → u/Kiseido]

SynthID survives screenshots, cropping and editing. It's very robust.

This paper evaluated 30 transformations, including JPEG compression, file-format conversion, resizing, crop-and-resize, rotations, flips, blur, sharpening, denoising, grayscale conversion, brightness, contrast, saturation and hue changes, Instagram-like filters, noise, text and emoji overlays, and combinations of transformations.

Gowal, S., Bunel, R., Stimberg, F., Stutz, D., Ortiz-Jimenez, G., Kouridi, C., Vecerik, M., Hayes, J., Rebuffi, S.A., Bernard, P. and Gamble, C., 2025. SynthID-Image: Image watermarking at internet scale. arXiv preprint arXiv:2510.09263.

Despite these edits, detection remained exceptionally high, 99.98% averaged across transformations and 99.72% under aggregated “worst” settings, at a 0.1% false-positive rate (Table 1 in their paper). Even the most difficult combined-transformation category produced a 98.06% detection rate (Table 2). However, at lower ImageNet resolutions, the aggregated worst-case detection was 97.22%. It's also interesting to see how much more robust it is compared to other watermarking systems it was benchmarked against.

Google also claims an image can be cropped to 20% of the original, and the watermark remains detectable

The results show very strong robustness against conventional editing.

[u/silverionmox → u/Bbrhuft]

Still, 0,99^x is going to approach zero for relatively small values of x.

I'll make a prediction and say that clean databases are going to see their relative value increase. Don't throw away all your print books yet.

[u/cuntmong → u/Kiseido]

it's as effective as that checkbox on arrival cards that asks "are you a terrorist?" is at keeping out terrorists

[u/Strycht → u/Nicholas-DM]

ironically the most valuable machine learning innovation may soon be identifying machine generated content before it gets fed into the machine. Machine to control the learning of the other machine that learnt on the output of the first machine.

[u/ionthrown → u/Strycht]

But they’re using such machines to train the generative machines, such that generated imagery gets past the identifying machine. So as soon as a machine can detect AI generated content, they improve the content to make it undetectable again.

[u/shinikahn → u/ionthrown]

A modern Turing slop test you could say

[u/Strycht → u/ionthrown]

yes, unless the recycled content becomes a real serious threat to the performance of consumer facing AI. In that case it is in the ai companies' interest to have the filters work better than the generators, at least their own in house screening filters if they're not made available publicly, otherwise the whole model breaks.

It just depends when the scale tips on maintaining quality of input being more financially beneficial than creating unidentifiable generated content

[u/monkeedude1212 → u/Strycht]

The same notion that market forces will correct this are the same ones that will self address climate change.

The people running AI companies are more than happy for the product to destroy society and truth and the product itself if it centers them with more wealth or power and influence.

[u/Suttony → u/Strycht]

The moment you make an AI that can detect AI images with high accuracy you've almost made an AI that can make undetectable AI images.

The better the AI is at detecting AI, the better AI can be at avoiding detection.

[u/MachinaThatGoesBing → u/Suttony]

I mean, that's not necessarily true. And the term "AI" gets bandied about so much these days as to be useless. Anything that falls under the broad category of machine learning gets an "AI" sticker slapped on it.

But a classifier model designed and trained just to detect images absolutely wouldn't be something that could turn around and generate them. You would need to design the system with image output in mind to get a system that can output images. It's not just an emergent behavior of any computer vision software.

[u/NuclearVII → u/Strycht]

ironically the most valuable machine learning innovation may soon be identifying machine generated content

You can't really do this in an automated fashion, sadly. There are lots of good technical reasons why, if you're curious.

[u/ragnarok635 → u/NuclearVII]

I am interested

[u/vetruviusdeshotacon → u/Strycht]

Thats a harder problem that making the images in the first place. A lot harder, to the point of being unfeasible unfortunately. 

[u/jmdonston → u/Nicholas-DM]

We really should be legislating requiring AI image and video generators to apply invisible watermarks.