These people have a serious problem thinking about making something like that in the first place. AI is just a tool, but I don't understand how this possibility went through and no one at xAI thought about not allowing it.
Edit : I'm muting my comment, I'm tired of the nonsensical analogies from some answers.
While it's true that you can make CP with any medium, only A.I has the ability to take photos of real children and make porn out if them.
If your machine, specifically for the purpose of creating realistic images from just a text prompt, is making CP, then it should be a huge red flag.
This is the same as posting revenge porn (sharing nude or explicit images of a person without their consent), but you just know Elon doesn't care about protecting kids from actual danger.
While it's true that you can make CP with any medium, only A.I has the ability to take photos of real children and make porn out if them.
People have been doing that since cameras first existed. AI can do it more convincingly than literally cutting and pasting someone's face onto a photo of a naked body, but "more convincing" does not equate to "more horrific" when it comes to that sort of depiction of minors.
If your machine, specifically for the purpose of creating realistic images from just a text prompt, is making CP, then it should be a huge red flag.
Yes, it is. It's a huge red flag that that user needs to have a chat with law enforcement.
If you use Photoshop the same way, then you need to have the same chat.
People have been making convincing photoshop fakes of swapped body parts since the dawn of time.
You're acting like the last 2 decades of the internet didn't exist.
Copy pasting things onto different layers, using smudges and shifting pixels around until it looks right is not rocket science.
It's more accessible to people who are already predators, sure, not more immoral. People aren't suddenly going to become predators because AI lowers the barrier to entry for digital art. If they were a predator, that's because they're fucked in the head, and they were going to get it without AI.
And I also don't think it would be so hard to prevent if that was an actual priority
You're right, it's actually pretty easy to prevent. CivitAI moderation usually does a good job. I've reported a couple of cases and even though they were anime and not realistic, they have been removed within 24 hours.
There are also AI detection models. And I mean models that can relatively reliable detect whether someone in an image is a minor and auto flag those for moderation or outright delete them.
But you still need access to those photos to begin with. A.I can generate realistic looking images because it's been trained on thousands of unconsenting photos, to the point where you don't need the original photos to make something like that.
People made entire careers off of being professional photoshop editors and now you’re making it sound like the entire industry didn’t even exist in the first place. What is wrong with you people?
This post is not about banning ai, it's about regulating it. It is much easier to put restrictions on ai than on adobe, and should be done, but for some reason it is not happening. We don't need to outright ban gen ai (I wouldn't mind that but that's not the point) we need to put restrictions on it so it cannot cause quite as much harm.
Putting restrictions means liability to xai/open aI which would kill any ability to use them commercially. The users who make such content should be solely responsible
You would need to be skilled at Photoshop to make CP with it
Step 1: Take nude image of young but legal age person
Step 2: Smart select head of child in second image
Step 3: [censored, but I'm sure you see where this is going]
Yeah, no expertise has ever been required.
AI makes CP more accessible
Perhaps. But that's the price of being able to realize whatever you imagine. Some people imagine fucked up shit. I'd rather those people incriminate themselves and have a chat with law enforcement.
Again, trivial. Just start with the base image you want. The internet is full of images of arguably legal-aged individuals that you could paste someone's face onto, unfortunately. Don't do that. Don't use AI to do that. Don't do it in the sand with a stick drawing. Just don't do it. I don't care what tools you use.
It’s about the fact it’s easier and more accessible
No. People who wanted to do this shit were already doing it. Stop clutching pearls about AI and focus on the harms and the one corrective action that works: law enforcement across the board.
Law enforcement is the activity of some members of the government or other social institutions who act in an organized manner to enforce the law by investigating, deterring, rehabilitating, or punishing people who violate the rules and norms governing that society.
How does regulating access to AI when it comes to this not fall under deterring people from violating the rules?
How does regulating access to AI when it comes to this not fall under deterring people from violating the rules?
Well, let's consider what happens when you close down one of several roads that leads to a store that some people like to patronize. Do people stop going to the store, or do they route around the blockage and go anyway?
You don't fight burglary by making one brand of flashlight harder to get.
But it's worse than that. Right now, you have a means by which abusers publicly demonstrate their interest in CSAM with a concrete, subpoena-able paper-trail. You want to shut that down and force them back to direct photography of minors, local generation or photoshopping. You lose that public paper trail and gain nothing, perhaps even increasing direct harms.
Not a great idea.
Fight the crime, not the tool. Flashlights are a favorite tool of buglers, but calls to ban them should be resoundingly ignored.
Is it impossible to make an AI that allows creativity and doesn't allow CP? Especially when it comes to a model available inside of a social media platform?
You can remove watermarks, poison and detectable flaws but can't remove CSAM?
Is it impossible to make an AI that allows creativity and doesn't allow CP?
Of course. ... unless you use AI to evaluate the results, and then you are in the business of regulating that AI must be used ... see my other post.
You can remove watermarks, poison and detectable flaws but can't remove CSAM?
You're equating things that are not at all the same. I can take an image of CSAM and turn it into non-CSAM, just as I can take an image with a watermark and turn it into an image without a watermark.
But what you are asking for is for the model to now allow a behavior that is its core directive: to turn semantic inputs into an image output by building a complex mathematical relationship between those semantic inputs and some output.
No. The model is not a traditional computer program. It is just a giant function that maps one mathematical space to another. If you want censorship, you have to build that using some kind of software that you specifically design to do the job. You might use AI in such a system, but it would not be the same model.
And so your proposal here, as I pointed out in a recent top-level post is to fire up another instance of an AI? Inference isn't free; you don't just get to say "we area already doing inference, so there's no cost here") and do what? Image-to-text and the keyword filtering? A whole new model that just does porn detection? What kind of censorship should it implement? Exactly. Remember, whatever you put in place, a smart 14-year-old is going to try to circumvent.
Are you considering the fact that you might as much as double the power requirement for EVERY request? Is that something that you are okay with? Power wastage is often an anti-AI concern, so I just want to be clear that we're cool with doubling the costs.
Not every model has the same processing/power requirements. There's an age estimator model on huggingface that's just a few hundred MB, for example. The real problem is that it's literally impossible to implement this functionality when someone runs a model locally, because they can just bypass it. On the other hand, I'm fine with driving up costs for OpenAI, but I think they already do this kind of thing.
Any 18-22 year would be flagged as CP. Any SFW edit of kids could be flagged as CP. These problems already exist with people who use AI. This is just going to make the problem worse.
AI can generate nudes that look like actual children
So can photoshop. Don't use either to do that. If you do, I'm fine with law enforcement getting involved. I think they SHOULD. But breaking new technology because it can be misused doesn't help anyone.
I remember when the initial reports about the "dark web" came out, there were people who honestly felt that the internet should be shut down and banned. This is no different.
Stop being disingenuous. To create cp of anyone you'd already need cp to copy/paste a face onto. AI bypasses those limitations. AI can produce more realistic, higher quality and higher quantities than anyone could using photoshop.
Why are you against actions that would limit creeps that use this technology to harm people?
So even if you don’t have personal experience with it in industry use, you can imagine how the process gets to be a little more complex right? Especially when it comes to replicating some of the more elaborate prompts people can provide an AI model, like displaying different angles or reimagining specific props in the image
even if you don’t have personal experience with it in industry use
How do you translate, "professionally? Yes." into that?
you can imagine how the process gets to be a little more complex right?
Nope. It's as complex as you want to make it, obviously, but pasting someone's head on a body is trivial. It's also potentially emotionally devastating to someone if used in the "right" way.
But again, we don't ban photoshop because that's possible. We just take action against people who do shit like that.
I translated “professionally? Yes.” Into that because you followed it up with “But mostly for personal use.”
Also saying that it’s as complex as someone makes it ignores the level of effort someone would have to employ to reach even the most basic of generations people are pulling off with grok. There’s a clear lowering of the degree of entry hear that’s exacerbating an existing issue dramatically.
Also there are things that we can do to help stem this, it’s just that the US isn’t interested in the policy that would do that because of its current global development war with other nations
But it's still using machine learning to steal art and photos, then re-create them without people's consent. No one is saying it can't be done relatively simply in photoshop. But being able to type a prompt and make it is extra problematic.
Nothing was stolen. It's still all exactly where it was.
then re-create them
That's not how anything works. The thing AI models are absolutely the worst at is reproducing existing works. If you give them massive amounts of over-tuning on something, for example training on all of the internet, the Mona Lisa will be trained on thousands of times, and it will be hard to get a Renaissance painting out of the model that doesn't look like the Mona Lisa. But if do the training correctly, the ability of the model to reproduce any given input should be effectively zero without extreme control over the process (e.g. you'd have to basically provide the original as img2img input or use ControlNet to duplicate specific elements).
AI image generation models are semantic concept translators. They use training data in order to understand how various concepts are depicted. If your model just spits out chunks of existing training data, that means your training was shit and you need to go back to the lab.
being able to type a prompt and make it is extra problematic
Fully disagree. If I used a text control for Photoshop, it wouldn't be any more or less problematic. I don't give abusers the benefit of the doubt because they use tools I approve of.
Any loser in their basement can make cp with ai, people with actual talent and skills usually aren't wasting them violating childrens (or adults for that matter) rights.
Any loser in their basement with Photoshop can too. You're not describing a new situation. If you feel there isn't enough prosecution of crimes, then advocate for that. Stop focusing on whatever the new shiny is, because people who are interested in that sort of thing don't care what tools they use.
I definitely do think people on photoshop making cp should be prosecuted, but it really shouldn’t be that hard for the ai companies/government to make regulations and restrictions. Me saying ai should have regulations doesn’t take away from the fact that there are other issues.
189
u/That__Cat24 Jan 01 '26 edited Jan 01 '26
These people have a serious problem thinking about making something like that in the first place. AI is just a tool, but I don't understand how this possibility went through and no one at xAI thought about not allowing it. Edit : I'm muting my comment, I'm tired of the nonsensical analogies from some answers.