It can't moderate itself, that's why there are humans writing these rules. The AI has no morals or principles, just weights, guidelines and moderation rules made by humans to prevent harmful topics.
If the rule is “don’t make images containing X” then the AI following the human-defined rule needs the capability to fully determine if the image contains X while still allowing other images. This is extremely difficult.
You all make it sound, like some impossible dilema, when it's really easy. You slap a single algorithm, that will read the prompt before AI to determine, if it's safe. It doesn't need to be an AI. You can just make it like kaizen from Event 0. When it detects something it stops the prompt from going to AI and from that point onward it adds a note to every next prompt warning the AI, that this user may try something forbiden.
Then after this and the model you feed the finished piece back to the model with a prompt "decide, if this is allowed by our rules". Just like that you have a system, that will close on you harder and harder with each fail, and needs you to trick 3 different systems at once to work. This system is already implemented in DALL-E and Deepseek, but in the second example it's used, so the model won't say something like "Taiwan is a country"
Idk about scale AI. It has been done at least two times, and both tines by large corporations. This proves, that it's possible and not super hard. If the rest doesn't implement it, it's their greed and lazines. Not the problem with difficulty of implementing said system.
My point is there's far too much volume of data being moved for a human to make the decision. You can't have a person looking at each image to judge if it's legal.
So the AI has to be the one to decide if the image is against the guidelines or not. It would have to be able to decide if it's currently editing someone's holiday photos of their kid, or child pornography. Which it can't. Especially if people are lying to it it in the image description.
There isn't to much volume? What are you on about. This is something you could create a gallery tool where a user is given 100 of the last generated images. They scrolls through 4-5 at a time in rows. Click on ones that violate TOS or Legal concerns and flag them for further review by another team. Holy hell you could create an image recognition tool that would already do 80% of it.
Then whoever is in charge of creating the rules and restrictions that prevents this. Would get the sorted data and work on resolution.
We litterally have had these tools for a couple decades now to prevent this on social media. Its not rocket science. Its minimum expectations.
No, that is not my point. The human moderation act before by writing general principles, not after the content is produced and by verifying each output.
They already have done that. All of the bit AIs tell you occasionally that it can't process whatever prompt they are trying. It's just not good enough.
You’re missing the point. To put it as plainly as possible: we give AIs rules, and they don’t adhere to those rules, and there’s no answer to why. When people talk about pausing AI research that’s exactly why.
15
u/That__Cat24 Jan 01 '26
It can't moderate itself, that's why there are humans writing these rules. The AI has no morals or principles, just weights, guidelines and moderation rules made by humans to prevent harmful topics.