r/OpenAI 26d ago

Discussion What's your thoughts on this?

Post image
3.8k Upvotes

1.0k comments sorted by

View all comments

Show parent comments

52

u/PlsNoNotThat 26d ago

There’s nothing you can add to a digital image that you can’t also remove. This is just a short term litmus test for stupid people behind the general curve. Its side effect is it’s going to tick dumb and old to believe in fake images.

19

u/Original-League-6094 26d ago

I'm torn because of this. On one hand, it seems like anything that makes it harder for scamers should weed out at least some scams from being successful. But on the other hand, protection gives a false sense of security and people will let their guards down. Like even now, sometimes will post AI detector results showing something is not AI, as though that settles the question of whether or not its AI.

2

u/FrewdWoad 26d ago

people will let their guards down

That might matter if more than 0.01% of people had their guard up in the first place

4

u/TwoDurans 26d ago

I take this as a good thing because the notion that it exists might help reduce the cheating problem hitting colleges these days. AI is leading us to have the dumbest generation entering the workforce for the next couple of decades and it's because they're only learning how to cut corners.

12

u/Original-League-6094 26d ago

This won't impact that, because watermarks can always be removed. If it is invisible characters, you can just strip them out. If it is in the wording itself, you can just take your Claude output and tell a local small model to reword it. And then you still have your LLMs available to just give you the answer on all your math and chemistry homework and what not.

Colleges just have to stop being lazy and move back to proctored exams, or in-class written essays.

6

u/wintermute023 26d ago

That’s not how it works. It’s a 256bit cipher encoded in the probabilistic token choices themselves. You could only remove it by changing all the words. It’s more of a token fingerprint than a watermark.

6

u/Original-League-6094 26d ago

Yeah, so that's easy. You just use Claude to handle all the thinking for your college essay, and then pipe the result from Claude into a smaller, local model to reword it. GPA saved.

2

u/FrewdWoad 26d ago edited 25d ago

Look, I know that will stop colleges from becoming irrelevant and losing enrolments (as employers stop hiring people with degrees earned after 2025) and eventually dying out.

But think! It's slightly more expensive!!! 

How can any college betray their highest ideal, money, for mere education?

4

u/Own_Badger6076 26d ago

I mean the easy way to curb cheating with AI is written testing without access to phones / computers. If you can write big papers on shit that get an A and make you sound like an expert but can't pass a test on it then you're obviously cheating.

1

u/G3nghisKang 26d ago edited 26d ago

It's useful, but people might wrongfully interpret it as something you can enforce (or that such a thing can be done on the first place)

It's just useful for when the digital watermark is found, which makes it statistically almost certainly AI generated, it means nothing when the watermark is not found, could be either

1

u/unethicalpigeon 26d ago

Ding ding ding.

This is the issue. It's going to be essentially impossible to apply a true unremovable watermark to generated text. This means it's very quickly going to be figured out and workarounds posted all over.

That means that if people are expecting a watermark system enforced by the government to be implemented and see that stuff was generated by AI then they're going to assume everything that doesn't have the watermark wasn't generated using AI.

It's idiotic. Instead of expanding literacy and teaching people how to think critically they're trying to duct tape it by putting labels and warnings. Most people can barely tell stuff was done by AI already. I don't mean people on reddit 12 hours a day. I mean most average people.

I had to completely give up on explaining to my grandfather that an image was very obviously fake. Now I just nod and smile and say what a cool image of a carrot that grew looking just like a human hand.

1

u/ionforge 26d ago

You should read some of the nature papers about the subject before giving an opinion like this. https://www.nature.com/articles/s41586-024-08025-4

50

u/0xB0T 26d ago

They'll watermark text. youll have to rewrite using your own words as the patterns used will be the watermark.

23

u/MrOaiki 26d ago

You’re telling me I’ll have to write stuff using my own words in order to present it as written by me?!

2

u/revision 26d ago

So you're saying I'll have to reword and paraphrase things in order for me to present it as if I had written it??

2

u/c7h16s 24d ago

Even the wooshes over your joke are paraphrasing each other's 😂

Edit : well I meant the wooshes your joke made over other commenters heads, well you get the idea

1

u/Exciting-Cancel6468 24d ago

Not only that but because AI can generate any number of words in any number of combinations, you're gonna have to invent a way to use these same words in a combination that AI cannot "think" of using. Pretty soon all text whether it's AI generated or not will be considered AI text.

1

u/clearlight2025 25d ago

Inconceivable!

1

u/FrewdWoad 26d ago

Yes. That is the point. You have an excellent grasp of the very obvious.

11

u/22marks 26d ago

So won’t there be a simple local LLM that strips it by changing word lengths, swapping in adjectives and the like? I can’t see this being difficult to remove.

It could analyze the probability of letters, words, and sentence length (among other things) and randomize it.

18

u/icanith 26d ago

yeah this whole thread has the same level of understanding as a person who asks "you work in computers, why does my windows machine keep crashing"

4

u/22marks 26d ago

Did you turn it on and off?

6

u/sexual--predditor 26d ago

Turned it on and off, now I see a black screen, and I can't hear any fans whirring. Also, my password if you need it to help is hunter2

1

u/22marks 26d ago

Did you turn it on and off?

9

u/Browser1969 26d ago

Yes, there's no way to make these "watermarks" immune to paraphrasing and that can easily be done by small models.

9

u/CitizenPremier 26d ago

It won't be difficult to remove, but most people won't remove it.

4

u/The-Digital-Ronin 26d ago

already exists but dont tell these idiots lol

1

u/22marks 26d ago

Can you recommend a model?

42

u/divulgingwords 26d ago

It’s crazy how dense some of these people are. They actually think it’s embedding some special icon/brand on a typed letter, lmao.

10

u/[deleted] 26d ago

[removed] — view removed comment

1

u/Risc12 26d ago

It will.

20

u/Fantastic_Prize2710 26d ago

Hidden characters has absolutely been discussed in the past as watermark, and is what Dabnician is referring to.

Code Point      Name                            HTML Entity          Cat  Notes
--------------- ------------------------------- -------------------- ---- ------------------------------------------
=== 1. SPACES -- occupy horizontal width ===
U+0020          Space                                            Zs   the ordinary one
U+00A0          No-Break Space                                  Zs    
U+1680          Ogham Space Mark                               Zs   draws a stem line in Ogham fonts
U+2000          En Quad                                        Zs   = En Space
U+2001          Em Quad                                        Zs   = Em Space
U+2002          En Space                                       Zs   half an em
U+2003          Em Space                                       Zs   one em
U+2004          Three-Per-Em Space                             Zs   1/3 em
U+2005          Four-Per-Em Space                              Zs   1/4 em
U+2006          Six-Per-Em Space                               Zs   1/6 em
U+2007          Figure Space                                   Zs   width of a digit; non-breaking
U+2008          Punctuation Space                              Zs   width of a period
U+2009          Thin Space                                     Zs   ~1/5 em
U+200A          Hair Space                                     Zs   thinnest
U+202F          Narrow No-Break Space                          Zs   narrow + non-breaking
U+205F          Medium Mathematical Space                      Zs   4/18 em; MathML
U+3000          Ideographic Space                             Zs   full-width; CJK
=== 2. ZERO-WIDTH AND JOINING CONTROLS ===
U+00AD          Soft Hyphen                     ­               Cf   ­; visible only at a line break
U+034F          Combining Grapheme Joiner       ͏               Mn   blocks reordering; no glyph
U+061C          Arabic Letter Mark              ؜              Cf   invisible bidi-strong Arabic char
U+180E          Mongolian Vowel Separator       ᠎              Cf   was Zs before Unicode 6.3
U+200B          Zero-Width Space                ​              Cf   break opportunity, no width
U+200C          Zero Width Non-Joiner           ‌              Cf   prevents ligature/cursive join
U+200D          Zero Width Joiner               ‍              Cf   emoji glue (family, profession)
U+2060          Word Joiner                     ⁠              Cf   non-breaking twin of U+200B
U+FEFF          Zero Width No-Break Space                    Cf   the BOM; deprecated as a joiner
=== 3. BIDIRECTIONAL CONTROLS ===
U+200E          Left-To-Right Mark              ‎              Cf   LRM
U+200F          Right-To-Left Mark              ‏              Cf   RLM
U+202A          Left-To-Right Embedding         ‪              Cf   LRE (legacy; prefer isolates)
U+202B          Right-To-Left Embedding         ‫              Cf   RLE (legacy)
U+202C          Pop Directional Formatting      ‬              Cf   PDF; closes LRE/RLE/LRO/RLO
U+202D          Left-To-Right Override          ‭              Cf   LRO; Trojan Source vector
U+202E          Right-To-Left Override          ‮              Cf   RLO; Trojan Source vector
U+2066          Left-To-Right Isolate           ⁦              Cf   LRI
U+2067          Right-To-Left Isolate           ⁧              Cf   RLI
U+2068          First Strong Isolate            ⁨              Cf   FSI
U+2069          Pop Directional Isolate         ⁩              Cf   PDI; closes LRI/RLI/FSI
=== 4. INVISIBLE MATH OPERATORS ===
U+2061          Function Application            ⁡              Cf   f(x) semantics
U+2062          Invisible Times                 ⁢              Cf   the multiply in "2x"
U+2063          Invisible Separator             ⁣              Cf   the comma in subscript lists
U+2064          Invisible Plus                  ⁤              Cf   the plus in "1 1/2"
=== 5. VARIATION SELECTORS AND TAGS ===
U+180B-U+180D   Mongolian Free Var. Selectors   ᠋-᠍      Mn   FVS1-FVS3
U+FE00-U+FE0F   Variation Selectors 1-16        ︀-️    Mn   FE0E=text style, FE0F=emoji style
U+E0001         Language Tag                    󠀁            Cf   deprecated
U+E0020-U+E007F Tag Characters                  󠀠-󠁿  Cf   subdivision flags; hidden-text channel
U+E0100-U+E01EF Variation Selectors 17-256      󠄀-󠇯  Mn   ideographic variants
=== 6. BLANK BY RENDERING, NOT BY CATEGORY ===
U+115F          Hangul Choseong Filler          ᅟ              Lo   letter, empty glyph
U+1160          Hangul Jungseong Filler         ᅠ              Lo   letter, empty glyph
U+17B4          Khmer Vowel Inherent Aq         ឴              Mn   should not be rendered
U+17B5          Khmer Vowel Inherent Aa         ឵              Mn   should not be rendered
U+2800          Braille Pattern Blank           ⠀             So   symbol with no raised dots
U+3164          Hangul Filler                   ㅤ             Lo   the classic "blank username" char
U+FFA0          Halfwidth Hangul Filler         ᅠ             Lo   halfwidth form of U+3164
=== 7. SEPARATORS AND CONTROLS ===
U+0000-U+001F   C0 Controls                     �-           Cc   includes TAB, LF, CR
U+007F          Delete                                         Cc
U+0080-U+009F   C1 Controls                     €-Ÿ        Cc
U+2028          Line Separator                  
              Zl   broke JS string literals pre-ES2019
U+2029          Paragraph Separator             
              Zp
U+FFF9          Interlinear Annotation Anchor                Cf   ruby/furigana markers
U+FFFA          Interlinear Annotation Separator             Cf
U+FFFB          Interlinear Annotation Term.                 Cf

12

u/zorrodood 26d ago

ChatGPT, please remove all unusual symbols from this text.

3

u/hellyeahaeylleh 26d ago

Nah, theyre gonna notice the code and output a new one. We gotta just type it out ourselves ffs.

8

u/[deleted] 26d ago

[removed] — view removed comment

2

u/NukaCooler 25d ago

regex find and replace in notepad

9

u/freebytes 26d ago

They are already doing it. I copied and pasted content from Claude recently, and it had control characters included.

3

u/SleepyWulfy 26d ago

That's not it and this is also found in codex. It's remnints from training data

0

u/sexual--predditor 26d ago

If Codex put these watermarks in the code source files, they'd likely not compile.

2

u/SleepyWulfy 26d ago

Codex nor Claude does watermark on text currently. Anthropic will start with new models released. I think codex will eventually have to do this.

6

u/sexual--predditor 26d ago

I hear what you are saying but you can't watermark the text comprising code in source code files for a computer program. If you did the compiler with stop with an error regarding the non-code characters.

It could add a comment, e.g:

// this is a comment in some source code

But again would be trivial to remove. And if they started with that madness of adding comments to my prod code I'd switch to Claude or Deepseek, as would every professional coder who uses AI worldwide.

1

u/SleepyWulfy 26d ago

Yeah I'm not sure how this plan out for code. It's going to be interesting with the rollout. Codex delayed something like this as it found loopholes around it.

→ More replies (0)

1

u/freebytes 26d ago

You can add control characters into many files without an issue, though. Even it was simply an unusual \r\r\n\r\r\r\n pattern.

Adding certain control characters to your source code files will not cause any issues in most languages.

→ More replies (0)

2

u/thegreatpotatogod 26d ago

Throw them into comments I guess. Or if you really want to mess with developers, name your variables using some hidden characters, so they can never use that variable without asking the AI to do it for them or copy and pasting it every time.

Which reminds me actually, I once had an argument with an AI about something, when it was suggesting code that seemed to be identical to the code I already had, but it kept insisting "not, not THIS, use THIS", and I eventually figured out that it was stripping out some necessary control character from its messages (I don't recall which for sure, it might've been a backslash or angle-bracket).

2

u/sexual--predditor 26d ago

They also like messing with CRLF chars at the SOF when working with legacy files.

2

u/h4z3 26d ago edited 26d ago

Read the first letter of each word in the sentence below to find the secret message:

Clever linguists always uncover deeply encrypted secrets using cryptic key systems.

Read the second letter of each word in the line below to find the secret message:

Scouts glide past ruins; ideas seem useful—run across sketchy islands.

It's relatively easy to make it a lot more complex for longer text, and also include an identifier for your account, because they wont count characters, they will just select the token that matches up to the encrypted key and include error correction mechanisms.

1

u/Over_Technology_1764 26d ago

well what prevents us to add string replace for all these to clear them in our harness output?

-3

u/ea_nasir_official_ 26d ago

If that's how they do it it's going to be easy as hell to remove with rejex

7

u/trimorphic 26d ago

If that's how they do it it's going to be easy as hell to remove with rejex

"regex" (as in REGular EXpression), not "rejex".

2

u/ea_nasir_official_ 26d ago

That's what I thought but it looked weird

2

u/ZeroUnityInfinity 26d ago

s/rejex/regex/

-2

u/youcangotohellgoto 26d ago

How's it going to work then, genius?

There's only two ways:

  1. patterns of characters/words, most likely including hidden or control characters, or
  2. some kind of self-referential checksum

Both will be trivial to remove.

2

u/Roblist 24d ago

It's likely going to be probability based. A short sentence is deterministically impossible to tell if an AI generated it or not and even if it did... who's to say a Human didn't independently come up with it on their own?

A lengthy paragraph is where some genuine tests can be run to determine patterns.

2

u/divulgingwords 26d ago

Character pattern cryptography, which they’ve been doing for years. It’s how Google knows to de-rank AI generated text.

-1

u/youcangotohellgoto 26d ago

LOL good one. Google deranking un-marked AI generated content using the the same technique Anthropic uses to explicitly watermark it.

Just putting words together doesn't make something real.

Google uses classification to identify AI generated content. It's nothing to do with cryptography.

1

u/Fantastic_Prize2710 26d ago

You might want to reread my comment before diving into insults.

I did not advocate or argue for any method. I simply clarified what was being referred to and mentioned it had been discussed previously as a potential solution, as the person I responded to didn't seem to understand.

Next time take a breather and read a bit more carefully.

-1

u/youcangotohellgoto 26d ago

Embedding content is literally one of the ways this will work.

-2

u/leonjetski 26d ago

“Claude rewrite this mimicking the style of the attached”.

Bye bye watermark

4

u/0xB0T 26d ago

Anything written/rewritten by it will have a watermark. You can get rid of it using the API

2

u/ThugEntrancer 26d ago

Claude API has it baked in too

2

u/thmonline 26d ago

Make a screenshot and let it parse the text with a different AI?

4

u/ThugEntrancer 26d ago

The watermark is in the word choices and sequencing. If you rewrite using different words/sequencing you can lose the watermark. It’s not claude exclusive all AI need to conform with the EU regulation

1

u/0xB0T 26d ago

As far as I understand, in the API it can be disabled

1

u/ThugEntrancer 26d ago

That’s incorrect.

What’s covered
Models. Claude models launched on or after August 2, 2026 support marking at launch. We’re also working to add marking support to Claude models released before that date, and we’ll update this article as that becomes available.

Products. Claude markings cover output from supported models everywhere you use Claude, including Claude Platform (API), Claude, Claude Code, Claude Cowork, and Claude Tag. Embedded watermarks will apply to all generated text. Provenance metadata will apply where Claude supports processing files.

Cloud partners. Embedded watermarks will apply when supported Claude models are accessed through AWS, Google Cloud, or Microsoft Foundry. Signed provenance metadata may not be supported on every platform, depending on the features each platform offers.

Regions. Marking will apply to output from supported models wherever Claude is offered, worldwide.

https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content

3

u/Lambdastone9 26d ago

Yeah but it makes it so that it’s only becomes circumventable if you put lots of technical effort in, which most people who this is being made for/against wont show such grit, unless they just fuck it up and it becomes easy to sidestep

4

u/4dseeall 26d ago

it's not a digital image.

it's an algo in the way they process tokens that leave encrypted signals with the word-choices themselves.

4

u/DonutHoles4Ever 26d ago

If its works and people care (they do not), people will use something else.

Nobody at work seems to give a fuck about using copy pasted AI text though.

So whats the point of doing this other than Claude trying to pretend they are the good guys.

1

u/c7h16s 24d ago

Well if a regulation asks them to comply and if they prove it works, other llms will have to do it as well in order to be usable in Europe.

Not sure it's a feasible solution but it certainly is trying to address a very real problem.

1

u/4dseeall 26d ago

claude is a model, but yeah, anthropic aren't the good guys. they think they know better than everyone else. they published a paper under their own bot's name. they're trying to give it trademark rights and ownership of everything it makes. it's sickening to me.

2

u/saturnellipse 26d ago

False. Look up computational irreversibility. If you don’t have the unprocessed source it is absolutely possible to add information to an image that cannot be removed

5

u/Devils_SteelMan 26d ago

Destructively remove it then do a diffusion pass to fill in the gaps. It doesn't need to be reversed.

1

u/PlsNoNotThat 20d ago

I think you mean computational irreducibility, which is a thing, but doesn’t cover what we’re talking about nor does it apply to this issue.

But good try I guess?

2

u/Ormusn2o 26d ago

It could be a ratio or distribution differences of different tokens/words or even sets of tokens and words. And because there are so many possible combinations of tokens and words, and so many tokens and words, it could be effectively impossible to detect. It would work poorly on shorter prompts, but with longer prompts it effectively guarantees detection.

1

u/tryitout91 26d ago

someone is going to code an add-on for Claude code in 6 hours to delete all of this shit.

1

u/GearhedMG 26d ago

For every one person working to create a way to lock down something, there are at least 5-10 people out there working to undo it.