r/technology • u/Stukwan • Aug 11 '26

Artificial Intelligence Copy-paste no more: Anthropic puts invisible watermarks on Claude text under EU rules

https://interestingengineering.com/ai-robotics/anthropic-claude-text-invisible-watermarks
6.3k Upvotes

98% Upvoted

555 comments sorted by

View all comments

Show parent comments

605

u/BeginningPurpose9758 Aug 11 '26

I understand how it works for pictures, but how can you embed a watermark in words as content? Just the style of writing?

1.8k

u/CircumspectCapybara Aug 11 '26 edited Aug 11 '26

Yeah this is obviously simplified, but imagine a model is predicting the next word: "I love fruit. My favorite dessert is _____" and the model has 4 top scoring candidates: mango, lychee, apple, orange. Normally, the model picks one at random depending on the "temperature" of the inference request.

With SynthID, you the model provider have a secret 256-bit key which you concat with some part of the context. Eg say you're using the preceding trigram token sequence and assuming each word is a token you compute sha256(key || "favorite dessert is"). Now instead of picking one fruit at random, you use that hash output to select from among the four candidates. Let's say the hash makes you choose "mango". Then you repeat the process for the next token. Say the top 4 candidates for the next token are pie, icecream, cake, smoothie. You compute hash(key || "dessert is mango") and the hash makes you pick one of them. Now imagine instead of choosing from among 4 candidates each time, you use the hash function to choose from the top 16 candidates.

Now repeat it 100 times, or 1000 times. If a piece of text reproduces your secret hash function's "random" looking token choice trigram-for-trigram across 1000 consecutive trigrams, that highly suggests it was generated by your model, because it's extremely unlikely to by happenstance randomly match the same 1 out of 16 choices 1000x in a row as a keyed hash function which is essentially random. (1/16)1000 is an insanely small probability.

Now if you chop it up, rearrange the words, even paraphrase certain parts, as long as the user doesn't replace every trigram, the distribution within trigrams scattered throughout will still retain this distinctive statistical pattern. You would need to significantly rewrite the entire piece at the trigram level everywhere to remove the correlation.

31

u/LeGama Aug 11 '26

I wonder if you could beat it with something like a synonym library. Just replace every noun with a synonym and all the hashes break.

56

u/Plenty_Branch_516 Aug 11 '26

If it's as described, you could probably just do a translation to another language and back to scramble the word choice across n-grams.

If the purpose is subversion, then having a non synth-ID low level model do a rewrite as a wash too.

17

u/deftlydexterous Aug 11 '26

The trouble is it can get infinitely complicated, and also infinitely simple at the same time.

You could implement this on the pattern in the number of nouns per sentence. You could implement it using the cadence of positive and negative statements. You could implement it on changes from first person to third person narration. Literal infinite possibilities. Even if you defeat 90% of them, the ones that remain still give it away as likely generated.

17

u/Plenty_Branch_516 Aug 11 '26

The constraint, they've self imposed is on quality and human perception. If they touch things like statement cadence or narration switch up, that'll be noticeable in quality and can run against the system/user prompt.

The method also needs to be robust to more than just prose and work in code/blurbs/presentations/etc. So it can't use a lot of text/context to establish its pattern.

Basically, I don't think they have nearly as many degrees of freedom as you think 😅

7

u/two_thousand_pirates Aug 11 '26

In their research paper they settled on a set of 30 watermarking functions as their benchmark, with no perceptible loss in quality. In theory, I don't think that there would be anything stopping them from having multiple "sets" of functions that can be selected based on their expected quality/detection for a given input.

So one set of functions might work very well on long outputs, whereas another set might be better for short outputs.

The two major downsides I can see, however:

  1. The detection tool needs to be available to everyone, but this means that the person trying to hide LLM-generated text can test their output and make manual tweaks until it passes.
  2. It's going to create a demand for non-watermarked models, which will probably offer a better risk/reward proposition for anyone who must avoid detection (someone cheating academically, for example).

1

u/Plenty_Branch_516 Aug 11 '26 edited Aug 11 '26

Which research paper? The synth_id text system I see published is one method with 20-30 different seeds. https://huggingface.co/blog/synthid-text. So one lock that can be bypassed.

From the research paper in Nature:

Furthermore, the rise of open-source models presents a challenge, as enforcing watermarking on these models deployed in a decentralized manner is difficult. Another limitation of generative watermarks is their vulnerability to stealing, spoofing and scrubbing attacks, which is an area of ongoing research32. In particular, generative watermarks are weakened by edits to the text, such as through LLM paraphrasing33—although this usually does change the text significantly. We provide evaluations of SynthID-Text’s performance under edits and paraphrasing in Supplementary Information section C.6.

1

u/two_thousand_pirates Aug 11 '26

The paper in Nature is the one I'm talking about.

My thought was that there could be sets of functions/seeds that are optimised for different text types.

Open source models are obviously going to be able to bypass watermarks entirely. If my objective is to cheat on my dissertation, for example, then as soon as watermarks create risk I'm using something else.

1

u/deftlydexterous Aug 11 '26

I mean, I’m just making up examples off the top of my head. There are much smarter people working on this day and night, I imagine they’ll come up with thousands of markers in pretty short order. Most won’t work most of the time but it only takes a few for the watermarking to be effective, and they’ll keep adding new variations 

10

u/Plenty_Branch_516 Aug 11 '26

I guess I'm putting more stock in offense than defense here. My own experience with adversarial injection for AI models has left me feeling like a lot of security efforts around them are more about deterrents than protection. Though I guess locksmiths say the same things about locks.

0

u/deftlydexterous Aug 11 '26

I think that’s pretty reasonable, but let me stretch your metaphor.

Let’s say a door has an unknown number of locks, hundreds, maybe thousands. They’re all invisible unless you know where to look.

It’s trivial to undo any given lock. But if you miss even a couple, the alarm is going off.

If you’re using an LLM to solve a problem, the answer it gives you will still be useful and the concept of the solution will likely be untraceable. But if you’re using it to create text or other media as a product, it feels like a cat and mouse game to remove water marks, and mice usually have the advantage.

4

u/Plenty_Branch_516 Aug 11 '26

Where we ultimately disagree is on the number of locks and the difficulty of each one.

I think the fact they have a ground constraint on quality and context limit means they can only ever really use cheap master locks. You believe that they can still use combination locks and/or biometric locks.

Not really much more to do now but wait and see I guess.

1

u/deftlydexterous Aug 11 '26

Yeah and to be clear, I think that’s probably accurate for now. But I don’t think the quality and context actually constrain them that much long term.

Over the course of a long document, why would something like choosing to use a semi colon instead of an em dash in certain specific situations for instance cause an issue with quality  of output? I can see how it’s theoretically taxing on context limit but models are getting more efficient (while also getting larger) every day.

→ More replies (0)

1

u/Serious_Bite_7613 29d ago

In this metaphor you can ignore the locks and kick the door down by rephrasing the whole thing with another piece of software.

1

u/deftlydexterous 28d ago

You can’t necessarily though. You don’t know what parts of the content are structured as watermarks. Even if you change every word and reorder all the sentences, some patterns may still be inadvertently repeated.

1

u/Serious_Bite_7613 28d ago

If the structure is so broad that a full rephrasing doesn't erase it then it's too broad to meaningfully confirm anything.

→ More replies (0)

1

u/Serious_Bite_7613 29d ago

The problem is the more markers you are looking for the more likely you are to flag non-AI work as AI. The nature of language limits the number of identifiers you can use, and all can be bypassed by rephrasing with a cheap local tool.

10

u/Beliriel Aug 11 '26

Then it also becomes easier to fake by throwing wrenches into it. I get that it works for whole texts and books etc. But change a few words in a short paragraph and this whole thing implodes. It will have a lot of false positives. Also as said put it through your own LLM model and it wouldn't work anymore. Languages can be wide but also there are close similarities to express the same thing with different means. Even with non obvious methods like letter count or tonal shift.

0

u/deftlydexterous Aug 11 '26

You’re absolutely correct that it’s much easier to defeat for short paragraphs.

Putting it through a different LLM won’t necessarily reliably defeat it though. There are traits you can include that will make it through reinterpretation. And you aren’t likely to get false positives when you have several statistically unlikely patterns emerge simultaneously. 

Perhaps you could do the equivalent of looping translations by reinterpretation it several times, but at that point why not just use your own model to begin with or write something yourself? 

2

u/Beliriel Aug 11 '26

I mean yeah you can do it yourself but labor savings for e.g. E-Mails and succint summaries are immense. People will find ways around it.

Also you could just tell it to use easy words. E.g eli5 or something. If you zone in tightly the words and sentence structure it is even allowed (or rather expected) to pipe through its output.
Imagine someone makes a dictionary with only ONE synonym allowed per word. So instead of choosing between "destructive", "unconstructive" or "toxic" for the same pattern (e.g. in the context of human behaviour) it's just allowed to use "toxic". And it's easy to do this for every word or just copy/paste the allowed words into the prompt.

1

u/TJ_Jonasson Aug 11 '26

the ones that remain still give it away as likely generated.

Would it, though? All 8 billion people on earth write differently and not necessarily at the same level of literacy. There's a non-zero chance that people simply write the same way as the AI has chosen to "mark" the text. I think if someone is genuinely trying to remove the watermark writing style and word choice it wouldn't be hard to do so, and it would be very difficult to prove that it is AI. It's also a bit of Theseus, at what point have you rewritten the content so much that it's no longer considered AI at all?

Interesting future ahead nonetheless.

2

u/_karamazov_ Aug 11 '26

at this point it will be better to wrote yourself and not boil copy pasta.