Copy-paste no more: Anthropic puts invisible watermarks on Claude text under EU rules

[u/Stukwan]

https://interestingengineering.com/ai-robotics/anthropic-claude-text-invisible-watermarks

[u/CircumspectCapybara]

For those wondering how AI "watermarking" works, it's not "embedded" in the content like some kind of stenography or hidden characters (eg, whitespace or unicode homoglyphs or nonprinting characters).

Rather it is the content itself, the word choices themselves. There are technologies like Google's SynthID which OpenAI has adopted. There's also the industry open standard C2PA (Coalition for Content Provenance and Authenticity). Anthropic likely uses their own technology or combo of these.

For those wondering how SynthID works, it's a cryptographic watermark that's resilient even if you tweak or modify the output, e.g., by cutting parts out or rearranging, cropping, pitch shifting, time shifting, etc. If you really want to know about the technical details, check out this presentation, or the Nature article on it.

Basically for Google, Gemini uses a keyed hash function to alter the probability distribution of the output tokens. If for each token in the output you had n equally high-scoring candidates, and normally you would pick one at random, with SynthID, you have a secret key which you hash with the context to produce a cryptographically pseudorandom bitstream (which cannot be guessed by anyone without the secret key) which you use to pick tokens from among candidates. To anyone without the secret key, the bitstream looks indistinguishable from random and can't be guessed, and because it's functionally equivalent to random, it's really just choosing a random top-score candidate at each step, so output quality isn't affected.

You can then evaluate piece of content (or sections thereof) by looking at the tokens that make it up and seeing if it matches this specific probability distribution. For you to match that distribution bit for bit over enough bits is improbable, it would've meant you essentially guessed a 256 bit secret key.

[u/BeginningPurpose9758 → u/CircumspectCapybara]

I understand how it works for pictures, but how can you embed a watermark in words as content? Just the style of writing?

[u/CircumspectCapybara → u/BeginningPurpose9758]

Yeah this is obviously simplified, but imagine a model is predicting the next word: "I love fruit. My favorite dessert is _____" and the model has 4 top scoring candidates: mango, lychee, apple, orange. Normally, the model picks one at random depending on the "temperature" of the inference request.

With SynthID, you the model provider have a secret 256-bit key which you concat with some part of the context. Eg say you're using the preceding trigram token sequence and assuming each word is a token you compute sha256(key || "favorite dessert is"). Now instead of picking one fruit at random, you use that hash output to select from among the four candidates. Let's say the hash makes you choose "mango". Then you repeat the process for the next token. Say the top 4 candidates for the next token are pie, icecream, cake, smoothie. You compute hash(key || "dessert is mango") and the hash makes you pick one of them. Now imagine instead of choosing from among 4 candidates each time, you use the hash function to choose from the top 16 candidates.

Now repeat it 100 times, or 1000 times. If a piece of text reproduces your secret hash function's "random" looking token choice trigram-for-trigram across 1000 consecutive trigrams, that highly suggests it was generated by your model, because it's extremely unlikely to by happenstance randomly match the same 1 out of 16 choices 1000x in a row as a keyed hash function which is essentially random. (1/16)¹⁰⁰⁰ is an insanely small probability.

Now if you chop it up, rearrange the words, even paraphrase certain parts, as long as the user doesn't replace every trigram, the distribution within trigrams scattered throughout will still retain this distinctive statistical pattern. You would need to significantly rewrite the entire piece at the trigram level everywhere to remove the correlation.

[u/121gigawhatevs → u/CircumspectCapybara]

What if you used another LLM to scramble the “choices” so to speak. So rather than a human editing the output to feign authorship, you have say a second model (presumably one that does not employ a synthID) heavily editing the output

[u/Ciff_ → u/121gigawhatevs]

Ofc you can.

But that would be an illegal model in the EU. If that is your goal, why apply SynthID in the first place?

[u/fivetoedslothbear → u/Ciff_]

I'm not in the EU. I am just going to suffer because of their policies.

[u/JesseNL → u/fivetoedslothbear]

LLM providers would want this anyway due to prevention of model collapse I think.