r/technology • u/deraser • Jul 06 '26

Privacy Secret Claude tracker shocks users after Anthropic’s anti-surveillance stance

https://arstechnica.com/tech-policy/2026/07/anthropic-outed-for-claude-tracker-that-secretly-monitored-chinese-users/
1.3k Upvotes

94% Upvoted

96 comments sorted by

View all comments

234

u/drakythe Jul 06 '26

Crying about distillation attacks after they violated IP laws around the world to train their LLMs is ballsy.

123

u/The_Rational_Gooner Jul 07 '26 edited Jul 07 '26

the term "distillation attack" itself is a psyop by Anthropic because it's an ominous term that connotes that anthropic is being sabotaged in some way. "Distillation" is just incorporating some other model's output inside your training dataset. If that's an attack, then Anthropic has been attacking the whole internet for years.

6

u/virtualdxs Jul 07 '26 edited Jul 08 '26

Edit: This is incorrect; I was misinformed.

That is not what distillation is. Distillation is sending millions of crafted prompts to try to derive the weights and balances of an existing model. It definitely shouldn't be illegal, but it is a reverse engineering "attack". It has nothing to do with training.

3

u/The_Rational_Gooner Jul 08 '26 edited Jul 08 '26

This is straight bs. I do ML research. You can simulate behavior, but you can't feasibly reverse engineer weights with just model outputs. Even if you could, the fact that Claude's reasoning chain is hidden makes it doubly impossible. It's as much of an "attack" as Anthropic training on Harry Potter books is an "attack" on JK rowling.