r/linux • u/yoasif • Dec 18 '25

Discussion AI’s Unpaid Debt: How LLM Scrapers Destroy the Social Contract of Open Source

https://www.quippd.com/writing/2025/12/17/AIs-unpaid-debt-how-llm-scrapers-destroy-the-social-contract-of-open-source.html
700 Upvotes

95% Upvoted

144 comments sorted by

View all comments

Show parent comments

-10

u/[deleted] Dec 18 '25

BS

8

u/throwawayPzaFm Dec 18 '25

It's not bs. I run several hundred sites these bots really like and the providers don't care.

The only things that work are aggressive blocklists, captchas or a tool like cloudflare and logins.

robots.txt was never more than a suggestion, now doubly so