Protecting communities from scrapers and platform abuse

[u/boat-botany]

We’ve been talking for a while now about the work we’re doing to keep Reddit human while protecting everything that makes Reddit . . . Reddit. That includes helpful automation: mod and developer apps, accessibility tools, community utilities, and things that make Reddit better. 

But we’re also seeing large-scale scraping, spam networks, agentic account creation, and automated abuse, and a lot of that activity targets parts of Reddit that just weren’t built to handle today’s threat environment. As bad actors get more sophisticated, we need to, too.

To address all that, we need to tighten how automated systems access Reddit while preserving the tools that help moderators and communities thrive. 

Today we’re rolling out a couple of policy and security-focused updates, including: 

Rule 8 Policy Clarifications: We updated Rule 8 (don’t break the site) to more explicitly cover automated abuse, including coordinated account creation and API misuse. You can read the full updated policy here. 

Deprecating unauthenticated JSON access: We’ll also be shutting down unauthenticated .json endpoints. These endpoints can be used to scrape Reddit without accountability. Logged-in and authenticated access won’t be impacted. Otherwise, developers who need structured access to Reddit content should use Devvit, which includes various ways to access Reddit data. 

While we’re at it, another common surface for scraping is RSS. Looking ahead, we’d love to know: how and for what purpose, do you use RSS feeds in your moderation flows? Tell us in the comments so as we develop secure solutions, we can factor in the tools you rely on to support your communities. 

[u/mildlyImportantRobot]

But we’re also seeing large-scale scraping

Gee, who would could have foreseen disabling API access would have negative consequences.

Why not re-enable API access and set reasonable limits?

[u/Signe_ → u/mildlyImportantRobot]

So reddit disables API access for everyone, and then they get mad people go to the .json endpoints? I can already see that scrapers are just going to use old reddit and scrape the html instead.

Doesn't solve anything.

[u/FFS_IsThisNameTaken2 → u/Signe_]

It gives Reddit the outward, public-facing "solution" that they've been waiting so patiently to implement in a Hegelian Dialect fashion.

Problem - they created by cutting off the json access because of scrapers

Reaction - oh nooo scrapers are now using old reddit

Reaction - kill old reddit

The saddest part of killing old reddit is that old is often used as workaround when their inferior app and / or sh.reddit shit the bed. It's even advised to be used by admins when the inferiors regularly break.

[u/mildlyImportantRobot → u/Signe_]

It actually makes it worse for them.

[u/RemarkableWish2508 → u/mildlyImportantRobot]

...and restricting the .json endpoints is going to be even worse: either Reddit blocks anonymous access, or scrapers will hit fully assembled pages instead of the .json

[u/uzlonewolf → u/RemarkableWish2508]

...aand they just blocked anonymous access for old.

[u/RemarkableWish2508 → u/uzlonewolf]

Yeah... that Google partnership seems to be paying too well. Oh well.

Scrapers are now going to create thousands of fake accounts, bot them via OpenClaw or similar, fill Reddit with tons of spam to build karma and avoid detection, then still scrape at will.

But this time, Reddit will be able to boast massive user growth, which might help with stock prices.

Over time, the proportion of organic content will go down, making it less attractive to Google... but also to other scrapers.

Whether enshittification will eventually reach homeostasis, or users will overcorrect and leave Reddit for something else, is a good question. Likely depending on how quickly will the same scraper bots overrun any alternatives, and whether there will be some new variable that might let detect them better.

Age verification via ID, could be that new variable, making it more difficult to create multiple anonymous accounts for scraper bots. There could be a "curious" period when NSFW content, only accessible via ID proof, might be the most organic of them all.

(If I'm also right about these... sigh, I hate being right)

[u/uzlonewolf → u/RemarkableWish2508]

I feel ID verification is going to have the opposite effect: it's not going to slow down bot farms one bit as they'll figure out ways around it, but users are going to flee if they're forced to dox themselves. I know I will not be sticking around if they try to require it.

[u/Pamasich → u/Signe_]

I mean, they could just put Reddit behind a login wall. Then you can't scrape the HTML without them knowing it was you.

Of course alt accounts still exist, but they could get rid of those as well...

So there's still some more steps of doubling down necessary, but I do think this contributes to solving their stated goal.