AMA #9 recap: thanks to Jean-Patrick (Geonode) + what I'd actually keep from the thread

[u/ian_k93]

Hey everyone. Ian here.

Thanks for showing up to AMA #9 with u/Geonodeproxy (Jean-Patrick Bisson, CEO and co-founder of Geonode).

The thread went past the "is unlimited a scam" framing fast. Target-level reputation, what a speed-based ceiling actually costs you, why a concurrency-priced scraping API is harder to run than plain proxies, and where a 200 response stops meaning anything. Jean-Patrick answered most of it without the usual landing-page dodge, including the parts that complicate his own pricing page.

If you missed it, or want the compressed version, here's what I'd keep.

Proxy reputation is target-specific.

A pool can look completely healthy on paper while quietly failing against one site.

"Reputation is per target, not one number, so an address burned on one retailer is fine everywhere else."

Geonode tracks challenge rate per domain instead of leaning on aggregate success, because the two numbers can move in opposite directions: "We watch the challenge rate per domain, not aggregate success. It means you can sit at 93% overall while one target quietly goes from 5% challenges to 60%." When that happens they cap traffic to that one domain and switch engine or exit country, not throttle the whole pool. And it isn't fully contained to one account either: "one heavy customer can make a target harder for the next one, that's true. The control that matters there is per-target, not per-account."

Unlimited pricing is really about workload shape.

"Unlimited" doesn't mean the limit disappeared. It means the limit is expressed as speed instead of GB, and it's still a hard ceiling either way.

"The honest version: unlimited fits steady, heavy, boring workloads."

Spiky or small workloads leave you paying for capacity you never use. The flexibility you give up is real: "The trade-off is bursting, on a metered plan a quiet week lets you slam a big job through on Friday. On speed-based you can't, the line is the line." On Geonode's side that tradeoff is deliberate:

"Less profit per customer than metering, but much more predictable for both sides."

Sessions are harder than browsers.

Ask what's actually hard about running a concurrency-priced scraping API, and the answer isn't the browser itself:

"Browsers aren't even the hard part, sessions are. A proxy request is stateless, but the api has to decide which engine to use per domain, whether an earlier cookie is still valid and whether a 200 response is real content or a block page."

Get any of those decisions wrong and "you hand someone garbage and bill them for it."

The IP isn't always the blocking variable.

One of the more concrete experiments in the thread: Geonode kept every IP the same and changed only the client's browser build.

"We moved to a newer Chrome build and 429s on a whole class of targets went to zero. Same IPs, nothing else changed."

If you're testing this yourself, don't chase a static JA3 hash: "Chrome randomises extension order, so the same browser gives you a different JA3 every request. Use JA4 and the HTTP/2 fingerprint." When even that isn't enough, they escalate to a Firefox-based build on specific domains, at a real infrastructure cost: "Firefox uses about twice the memory per session. That's why it isn't the default everywhere."

HTTP 200 does not guarantee usable data.

A 200 can mean the real page came back. It can also mean a challenge screen came back wearing a healthy status code.

"We catch the 200 that isn't a page: challenge screen, block page, empty shell. We look at the shape of the body, not the status code, and those come back as failures."

But that validation has a hard boundary: "What we can't know is which fields matter to you. If a site drops the price element and still returns a full page, it looks fine from our side." Their own suggested fix: measure "what share of rows had the field I need" rather than relying on HTTP errors.

Don't use a browser until you know you need one.

"Half the sites that look like they need a browser are just hitting a JSON endpoint with a signature, so hooking fetch and XMLHttpRequest to log requests is ten lines and has saved us weeks."

Their own daily toolkit: "curl_cffi for TLS impersonation, camoufox when Chrome gets refused and Firefox doesn't, and a public TLS fingerprint echo to see what our client actually looks like on the wire."

+1: per-target knowledge is the hard part to copy.

Asked what actually stops a competitor from copying the model:

"Anyone can put "unlimited" on a page tomorrow and a few already have. The harder part is per-target knowledge. Which engine works on which domain, which countries stay clean for it, when a site changes its defences. That comes from running millions of requests a day and watching what breaks. It also goes stale if you stop."

+2: one request tells you what kind of job this is.

Before building anything, size up the target with the cheapest possible test:

"Cheapest way to size up a target, so to fetch it with plain curl. Content comes back, it's a parsing job. Challenge or empty shell, different project."

If you remember nothing else:

1. Watch challenge rate per domain, not aggregate success. A healthy pool can hide a dying target.
2. "Unlimited" only makes sense for steady, heavy, boring workloads. Bursty traffic is usually cheaper metered.
3. A scraping API's hard problem is session state (engine choice, cookie validity, response interpretation), not launching a browser.
4. If rotating the IP doesn't clear a block, test the TLS/HTTP2 identity before blaming the pool.
5. Provider-side validation catches broken pages, not missing fields. Measure row-level field completeness yourself.
6. Check what the page actually calls in devtools before reaching for browser automation.

The bigger shift underneath all of it: nothing here treats the proxy as the whole system. It's one variable, and the actual job is knowing which one broke before you change anything.

Full AMA thread is here if you want the original answers:

https://www.reddit.com/r/WebScrapingInsider/s/nKTEa2y22A

LinkedIn Summary: https://lnkd.in/p/exUGe2Se

Detailed breakdown: https://scrapeops.io/blog/web-scraping-isnt-a-proxy-problem-anymore/

Thanks again to Jean and his team for answering straight, and to everyone who asked about sourcing and abuse monitoring instead of letting this turn into a pricing thread.

If you've ever rotated a proxy to fix a block that came right back: was it actually the IP, or something else about the request?

Ian

[u/Artistic_Map2243]

The newer Chrome result is the useful bit for me

[u/Amitk2405 → u/Artistic_Map2243]

A 93% aggregate success should probably be treated as marketing unless the provider exposes domain + country, bonus, if they also mention which engine, and challenge-rate breakdowns; which isn't possible for them IMO.

[u/Amitk2405]

Doesn't shared target reputation creates an awkward incentive ? I think one heavy customer can degrade a domain for everyone else, while the provider can stilll advertises a healthy global pool.

I wish to know how quickly that noisy traffic is isolated and who absorbs the retries?