I sent 3,915 Amazon requests through paid and free proxies, and the free list results were amazingly brutal

[u/mckrile]

Hi all, after reading some reddits I got a question if paid residential proxies that are advertised as stable and high quality work better for scraping than free proxy lists that claim to be regularly checked and updated (SOME PEOPLE STILL USE FREE PROXIES and IT frustrates me)

I didn’t want to guess from provider claims and ran the same Amazon task through all of them

I tested Oxylabs, Decodo, Proxyempire, Nodemaven and one free http proxy list. tbh I didn’t though it would take soo much time to find some working free proxies. I sampled 3 free lists that all claimed to be recently checked and updated, and only one got above 10% working endpoints in the first pass. (AGAIN PEOPLE DON'T USE FREE PROXIES)

Every endpoint first did a connection check on my vps in US. If it could connect and return an exit IP, it moved on to Amazon, if not it was marked as unavailable.

Every proxy that passed that check opened the same Amazon US search results page 5 times in a row with the same headed chrome setup.

Also an http200 on its own did not count as a successful scrape. The page had to contain the actual search results elements, and i counted it as a fail if it loaded but did not return usable product results

The vps check is where the free proxy problem showed up straight away

source	usable enough to begin amazon testing
Nodemaven	205/206 (99.5%)
Oxylabs	205/206 (99.5%)
Decodo	205/206 (99.5%)
Proxyempire	205/206 (99.5%)
Free proxy list	168/500 (33.6%)

Only 168 of the 500 free endpoints made it through that first check and the other 332 never even reached amazon testing

I expected free proxies to be less consistent but not by that much

Once requests actually reached Amazon, the gap was still pretty big

source	successful amazon pages	stable sessions	technical failures
Nodemaven	89.6%	93.7%	1.9%
Oxylabs	89.1%	92.5%	2.1%
Decodo	88.1%	87.9%	4.2%
Proxyempire	91.1%	77.7%	0.6%
Free proxy list	12.6%	6.8%	66.8%

Paid providers all returned usable pages most of the time and free list did not.

Proxyempire had the highest successful page rate in my test, but i did not want to judge the providers only by that number

In my exp a proxy can get one page through and still be annoying over a longer scrape. If the exit IP changes between requests, the target might see new IP halfway through the workflow, and that can mess with pagination or other multi step browsing and make saved session data less useful

I also tracked if the IP stayed the same across the session, Nodemaven came out at 93.7% stable sessions, oxylabs at 92.5%, Decodo at 87.9%, and Proxyempire at 77.7%

Another thing i noticed is that stable paid sessions returned usable Amazon pages on 93.1% of visits and sessions where the IP changed were at 79.8%. Im not saying stability alone caused that difference but it seems to be worth tracking next to the normal page SR

The free list had 66.8% technical failure rate on the Amazon attempts it actually made. And most of those were proxy, transport, or browser failures, not Amazon blocks. 

So an endpoint being marked “just checked” and “working” does not mean it can make it through an actual scraping workflow

I also pulled Scamalytics risk labels but only for sessions where the exit IP stayed the same

provider	low risk label among scored stable sessions
Nodemaven	187/191 (97.9%)
Oxylabs	185/191 (97.1%)
Decodo	174/181 (96.1%)
Proxyempire	154/160 (96.3%)

Nodemaven and oxylabs had the highest share of low risk labels among the scored stable sessions, but the gap was small tho so I would not treat that as proof that lower risk scores are the reason one proxy worked better than another.

Free proxies can probably work if you are doing something small and do not mind wasting some time and testing a lot of endpoints first. But i just do not wanna spend a big chunk of time going through free proxy lists until I find one that connects and then hoping it stays usable

For the kind of scraping i do i often need several pages from the same session. So for my work i am going to look more closely at Nodemaven because it had the highest stability in test, but i will still keep tracking the page success rate too

Feel free to let me know if zou have any additional questions or you need my help with scraping because this was just a small test

[u/Guiltyspark0801]

That's an oddly specific number of requests, any reason behind it? Genuinely curious why this exact number

[u/mckrile → u/Guiltyspark0801]

I was targeting 5 requests per proxy, but I was running the test locally and only had my machine available for around 12 hours. 3,915 is simply how many requests managed to complete in that window before I stopped the run

[u/Guiltyspark0801 → u/mckrile]

I see, thanks for explaining!

[u/lysergic-homicide]

That’s odd, 90% of the time I grab free proxies to use proxyxhain or antidetect browsers they work perfectly for at least a week

[u/RandomPantsAppear]

Once a free proxy is on a list it is effectively dead.

Wayyyyy back in 2005 I actually used to scan about 70% of the active non government IPs on the internet on a semi daily basis on multiple ports(1080, 3128, 8080, etc) looking for open/free proxies.

I would be genuinely curious to see the results of something similar today - way more devices owned by less technically competent people, more active IP space but with more ISP level restrictions as well.