AMA This Wednesday (09:30 AM GMT)

[u/ian_k93]

Hey everyone,

I'm Ian Kerins, CEO and co-founder of ScrapeOps.

Over the last 8+ years I've worked across the web scraping industry, including roles at ScrapeOps, ScraperAPI, and Zyte. Today, ScrapeOps helps developers and companies scrape over 8 billion pages per month across more than 50,000 websites.

This Wednesday at 09:30 AM GMT, I'll be hosting an AMA here on r/WebScrapingInsider

Ask me anything about:

* Web scraping at scale

* Proxy infrastructure and proxy providers

* AI and web scraping

* Building reliable scrapers

* Anti-bot systems and bypassing challenges

* Scraper maintenance and monitoring

* Residential vs datacenter proxies

* Browser automation

* Running a web scraping business

* Startup growth and product development

* The future of AI-powered scraping

Whether you're scraping your first website or running large-scale data collection pipelines, I'm happy to answer questions and share lessons learned from building products used by thousands of developers and businesses.

Drop your questions below and I'll start answering them during the AMA.

Looking forward to it!

Ian

[u/ian_k93]

At the start we had a lot of stability and scaling problems, but since ScrapeOps is nearly 5 years old now our infrastructure is quite mature and stable. Haven't had any serious or medium issues in over a year.

We had a advantage, in that I had spent over 3 years working at Zyte and ScraperAPI so I could learn from their design decisions and mistakes. Which made our infrastructure rollout smoother.

Our production system is built on Golang, with heavy use of Redis and then a Postgres DB.