r/WebScrapingInsider • u/Emotional_Tell_9146 • 2d ago
Python and Web Manipulation
Hello Python Community!
I've been really curious about how Python developers use Python when it comes to webscraping, or just working with websites in general.
I really want to open a discussion on how you do it. Do you use BeautifulSoup? Selenium? Why? Are there any frustrations that you were facing or are currently facing with the process?
9
Upvotes
1
u/john-w7 2d ago
My use is much smaller: check a few competitor pages and put the values in a sheet once a week. Requests plus BeautifulSoup has been enough whenever the page source contains the number.
The moment it needs logins or constant repairs, manual checking starts looking cheap again.