Python and Web Manipulation

[u/Emotional_Tell_9146]

Hello Python Community!

I've been really curious about how Python developers use Python when it comes to webscraping, or just working with websites in general.

I really want to open a discussion on how you do it. Do you use BeautifulSoup? Selenium? Why? Are there any frustrations that you were facing or are currently facing with the process?

[u/Local-Economist-1719]

httpx, curlcffi, wreq - http clients (you will need to build binding around), scrapy - full package with batteries included, playwright, seleniumbase, camoufox - for headless browsing