1
2 Comments

[Poll] What is your biggest challenge related to web scraping?

I'd be curious to find out what your biggest challenge is related to web scraping.

What is your biggest challenge related to web scraping?
  1. Coming up with robust element selectors
  2. Scraping AJAX pages
  3. Website's structure updated quite often
  4. Bypassing anti-scraping measures
  5. Getting past geo-blocking
  6. Something else (please comment)
Vote
posted to Icon for group Webscraping
Webscraping
on February 8, 2021
  1. 2

    Selenium does almost everything “well” BUT scraping sites with dynamic content is challenging. Don’t you think?

    1. 2

      It is indeed, especially for scalability reasons. A headless browser is much slower and more resource-intensive as a HTML parser, so a good practice would be to use headless browsers only if absolutely necessary.

Trending on Indie Hackers
I spent $0 on marketing and got 1,200 website visitors - Here's my exact playbook User Avatar 58 comments Veo 3.1 vs Sora 2: AI Video Generation in 2025 🎬🤖 User Avatar 29 comments Codenhack Beta — Full Access + Referral User Avatar 21 comments I built eSIMKitStore — helping travelers stay online with instant QR-based eSIMs 🌍 User Avatar 20 comments 🚀 Get Your Brand Featured on FaceSeek User Avatar 18 comments Day 6 - Slow days as a solo founder User Avatar 16 comments