Day 3 of building ScrapeForge

Today was one of those days that remind you why scraping is hard.

Morning routine stayed the same. Shared yesterday’s update and solved one LeetCode problem.

Then I started building the SERP scraper using Playwright. Things worked initially, but very quickly I ran into captchas, IP blocking, device fingerprinting, and other anti-bot protections.

I spent the rest of the day trying to improve the scripts to behave more like a real user by adding delays, reusing browser contexts, and tweaking execution flow.

I did not finish the scraper by EOD, but I learned a lot and gained serious respect for anyone building reliable scrapers.

Would love to hear how others here approach scraping challenges.
On to Day 4.