Just scraped 30,000 prospects in 6 requests

Just thought I'd share.

Found an app online that I thought I could build a nice list from. GraphQL API was loading 10 prospects at a time. Tweak the request to 5000, 6 requests to get all of them, skip the rate limiting problem. Tried 10,000 but it crashed.

Thx for the list!

  1. 2

    Hell yeah! Love using the Network tab to find internal API's. This is a nice haul!

    A similar thing I've found on some old websites, sometimes with the pagination they also use a size parameter that controls how many items to load per page. If it's not locked down, you can often just force all results to load through that parameter. I found a website the other day where they had locked that parameter down on their website, but not the internal API they were using to fetch results from. Handled hundreds of results out just fine.

    1. 1

      LOL classic mistake implementing security in the browser.

      Way back in the day, I "hacked" my way into paid hotel wifi by just looking at the page source. If you paid at the front desk they would give you a paper code, and that code was just in the JS.. and they would just redirect.

      Love stuff like that.

  2. 1

    Hey @connorbode

    I'm new to this all...

    How can a non-technical person like me scrape the web for prospect emails?

    I have some linkedin plugins but it's too manual of a process...

    1. 1

      I dont actually automate everything (because it's illegal in Canada). I do automate grabbing the first list, but then i manually go through to qualify (based on the bio or website or whatver), before manually reaching out.

      I dont know any non-technical tools.

Trending on Indie Hackers
Shoutout your project: What's your unfair advantage? ✨ 46 comments I'm a non-technical founder who built a fully automated, AI powered patient tracking platform with nocode tools (Mobius). AMA! 19 comments Why does every SaaS website on Indie Hackers look exactly the same? 16 comments How to find billion $ statup idea 10 comments From K8s to Serverless and Back Again 8 comments Which startups started as a spreadsheet? 2 comments