3
4 Comments

Just scraped 30,000 prospects in 6 requests

Just thought I'd share.

Found an app online that I thought I could build a nice list from. GraphQL API was loading 10 prospects at a time. Tweak the request to 5000, 6 requests to get all of them, skip the rate limiting problem. Tried 10,000 but it crashed.

Thx for the list!

  1. 2

    Hell yeah! Love using the Network tab to find internal API's. This is a nice haul!

    A similar thing I've found on some old websites, sometimes with the pagination they also use a size parameter that controls how many items to load per page. If it's not locked down, you can often just force all results to load through that parameter. I found a website the other day where they had locked that parameter down on their website, but not the internal API they were using to fetch results from. Handled hundreds of results out just fine.

    1. 1

      LOL classic mistake implementing security in the browser.

      Way back in the day, I "hacked" my way into paid hotel wifi by just looking at the page source. If you paid at the front desk they would give you a paper code, and that code was just in the JS.. and they would just redirect.

      Love stuff like that.

  2. 1

    Hey @connorbode

    I'm new to this all...

    How can a non-technical person like me scrape the web for prospect emails?

    I have some linkedin plugins but it's too manual of a process...

    1. 1

      I dont actually automate everything (because it's illegal in Canada). I do automate grabbing the first list, but then i manually go through to qualify (based on the bio or website or whatver), before manually reaching out.

      I dont know any non-technical tools.

Trending on Indie Hackers
How I grew a side project to 100k Unique Visitors in 7 days with 0 audience 49 comments Competing with Product Hunt: a month later 33 comments Why do you hate marketing? 29 comments My Top 20 Free Tools That I Use Everyday as an Indie Hacker 16 comments $15k revenues in <4 months as a solopreneur 14 comments Use Your Product 13 comments