7
9 Comments

What is your dream data set? I'll try and get it for you.

There is a lot of data across the internet. If you could have access to something specific, what would you want?

During my day job and while putting together scrapers for my site, I stumble upon all kinds of interesting data and am constantly surprised about what is freely available to anyone with a little bit of web and scripting skills.

  1. 1

    Just a small bug report: your website has some z-index issues with the background on macOS Safari.

  2. 1

    A list of all existing domain names of the .be tld.

      1. 1

        Interesting, thanks

      2. 1

        This comment was deleted 3 years ago.

  3. 1

    That's a big question!

    Right now, it would be a list of all residences in a few ZIP codes around my city, who owns them now, and when they last sold.

    I'm working on a real estate app with a friend, and a lot of this data is available via county records/maps online. I just don't know how to scrape it. We'd use this data to populate the app for our MVP.

    1. 1

      know python?

      a nice trick to speed up / jump start any scraping project is to google "<name of site you want to scrape> scraper site:github.com". if it is a popular site, someone will have likely already published a project.

      these might also be interesting sources for you:

  4. 1

    Hey Steve, what are some examples of the types of data you come across?

    1. 1

      lately, i have been working on putting together a dataset on rising/successful companies to add to my jobs site and so i'll use it as an example:

      • enrichment api's - clearbit for example can provide logos, locations, and social links for companies by name
      • venture capital / investment portfolio pages - pages like this one are on every vc's website. they are easy to scrape and provide great leads for new companies to look into

      other useful stuff:

      for non-standard example, i had been an solar engineer / analyst in a prior role. you can find weather data for free on government sites. combine this with pricing data supplied by solar manufacturers and you can model the profitability of a solar project.

      there is sooo information much you can find. what interests you?

Trending on Indie Hackers
How I grew a side project to 100k Unique Visitors in 7 days with 0 audience 49 comments Competing with Product Hunt: a month later 33 comments Why do you hate marketing? 29 comments My Top 20 Free Tools That I Use Everyday as an Indie Hacker 16 comments $15k revenues in <4 months as a solopreneur 14 comments Use Your Product 13 comments