What is your dream data set? I'll try and get it for you.

There is a lot of data across the internet. If you could have access to something specific, what would you want?

During my day job and while putting together scrapers for my site, I stumble upon all kinds of interesting data and am constantly surprised about what is freely available to anyone with a little bit of web and scripting skills.

  1. 1

    Just a small bug report: your website has some z-index issues with the background on macOS Safari.

  2. 1

    A list of all existing domain names of the .be tld.

      1. 1

        Interesting, thanks

      2. 1

        This comment was deleted a month ago.

  3. 1

    That's a big question!

    Right now, it would be a list of all residences in a few ZIP codes around my city, who owns them now, and when they last sold.

    I'm working on a real estate app with a friend, and a lot of this data is available via county records/maps online. I just don't know how to scrape it. We'd use this data to populate the app for our MVP.

    1. 1

      know python?

      a nice trick to speed up / jump start any scraping project is to google "<name of site you want to scrape> scraper site:github.com". if it is a popular site, someone will have likely already published a project.

      these might also be interesting sources for you:

  4. 1

    Hey Steve, what are some examples of the types of data you come across?

    1. 1

      lately, i have been working on putting together a dataset on rising/successful companies to add to my jobs site and so i'll use it as an example:

      • enrichment api's - clearbit for example can provide logos, locations, and social links for companies by name
      • venture capital / investment portfolio pages - pages like this one are on every vc's website. they are easy to scrape and provide great leads for new companies to look into

      other useful stuff:

      for non-standard example, i had been an solar engineer / analyst in a prior role. you can find weather data for free on government sites. combine this with pricing data supplied by solar manufacturers and you can model the profitability of a solar project.

      there is sooo information much you can find. what interests you?

Trending on Indie Hackers
📈 We raised $500K pre-seed for our Reddit Marketing Tool 24 comments How I Acquired a Mobile App on Flippa (and grew it from $700/MRR to $5.4k) 17 comments Steph Smith on making $130k w/ an ebook, creating a course in 20 days, and the latest trends 10 comments Feedback, please: does the landing page explain the product? 7 comments Frictionless Customer Knowledge Base needs beta users 6 comments Tool to edit a mockup video? 2 comments