Where do you get data for your projects?
I have noticed that a lot of projects involve access to lots of data, coming from either third-party APIs or proprietary databases. I would like some good sources for finding general data, be it datasets or APIs, but am not sure where to look.
It's best if the data is continuously updated and fresh, but if it requires scraping every few months that's ok too. Any tips are greatly appreciated!
Trending on Indie Hackers
Here's a great database if you are looking for APIs: https://rapidapi.com/
I used many of them for my own projects, sometimes free
I recently published public APIs browser. It contains more than 600 APIs: https://www.apipicker.com
A big reason I started building Podcasting products (ie. Casts) was because there was so much free, open, and easily accessible data available. If you go the extra step and start downloading and transcribing the audio content, that will result in 10x more data to play around with.
You can try our product IndieData. We provide scraped databases, you can either request a custom one or choose from the existing ones! Shoot us a mail at [email protected] if you would like to know more.
It depends on the nature of the project you're trying to build.
Mostly, people go towards scouting for data to use in their projects and that's one of the reasons why I've built Screpto (shameless plug). It allows you to easily build web scrapers, launch them to run on the cloud to get you fresh data on a timely basis and access results via API / Spreadsheet or webhooks.