Anyone here made a product out of public data (from gov)? Or selling data in general?
http://aggdata.com
I came across aggdata.com - I have no idea about the legality of what they are doing, but it does seem like a valuable service.
Anyone here doing similar work? Not just scraping, but using public data from gov? US and UK governments especially release tons of data, though a good portion of it is not user friendly (obscure data formats, insufficient documentation etc).
Just curious
Hey there @anabayan. I actually founded AggData and so I can give you a bit more insight on this. Basically, as long as the data is a "collection of facts", it can't be copyrighted and is legal to sell. We extract data that already exists in a public setting (retail chain store locators), and offer it in a more usable format. I think public government data would fall under the same category, but if you're not adding value to the data in some way, you may have trouble attracting interest.
Let me know if you have any other questions.
Thank you for your answer.
Do you scrape the data? or do you use company's APIs to collect the data?
Typically it involves scraping, but if we are able to minimize our traffic to the site by using APIs, we will always try to do so. Scraping gets a bad reputation, and there are definitely wrong ways to do it, but if you scrape the right type of data and are respectful to the website, it's a great way to organize the web's mess of information. We focus on store location data, but I think the same techniques could be used in many other directions.
I've been recently thinking about this too, @anabayan. As you and @hostedmetrics mentioned, government data is not in the most usable form, so transforming it into a more usable product potentially adds a lot of value. I'm just not sure how much money is in it, you know? 10's of $$/month? 100's? 1,000's? 10's of $1,000's?!?
I have a buddy that has a Saleforce.com App that takes census, economic, and geolocation data by zip code (that he gets from government sources), aggregates it and makes it easy to use in Salesforce.com. Salesforce Apps have kind of a higher barrier to entry, though, since you have to go through their application process and code review process. But he does alright. I was wondering if something with a little more ease of entry might work, like putting and API on rapidapi.com.
In my last startup, I got data from the US Census site and formatted it into private api that our product consumed. I was thinking of tweaking that and putting it on rapidapi.com to test the waters. But I keep reminding myself, I'm NOT looking for a "cool idea"; I'm looking for an actual PROBLEM that people/business are having AND that I can create something to solve it.
I've heard of companies taking free raw govt weather data, packaging it into a usable form, and selling it back to the government.
Typically, when we demand something of our government, it will do the bare minimum to meet the requirement. So is the case with demanding open data. They will put it out there but they will not produce the products that people might want to use that data for.
As long as you respect the licensing, go use the available data!