I have seen a lot of posts regarding web scraping and data collection, be it leads, reviews or other data from web here in Indiehackers.
A lot of products exist for web scraping like Phantombuster, Simplescraper etc. But at the end these are products not services. Sometimes you need a customized scraper that does exactly what you need which may not be possible using these tools.
Would you prefer a productized web scraping service that is configured by professionals and does exactly what you want?
I would totally welcome it. I am looking to scrape to build my data sets and few things I found are close but not exactly.
I think this is a good idea! One way to control variability of effort required is by limiting it to popular sites like Amazon (for reviews), craigslist (for listings) etc. Since you will already have a script ready for these sites you can offer a good rate and make a healthy profit without spending too much time. :)
Well.. it can be interesting for some people but if I were in your shoes I wouldn't do it because it can't be Scalable.
I agree about the scaling thing but I do think people would be willing to pay more for a dedicated service.
You can combine both approach.
It is what apify is doing; They have productized API listing plus they have a some network of developer you can query for custom dev. So they are using the productized API as lead magned.
I'm doing something similar with Phantombuster. I'm managing a community of users and offering custom development for use case not supported by the official API store.
Yes I am familiar with Apify, they offer both. Nice to hear about a community of users looking for scraping and automation. How did you manage to build one?
It is quite simple: create a linkedin group with "Phantombuster" in the name. As it is the only group mentioning "Phantombuser" in the name I'm getting 5 to 10 request to join per week.
If you know other webscraping tools without groups it is a nice hack to implement ;)
There are a ton of companies that do this, and they are pretty successful.
Some examples on the first SERP:
Search for "data as a service +scraping" to find more.
Hi Sagun,
Can you give an example? Would this be specific to one industry?
No it won't be industry specific. Some of the use cases might be
And the list goes on. Basically a customized scraper as per your need that will run every day/week/month and get you the data. In return you pay a certain recurring fee just as you do for a product. Additionally you can request future minor changes, a human would be on the other side listening your request.