Scrape On Demand

Shorter the better, not always but in this case.

Two weeks ago I decided to take action without researching/overthinking. Today, I'm at the same point more confused than before knowing that I couldn't keep my promise.

In essense I'm trying to make money scraping the web.

Problem that I think exists;

There are thousands of online web scraping tools, not suited for everybody. Pay the price and do the work yourself. Starting around $49 monthly commitment for a very limited usage. It's great for those who makes it work, but I think there are edge cases.

To dig it further I've spent the last weeks on upwork to see the demand, it seems like there. Also on fiverr. However competing in these platforms is not for me for many reasons. It's a rat-race to the bottom, a client can easily destroy your profile and money flow. I also suck with clients and most importantly I couldn't land a single job the whole time. 🚀

So what I'm thinking is offering an on demand scraping service.

Dream scenerio would be to offer it as an unlimited scraping service as in DesignJoy offers "Unlimited product design as a service." I don't know how it works but I know I can't say "unlimited" for scraping because I saw this week on upwork, someone requested to scrape Yelp -entirely- for $40 or so. :) Nah bro, not gonna happen.

This is where things turns my brain into mashed potato.

First of google disagrees, not enough search volume. So SEO might not be an option and there is no target audience as far as I can see. Anyone can be a potential client. Someone looking for rent prices on SF to someone looking for private schools in India. (These were from upwork job posts.)

I kinda want to shape it as a weekly interactive newsletter thingy, where I gather requests from customers which they don't exists, yet. Like I can scrape the newly funded startups for leads, I can scrape product hunt upvoters for competitors, I can do some stuff around domains. I already sold these as digital products before but it's hard to create content around them so hard to market. This is me trying to bundle them all into one bucket. As it's obvious I need input to make it better. I need a customer to tell me "yo, you can scrape good performing tweets on this niche so we can change a few words and publish it."

As there is no search volume in google, I thought I can rely on compounding effect and showing up everyday. I can get "inspiration" from upwork, scrape those random sites and share one publicly to get attention, keep the rest for paying customers. That's my best plan so far. 🤯

One part of me wants to keep it so cheap so everyone can benefit while doing stuff with a white hat, whereas other part says who do you think you are. I'm not good with communicating so it's hard for me to handle many people at once. Also as scraping is in gray area, I wonder if I should make it go full black hat and keep it highly pricey and highly exclusive.

Funny or sad part about it, is my goal. All I'm trying to make is around $300-$500 to get my sanity back. I've been haunted by the feeling of not being able to make $10/day. That's annoying, very annoying.

IDK there has to more questions I should be asking but yeah that's basically it. I'm not even sure if there is a meaningful question. If you can make sense of this word salad. Please share your opinion about how dumb it is or how to make it work.