Web Scrapers
Run, Create and Share Web Scrapers Online
It took 2 weeks, but I'm finally done building the minimum viable product for Web Scrapers. I worked mostly nights and weekends, and I coded the whole thing in ReactJS since that's that I'm the most comfortable with. In hindsight, there were a few features I built that probably could've waited until later. But I'm excited to start talking to customers and see what they think about the product so far!
Since I have experience in web scraping and I have this disturbing problem by my local machine. The problems are I can't run my scrapers at the same time and I need to find the folders of the source code everytime I want to scrape something.
So I decided why not I create an app where I can code, save and share web scrapers online without any hassle of a local machine...
-
1
Hey Robert! Just stumbled upon your MVP, and as a Python & Automation Developer with 5+ years of experience in web scraping, this hits close to home. Running multiple heavy scrapers on a local machine is a massive pain point for everyone in this space.
Since you built the frontend in ReactJS, you've got a clean UI ready. But to scale this from $0/mo, you’ll need robust cloud-based background workers to handle concurrent scraping tasks without crashing the server.
Are you planning to integrate headless browser clusters (like Puppeteer/Playwright) or proxy-rotation pipelines in the backend? If you ever want to brainstorm how to optimize the scraping architecture or connect your React frontend to powerful Python scraping APIs, drop me a DM.
Love the initiative, upvoted! Let’s connect.
Comment