I can scrape that website for you
Hi everyone,
I’m Vishwas Batra, feel free to call me Vishwas.
By background and passion, I’m a full stack developer. Over time, project needs pushed me deeper into web scraping and I ended up genuinely enjoying it.
A bit of context
Like most people, I started with browser automation using tools like Playwright and Selenium. Then I moved on to crawlers with Scrapy. Today, my first approach is reverse engineering exposed backend APIs whenever possible.
I have successfully reverse engineered Amazon’s search API, Instagram’s profile API and DuckDuckGo’s /html endpoint to extract raw JSON data. This approach is far easier to parse than HTML and significantly more resource efficient compared to full browser automation.
That said, I’m also realistic. Not every website exposes usable API endpoints. In those cases, I fall back to traditional browser automation or crawler based solutions to meet business requirements.
If you ever need clean, structured spreadsheets filled with reliable data, I’m confident I can deliver. I charge nothing upfront and only ask for payment once the work is completed and approved.
How I approach a project
You clarify the data you need such as product name, company name, price, email and the target websites.
I audit the sites to identify exposed API endpoints. This usually takes around 30 minutes per typical website.
If an API is available, I use it. Otherwise, I choose between browser automation or crawlers depending on the site. I then share the scraping strategy, estimated infrastructure costs and total time required.
Once agreed, you provide a BRD or I create one myself, which I usually do as a best practice to stay within clear boundaries.
I build the scraper, often within the same day for simple to mid sized projects.
I scrape a 100 row sample and share it for review.
After approval, you provide credentials for your preferred proxy and infrastructure vendors. I can also recommend suitable vendors and plans if needed.
I run the full scrape and stop once the agreed volume is reached, for example 5000 products.
I hand over the data in CSV, Google Sheets and XLSX formats along with the scripts.
Once everything is approved, I request the due payment. For one off projects, we part ways professionally. If you like my work, we continue collaborating on future projects.
A clear win for both sides.
If this sounds useful, feel free to reach out via LinkedIn.
https://www.linkedin.com/in/vishwas-batra/
Very relevant — data scraping is often one of those hidden dependencies many early products need, especially when there’s no API or partners aren’t mature yet. It’s not just about pulling HTML — the quality of extracted structure and how quickly you can iterate on selectors really determines whether it becomes a tool or just a one-off script.
One pattern I’ve seen work well is surfaces + rules instead of big crawls — identify the few key data points you absolutely need for early validation and build lightweight extraction for just those, then expand as use-cases clarify. That avoids scraping becoming a maintenance sink early on.
Curious — are you focusing on scraping for one specific use case right now (e.g., lead data, pricing, reviews), or are you building a more general scraper that can be configured per target site? How you frame that usually affects usability and support friction.
The "API-first, browser automation as fallback" approach is the right order of operations. So many scrapers jump straight to Selenium/Playwright and wonder why their scripts break every few weeks or hit rate limits constantly.
The reverse engineering skill is underrated. Once you understand how sites structure their internal APIs, you realize most of them follow predictable patterns - pagination tokens, GraphQL queries, authenticated endpoints that return way more data than the public HTML.
One question on the business model: are you seeing more demand for one-time data pulls or ongoing scheduled scrapes? I'd imagine the recurring work is where the real value compounds - clients who need fresh data weekly/monthly and don't want to maintain infrastructure.
I also believe in recurring work but in reality often things are not that straightforward.
Apart from that, I've only seen traction from reddit yet, indie hacker is almost dead for literally any kind of post.
The "not straightforward" part is the real business insight. Clients often think they want one-time data but then realize their use case requires freshness - competitor pricing changes, lead lists go stale, market data shifts. The gap between "I want data" and "I want current data" is where recurring contracts live.
Interesting observation about Reddit vs IndieHackers traction. Reddit's volume is higher, but I've found IH conversations go deeper - fewer but more engaged responses. Different distribution, similar intent-to-quality ratio.
What types of scraping projects tend to convert best for you? I'd guess e-commerce price monitoring or B2B lead generation based on the skills you described, but curious if there's a niche that surprises people.