Web Scraping API

No Employees
Multiple Founders
Founders Code

My co-founder and I worked on Web Scraping projects for many years, for startups and bigger companies. We realized we always encountered the same problems. We decided to create an API to solve these problems.

December 3, 2020 Open report November - $200k ARR

โœ… $200k ARR

๐Ÿ’ฐ MRR: $17,549 (+51%) ๐Ÿคฏ
๐Ÿ’ต Revenue: $21,129 (+69%) ๐Ÿ˜ฑ
๐Ÿ˜€ Customer Count: 185 (+ 20%)
๐Ÿ’” Churn: 13 (vs 20)
๐Ÿ‘‹ New Trial: 950 (vs 840 )
๐Ÿ”Ž Visitors: 41k (+24%)

Here is what @Daolf and I did this month ๐Ÿ‘‡

First of all, we reached $200k ARR. ๐Ÿ”ฅ

The next stop is at $300k ARR.

At this stage, we will be able to pay ourselves better salaries than what we were paid at our full-time jobs two years ago ๐ŸŽ‰

The revenue growth was insane ๐Ÿ˜ฑ, it is by far our best month ๐Ÿš€

We got "lucky" by signing several big customers on enterprise subscriptions in the same month.

๐Ÿ“ˆOur content is still ranking better and better every month.

โœ๏ธIn terms of content, we spent a lot of effort on a piece about Web Scraping with Charles proxy:

We got a lot of shares and backlinks from this one, but the keywords are not that easy, hope it will rank ๐Ÿคž

I recently started to answer some HARO queries:

Here is what my inbox looks like now ๐Ÿ˜ฑ

It takes a LOT of time to filter out the noise, but we got featured on a few media outlet/company blogs.

Any tips on making this more effective?

About the product, @Daolf worked on scaling our infrastructure and fixed a lot of things with our hosting provider.

It can be a bit frustrating on this part, not having the time to do as much as we would like to. ๐Ÿคทโ€โ™‚๏ธ

But it should change!

Support was sometimes insane:

๐Ÿ’ต We also increased our salaries to 2500โ‚ฌ/mo

It feels great ๐Ÿ‘

And that's it.

See you next month, happy scraping ๐Ÿ :)

October 7, 2020 Open Report September - $9,854 MRR ๐Ÿ“ˆ

๐Ÿ’ฐ MRR: $9,854 (+10%)
๐Ÿ’ต Revenue: $11,412 (+26%)
๐Ÿ˜ŠCustomer Count: 143 (vs 127)
๐Ÿ’”Churn: 15 (vs 7)
๐Ÿ‘‹ New Trial: ~800 (vs 600)
๐Ÿ”Ž Visitors: 33k (+13%)

Stats are looking good and for the first time, we crossed $10k revenue in one month.

We hope to "MRR" this in October.

We also had a lot of churns but I'm not worried.

I reached out to everyone asking them why they churned, and almost everyone told me something like this:

"Good product, but I don't need it anymore"

BTW, here is a template that works well for requesting churn feedback (75% reply rate):

Regarding the product, we launched the Google Search API:

And several new features to our web-scraping API, such as the ability to easily scroll on a web page.

We have also published a very nice PHP tutorial and already featured in the Symfony newsletter ๐Ÿ˜Ž.

We also improved our marketing website following Lianna Patch's advice.

Being part of TinySeed allowed us to have a private review of our landing page and Lianna gave us great insights Folded hands.

Those videos helped a lot too:

This month felt a bit like the end of the "Phase 1" for the ๐Ÿ.

We have managed to make the product do almost everything we wanted to do 15 mo ago.

We also deployed a content strategy, designed in January, that worked very well but we feel it's time to move on and try smth new.

We have also decided to develop a new product, built upon the
ScrapingBee core web-scraping engine but targeting people who don't know how to code.

Stay tuned.


September 2, 2020 Open Report August - $8890 MRR ๐Ÿ“ˆ

August Open Report for @ScrapingBee

๐Ÿ’ฐ MRR: $8890 (vs $7875 )
๐Ÿ’ต Revenue: $9200 (vs $8290)
๐Ÿค New Customers: 31 (vs 18)
๐Ÿ˜€ Customer Count: 127 (vs 103)
๐Ÿ’” Churn: 7 (vs 7)
๐Ÿ‘‹ New Trial: 634 (vs 651)
๐Ÿ”Ž Visitors: 28,800

Here is what @Daolf and I did this month:

Our content is ranking better and better on Google, more traffic, more sign-ups, and more paying customers ๐ŸŽ‰ So even the traffic increase is not as impressive as last month, the content strategy is still working well.

This month we didn't have a lot of requests for enterprise plans (>$99/mo), and since we almost rely exclusively on inbound traffic, I guess we didn't have much luck with that.

We've tested different ad channels:

Facebook ads retargeting ๐Ÿ˜ - It could work but the cost per sign up is high. We could probably tweak the copy/creative to improve it.

Reddit ads ๐Ÿ‘Ž We didn't find how to target people interested by web scraping. Didn't convert AT ALL
Google Display Network retargeting ๐Ÿ‘Ž - The CPC was really high and conversion rates really low.

Reddit Ads ๐Ÿ‘Ž- CPC was low, but the traffic didn't convert. The targeting options are limited on Reddit. You cannot target every sub, and the interesting subs for us have very low impressions.

The next step is to test Twitter / Linkedin. ๐Ÿ”ฅ

Ideally, we'd like to leverage our existing traffic to retarget people.

Regarding content marketing, we are working on lot of content but we didn't publish anything this month.

Two articles were updated, like this one:

๐Ÿ‘‹ Last month we had a HUGE problem of sign-ups abuse (people signing up with disposable email addresses ) and it seems to be solved.

We used a third-party API to detect those emails.

@Daolf added Google Auth and Github Auth to our sign up page... and 50% of new sign-ups are now using either Google or Github Oauth ๐Ÿ˜ฑ

It also increased our conversion rate on the signup page ๐ŸŽ‰

We are soon going to work on our copywriting, both on the landing page and email campaign. I think it is even more important because we are not native English speakers.

Regarding the next marketing steps, there are still too many things we want to try so we need to focus... ๐Ÿ˜€

๐Ÿ On the product side, we didn't do a lot this month. But lots of things are coming!

And that's it, thanks for reading this far.


Kevin & Pierre

August 26, 2020 Reaching $100,000k ARR. A highly symbolic goal ๐Ÿ‘‡

After 14 months ScrapingBe is now a 6-figures company ๐Ÿ˜Š

50% of $200k ARR.

A highly symbolic goal for me.

When Kevin and I left our full-time job, 2y ago, we were making approximately $50k per year (๐Ÿ‡ซ๐Ÿ‡ท salaries).

So with taxes, we were costing our companies ~ $100k per year each.

So assuming they were not losing money on us, we were "creating" around $100k per year of "value"

Reaching $200k ARR means that with ScrapingBee we're creating as much value as when we were employed.

I'm glad we're halfway to that point, it means we've made up for 1 full-time salary.

Looking forward to completing that goal, and then go as far as possible. ๐Ÿคž

August 18, 2020 โฑ Setting up our first attribution model in 1 min

At ScrapingBee we wanted to know what were the marketing channels that brought the most ๐Ÿ’ฐ.

GA makes it very easy to know where do your visitors/users come from.

But when it comes to customers, and their values, it is harder.

Here is our very simple attribution system that takes 1 minute โฑ to integrate

First, store in a cookie UTM tags and referrer.

It's important to set up the domain of the cookie if you use a subdomain for your app. And it's also important to set the expiration date otherwise the cookie would disappear on each session.

Here is the snippet we added on every page of our marketing website.

Then, on your signup pages, add those 2 things:

hidden fields for the info you want

this snippet that will populate those fields with your cookie ๐Ÿ‘‡

Store those in your DB, subscription analytics and that's it!

It's not perfect, but it takes 5 minutes to implement and will help us a lot with understanding what marketing channels are worth focusing on ๐Ÿ“ˆ.

Curious to know how you do it.

August 6, 2020 Open Report July: 1๏ธโƒฃ0๏ธโƒฃ0๏ธโƒฃ customers edition ๐Ÿคฏ

๐Ÿ’ฐ MRR: $7875 +24%
๐Ÿ’ต: $8290 +5%
๐Ÿค: 18
๐Ÿ˜Š: 103
๐Ÿ’”: 7 (vs 11)
๐Ÿ‘‹: 651 (vs 287) <- Pirate flagHeavily inflated
๐Ÿ”Ž: 24,800 +35% Rocket

Firstly, this month we crossed the 100 customers mark.

When we built our first SAAS, at the top, we only managed to have 6 customers at the same time.

The number was so small that we could name all our customers on the top of our head.

We can't do that now ๐Ÿ˜Š

๐Ÿ“ˆ On the marketing side of things, our content marketing strategy continues to pay off with a 37% MoM increase in organic traffic.

And we have multiplied by 3๏ธโƒฃ our traffic from this channel since the beginning of the year.

We prefer quality over quantity regarding our blog posts.

Hence why we have only posted 2 in July.

One comparing all the free proxy tools available, with a detailed benchmark for each:

And another, written by Ari, explaining in detail how to use Scrapy with JavaScript.

A technical subject that could only be written by an experienced developer.

The same Ari developed our Scrapy integration, now available on Github Down pointing backhand index

We've also tried new things.

Like this tweet:

It was sponsored, cost us $500, and brought us a whopping 1๏ธโƒฃ customer.

So I guess we'll break even in about 18 months ๐Ÿ˜‚.

๐Ÿ‘‹ Regarding signups, we had at least hundreds of spam account looking to take advantage of our free plan.

It is annoying as it's really mess up with all our statistics/reporting but we might have found a solution.

๐Ÿ Regarding the product, we haven't pushed a lot of features.

We're currently not in our most productive phase.

We're trying a lot of new things here and there and probably need to organize ourselves a bit better ๐Ÿค“.

For example, regarding marketing, there are so many opportunities out there that it's hard to keep focus.

We also need to clearly decide what should be the next step for ScrapingBee product-wise and execute ๐Ÿ—

Until next month.

July 24, 2020 ๐Ÿคฏ 100 Customers in over 12 months

It took us more than one year but ScrapingBee has finally reached 100 recurring customers.

1 year ago, I never thought that this would be possible. With our previous project, PricingBot, the best we achieved was 6 customers after nine months.

And now, every day, we have more than 3 people paying monthly to use something we built.

We probably could have reached that milestone sooner, but in February we got rid of the $9 plan for several reasons.

We would like to thank the IH community for being so awesome and supportive since day one.

Now, let's reach 200!

Happy Scraping.

July 3, 2020 Open report June: ๐Ÿœ profitability edition ๐Ÿ“ข

1y anniversary ๐ŸŽ‚ and ๐Ÿœ profitability Edition

๐Ÿ’ฐ MRR: $6358 +27%
๐Ÿ’ต Revenue: $7877 +54%
๐Ÿค New Customers: 22 +46%
๐Ÿ˜Ž Customer Count: 94 +12%
๐Ÿ’” Churn: 11 +100%
๐Ÿ‘‹New Trial: 287 +11%
๐Ÿ”Ž Visitors: 18,200 +30%

So, what did we do this month ๐Ÿ‘‡

The huge milestone this month to reach ramen profitability.

Reaching $5k MRR means that we can:

  • pay 2 minimum wages (1200โ‚ฌ)
  • taxes
  • servers/tools etc

And not lose money.

Tbh, since we raised money with TinySee we pay ourselves more (2kโ‚ฌ).

ScrapingBee is also 1 year old this month and seeing it still alive after 12 months is a victory in itself โœŒ๏ธ.

Especially considering that our last SAAS, PricingBot, only lasted 9 months.

This month we got lucky with revenue expansion. 6 big customers increased their usage.

Kevin's SEO efforts are also still paying dividends +30% MoM organic traffic Clapping hands sign.

We published 2 great pieces of content this month written by ๐Ÿ‘ people.

A detailed guide about web scraping in JavaScript, and a detailed guide about web scraping in Ruby.

The later was โœ๏ธ by the talented Sylwia Vargas and was viewed by > 2400 people!

On the product side, we improved our custom routing algorithm and cut our error rate by half.

We also released a new version of our Google API, cheaper and more reliable.

I got used to stumbling across shitty copycat those last few months.

But 2 weeks ago I found one that takes the cake.

Like the others, they copy-pasted copywriting, doc, etc.

But they also took entire paragraphs of our blog posts and even stole our cover image. Funny thing is that this guy is also on IndieHacker and sharing his stolen content with 0 shame.

Our road map for next month is simple:

  • double down on content.
  • improving the speed and success rate of the API

We will also try some new things during the summer.

Kevin and I love to play with new tools and we have several ideas we want to try out

Until next month ๐Ÿ‘‹

June 2, 2020 Open report May: $4937 MRR ๐ŸŽ‰

๐Ÿ”Ž Visitors: 13,900 (+49%)
๐Ÿ‘‹ New Users: 273 (+ 35%)
๐Ÿค New Customers: 15 (+140%)
๐Ÿ’” Churn: 5 (-54%)
๐Ÿ’ฐ MRR: $4937 (+22%)
๐Ÿ’ต Revenue: $5100 (+7%)

What did we do this month?

As we mentioned in previous open reports, we are really focused on SEO.

What this means is that we want to write more quality content on different topics around web scraping / extracting data from the web.

Pierre has a strong Python background, and I have a Java background. The thing is, we wanted to write about web scraping with R, Node.js , Rubyโ€ฆ

So in May, we decided to hire our first freelancers to work on technical content.

We posted a listing onย dev.toย , asked our friends / Linkedin / Twitter, and we were surprised by the number of emails we received.

๐Ÿค After a few Zoom interviews, we hired three different content writers, and we are really happy about the content they wrote. Time will tell if these blog posts perform well, both in terms of traffic and conversion. Here is an example of a blog post:

The other big milestone this month was the migration of our infrastructure from Heroku to CleverCloud.

Heroku is great and saved us a lot of time in the beginning. But there are some annoying technical limitations, for example, the HTTP requests have a 30 seconds timeout on the Heroku router, and we needed to increase this timeout to at least 60 seconds.

๐Ÿ‡ซ๐Ÿ‡ท CleverCloud is a French company, and itโ€™s like the French Heroku. Their support is great, itโ€™s flexible, and the price is much more competitive than Heroku. The setup is very Heroku-like, Pierre did this alone in a few days ๐Ÿ’ช

In terms of metrics, we made a lot of SEO progress this month, old articles rank better, and new ones are starting to rank too so thatโ€™s great. The sign up also increased significantly, which means that the visitors weโ€™re bringing to the blog are converting.

About the trial-to-paying conversion rate, we were really worried in April because it was really low (~2.4%), this month is more than double this number at (~5.4%).

We donโ€™t really explain these numbers, is it Covid-19 related, did we broke somethingโ€ฆ?

In March 1st we increased our minimum monthly pricing from $9/mo to $29/mo, and we didnโ€™t see any effect on the conversion rate. Itโ€™s possible that the effect was delayed since our average sales cycle is 18 days. ๐Ÿคทโ€โ™‚๏ธ

There will be some exciting things coming in June, new content, new proxy offer, and maybe a new productโ€ฆ ๐Ÿ™ˆ Stay tuned!

May 5, 2020 Open report April: We are joining TinySeed ๐ŸŽ‰

Disclaimer: we forgot to publish the March open report. The variation you are seeing below a computed from March.

๐ŸŽ‰ Seed Round: 1
๐Ÿ”Ž Visitors: 9,302 (- 25%)
๐Ÿ‘‹ New Users: 202 (+ 5%)
๐Ÿค New Customers: 5 (-75%)
๐Ÿ’” Churn: 11 (-45%)
๐Ÿ’ฐ MRR: $4038 (-10%)
๐Ÿ’ต Revenue: $4759 (-6%)

First, let's talk about TinySeed ๐ŸŒพ.

We are thrilled to join TinySeed batch 2 and to spend the year with such good company. With ScrapingBee the goal has always been to build a strong, sustainable (understand profitable) mid-size company. This vision does not align very well with traditional VC but it does with those new kinds of funds such as TinySeed.

(If you want to learn more about this new way of doing venture capitalism, you can read this nice article:

I'll probably go deeper about the how, why, and what of that fundraising in a future blog post but basically it all comes down to two things.

๐Ÿ’ฐ First, this fundraising multiplied by 4 our runway. Meaning that we are now managing the company with much more serenity and less stress than 3 months ago. For example this month, we did -10% MRR, I know a few months ago this fact would have taken a big toll on my sleep quality.

Now we can take our time and think mid-term/long-term. The peace of mind that comes with this fundraising is invaluable to us.

๐Ÿ˜ฎ Then there is loneliness. Since we've began our Indie Hacking journey 18 months ago it's been hard to find external help. Sure you can always ask for some advice about very particular topic here and there, IH being a cool place to do that, but with TinySeed this is different as you are now part of a group of people strongly incentivized in your success (investors). Basically we:

  • have access to top-notch mentorship. If we have trouble, a question, we just ask to be put in touch with the relevant mentor and boom, we can have a 30 minutes / 1 hours private mentorship class.
  • are part of a batch of eleven other companies having the same problems as us. Being part of this smaller community is really great. The Slack channel is the perfect place to ask advice, give advice, or just hang around talking about the weather or your weekend plans.

That's is about the TinySeed ๐ŸŒพ, let's now talk about the ๐Ÿ.

Basically everything looks good beside trial to paid conversion rate. This one as completely plummeted in April. The simplest explanation is COVID19 and we know for sure that at least 3 "would be" big customers have not paid because of this.

We'll see in May how things go.

Product-wise March was a "tidy-up" month. The initial version of ScrapingBee has been built in less than 6 weeks and of course, some parts of the product were far from being perfect.

Before going further in the development of the API we felt it was a good time to clean things up a bit, and so we did:

  • refactoring the whole core API
  • tests, tests, tests, and a bit more tests
  • brand new documentation
  • brand new knowledge base
  • request builder in seven languages
  • usage graph
  • new marketing emails flow

We've also decided about our next 4-6 months marketing strategy and it feels good to have a precise vision of what we want to do. This I can't share right now for obvious reasons, but soon enough I will ๐Ÿ˜‰.

Next month we will go back improving and adding features to our products.

And that's it,

Happy Scraping

My co-founder and I worked on Web Scraping projects for many years, for startups and bigger companies. We realized we always encountered the same problems. We decided to create an API to solve these problems.