Learnings after 6 months of Mailbrew beta

After 6+ months of work we're finally launching Mailbrew in the wild. You can now go ahead and signup, and we're live on Product Hunt 🚀

Today I want to take this chance to share some of my learnings during this long beta phase. In fact we decided to ignore the "ship fast, iterate later" mantra and kept the product in private beta for months to iterate on it a lot based on early users feedback.

So here are some of the most important learnings for me and my co-founder during this long beta.

👨‍💻 You should manually onboard your first users

October 2019. For the first time since we started developing and launching our own products, we decided to manually onboard the first users. We've had this growing early access list (~800 people back then) and we picked a few of them that we knew personally, or that we felt would be a good fit for Mailbrew.

This was a big shift for us. In the past we would have just scraped together a fast launch plan, relying way too heavily on getting #1 on Product Hunt (we've all been there!) and hoped for the best. The product already "worked" back then, or so we thought, until we started onboarding users.

We asked these first selected early users to join us on a quick 30min call, so we could ask some questions and explain the product.

Well, I can't even think of not having the insights these calls gave us in the first few months of work.

Thanks to these calls:

• We basically realized that some of our core assumptions about the product were plain wrong.
• There were some huge gaps and issues with our UX that we simply couldn't see, because of all the things we were taking for granted having worked on the product for 2 months already.
• We had a much better understanding of the real problem we should try to solve.

Some tips for people who want to do these calls:

• If you set 30 minutes time slots, set a timer for 25 minutes before starting the call. It's super hard to just cut the call when the time ends, so leave a small buffer for ending them.
• When you ask to describe the product, or what they want to achieve with it, try your best to not put any words in the user mouth. You need to focus on what language they use to describe their problem, how they want to solve it, and how they see your product.
• Common questions like "How did you find out about us?" are pretty useless. Ask things you really want to know, and that can give you actionable insights.
• When you give them access to your product, ask them politely to share their screen, and take notes while you see how they interact with it.
• Remember to ask about pricing. You should ask what they think it's a fair price, and what is an expensive price that they would still be willing to pay. This last one is a good reference for your price point.

Mistakes we made:

• In hindsight, we should have scheduled just one call each day, maybe two. Instead there was this one time when we had 3 calls in a row and it was exhausting. Each call requires focus and attention so absolutely don't be overbooked.
• We scheduled these calls after 2 months of work on our first prototype. We felt (and feel) really strongly about the idea behind Mailbrew, so this boosted our confidence. In retrospect, it would have been safer to do some of these calls before writing a single line of code to better assess the problem and inform our decisions.

💸 Monetize early on

As soon as we finished working on our monetization, we launched it and put all our early users in a 30-day trial of our Pro plan, the one they were using for free until that day. This isn't the easiest thing to do: the product was still in its early stages, with only few features available.

Still, knowing if your users are willing to pay for your service and I mean, actually paying, not just saying they would pay is the single most important information you need.

Almost immediately after implementing it in early January, one of our early users jumped on the highest plan. This was a "Pro" plan that only had one feature actually developed, but Pat decided to jump onboard early on.

I think engaging our early users, asking for feedback, and pushing steady updates to the app showed a lot of commitment, and this helps in converting the first few users.

Mistakes we made:

• Our first paywall/pricing page was really well made, but it was too tailored for one pricing strategy. You might need to add plans, remove them, have a free plan or not and so on. So make sure to build things in a way that can accommodate this changes later. We basically had to rewrite monetization from scratch before launching.

📝 Understand your positioning

Since the start, we had various ideas of what this product was:

• A service to get automated emails from any sources.
• A way to follow your favorite topics.
• A tool to stay in touch with everything you need.
• A way to unplug from feeds and reduce FOMO.

These can all be true for us, but it's important to understand how the majority of users view your product, and what positioning resonates the most with users.

For example, in early February I've started sending a survey to our users asking them to complete some sentences.

The first one is completely open:

• Briefly describe Mailbrew, as you would do with a friend: ...

Then were the sentences to complete:

• Mailbrew is valuable to me because...
• The main problem Mailbrew solves is...
• What makes Mailbrew unique and different from other apps is...
• If I couldn't use Mailbrew anymore I...
• My main issue with Mailbrew is...
• My main feature request is...

This survey gave us fantastic insights on how our users perceived the product a few weeks after they joined the beta.

One response in particular really spoke to us, and also became an amazing testimonial:

"Mailbrew is the taming of the firehose that is today's online information landscape. For the first time, I feel like I can keep up with the things I love without getting completely overwhelmed or falling into infinite rabbit holes that the web is so good at getting us to fall into. Mailbrew is an information diet done right. It forces you to consider only that information which is essential to you and then delivers it seamlessly and beautifully. No more having to check multiple sites. No more hunting for interesting things—it’s all right there, right in my inbox. It will save me heaps of time this year."

See how far this is from "Mailbrew is a service to get automated emails from any sources".

This helped us learn what the product really is, what actually problem it solves and how we can explain it. This informed a lot of design decisions for our landing page.

📈 Define your key metrics, and stick to them

I have a personal tendency to get attached to some metrics and over-analyze them. I like watching numbers grow, spotting trends, and calculate the path to profitability.

By while monitoring your metrics can be fun, many of the numbers about your product and its users are meaningless and even distracting. So it's super important, especially in a beta phase, to define KPIs that are essential to your product success, and keep you eyes on those.

After some iteration and some reading, we defined a system of metrics that help us keep our eyes on the ball. We used Mixpanel recommended system and decided to define a Focus Metric that is our north star, Level 1 Metrics and are essential to know the product is growing and it healthy, and Level 2 Metrics which are important, but secondary.

Focus Metric

• Paying subscribers

Level 1 Metrics

• MRR ($)
• Net new subscribers
• Cancelations
• Activation
• Open Rate
• WAU
• All Users Retention

Level 2 Metrics

• Signups Breakdown
• Free vs. Paid Users
• Conversion to Pay
• Free Users Retention
• Subscription Cancels (brekdown by reason)
• Trials Ending

This is not a perfect system but it's helping us to have a great overview of how the product is performing, and spot any criticality.

Mistakes we've made

• For a lot of time we didn't have this system, especially for other products, and we thought it's enough to know your conversion, MAU and other easily readable things to know if your products are ok. Well, after finally choosing to use a tool like Mixpanel and start tracking more metrics, our product work improved as a consequence.

🔥 There are no hacks

This private beta also validated something we always thought true, at least for us: that there are no hacks. You need to relentlessly focus on building a great product, all the rest comes later.

By doing the manual onboardings, putting serious thought into positioning, and really focusing on providing value to the users and not develop anything that isn't essential, we feel like users naturally value the product

And that's why even before launching we sold a few yearly licenses, without pushing them particularly. This was probably the biggest validation and made us feel like we were on the right path.

We're now launching with $150 MRR, which might not seem a lot, but it's more than $0 and it shows how things are starting to work.

Thanks to this strategy, we're now putting our app out there in a state that we're really happy about, and we're sure that new user will at least understand what it is about, and have a good enough experience.

For example, one of the latest additions was an onboarding that presents the user with a few automated newsletters already prepared by us. This is because in the beta we initially saw a very low activation rate (Signup → Read the first issues of a newsletter). Imagine launching without realizing this and missing out on many users that could appreciate your product but are just not onboarded properly.

---

This is it. Of course there are many more lessons to learn, and I really want to thank you all in this community. Seriously, it's been an incredible source of inspiration and insights during this journey, and also while building other products in the past.

If you want to check out the product itself it's here.

Cheers!