Week #3 - Beta test is a game of Lemmings

This reference is going to give away my age, but have you played Lemmings?

The 1991 video game’s — available on floppy disk or here — purpose was for you to lead a group of walking lemmings to an exit area. They would just walk in a straight line, and it was up to you to lead without losing any off a cliff, in a pool of magma.

SaaS beta-test feels the same: Lemmings enter your world (website), and it’s up to you to lead them to the exit door (payment).

This week at Poked, we played Lemmings!

Lemmings

1. Find where your lemmings are stuck

They are 2 ways of finding where your users are stuck: Talk to them or track them on your app. I think both are complementary, but we’ll focus on the tracking part today.

François set up Splitbee to track page views, and, in my opinion, it was a perfect choice for the little indie app that’s Poked because it’s simple and light.

They are tons of things we could have improved: Adding clicks, forms, events on the Chrome extension download, etc. But keeping things light allows us to reap most of the benefits of tracking, without wasting dev time on screens that’ll have change in a week.

Here is what my first funnel looked like:

Funnel

The steps are:

Signup. Users enter their email and password.
Onboarding. New users download the browser extension Poked require.
Contacts. The extension is installed, users have successfully synced their LinkedIn profile, and finally get to see their CRM.
Inbox. Users browse to the inbox.
Archive. Users have digged a bit and browsed the Archive.

Yes, it’s simple, yes, it’s flawed, but as you can see, they are immediate learning from doing this.

2. Investigate the problem

In our example, you can see we managed to bring 170 people to the /signup page, but lost 2/3 of them on the next step: THIS IS BAD. VERY BAD.

Because of the over-simplistic data tracking, we don’t know where users drop. Do they:

A. Click on “Get started” but are not ready to sign up yet?
B. Sign up on mobile, but are unable to download a Chrome extension?
C. Are on desktop, but unwilling to download a Chrome extension?
D. Are on desktop, but on another browser, thus unable to use Poked?

After further enquiry, I found that out of all the lost users, the reasons were

A. Maybe!
B. 35% of lost users were on mobile.
C. Maybe!
D. 11% of signed up users were on desktop but not on Chrome.

This data is interesting. To what action should it lead, if any?

3. Take action… ?

According to the above numbers, here is my thinking:

A. Hard to estimate; Perhaps, we need to better sell the app on the marketing website, add testimonials and reassurance on the “Sign up” page. This is a permanent thing to do, so I wouldn’t make it my priority.
B. I guess we could display a special screen for mobile users. Ask them to come back again on desktop. Track them somehow to re-engage them later. That’s something we’ll do anyway. Not a priority either.
C. Chrome extensions are an extra step, therefore create friction. Poked requires a Chrome extension. We could improve our copy to get a few more people to try, but this is too small. The beta is about major improvements: Not a priority.
D. We could sprinkle a few more Chrome logos to prevent non-Chrome users… But that would take the risk of losing a few users’ email addresses that we could also re-engage later. Not a priority either.

Therefore, this issue is not the one on which my time is best spent. Let’s do this check again for the second biggest drop: From Onboarding to the CRM.

Take-aways

Did you just read a blog post about some issue I willingly didn’t fix? Yes, you did!

But my point is this: Building apps is messy. You can’t be a perfectionist about every little thing. And picking your battle is important.

This investigation actually lead us to work further down the funnel: Some slow requests and a key missing feature were totally blocking users from accomplishing some basic tasks.

At our stage, focus should be put on the most hardcore users, the ones that feel the pain and really want you to fix it for them.

Hope you liked this post. Next week, I’ll explain how we made our first customers with a Duke Nukem 3D analogy 🤘

Try Poked, the LinkedIn CRM for solopreneurs and follow me on Twitter