July A/B Testing Results for Mindful Suite

As someone who has done hundreds of A/B tests for my ex-employers, I thought I knew the best practices by heart. So for a few months, I've been "optimizing" without A/B tests. And the result sucked. I lost 60% of installs over May and June. So I decided to do it right. And the results are promising so far.

App Store Images +10% in conversion

Theory 1: the font size is too small when displayed as thumbnails, conveying no information.
Theory 2: Adding featured by Apple might serve as social proof.

I replaced the first two screenshots with a large title and a fullscreen phone mockup. I also put the Featured by Apple badge. Over the next two weeks, the average store view to installation conversion increased to ~30% from the previous average at ~20%.

New Icon +30% in Conversion

The icon on the left is in use since May, coincides with the start of the drop I mentioned early. I like the icon, but as I scroll in the Play Store, I realized that the white background doesn't make it stand out. It's easy to overlook it. Though it's modern, it's not "delicious."

So I created a new icon mimicking the new MacOS's icon style with a dark background. This time, instead of replacing the current icon with it right away, I utilized the A/B testing tool in Play Console. (It's so easy to use. Why did I not use it, stupid me?)

The experiment lasted 6 days. The new icon helped get 30% more downloads versus the old one!

New Short Description on Play Store +20% in Conversion

Old
"A System for Success: Forming habits -> Reaching goals -> North Star"

New
"A dashboard for your goals and habits to become a better version of yourself"

One thing I screwed up badly was changing the app description in Play Store once a day. I used it as if it's my blog lol. I think that careless action caused Play Store to penalize my app rankings quite a bit as it thinks I am trying to game their system maliciously.

Again, I used the A/B testing tool in Play Console. Like I mentioned in the last post, I've just read 7 marketing books, and two of them taught me what positioning is. So I looked at the user feedback and came up with an obvious metaphor that the app is "a dashboard for your goals and habits."

The problem with the old short description "A System for Success: forming habits -> reaching goals -> North Star" is it's too abstract. If I were a user, I'd think, "What's a system?" "What does the -> symbol mean?" "What's the relationship between habits and goals?"

Who came up with that tagline? Well..it's myself!

So far, the new short description has brought 20% more installs than the old version.

No Free Trial +30% Revenue

I accidentally turned off the free trial feature in the last update. I added free trial feature in April to see if it would help with retention. It didn't. There was no change in retention before or after the free trial was introduced.

I only discovered that I broke it after some users emailed me back asking because they read about it in some emails. I granted them free trials and asked some questions with great responses.

Yet the magical thing was, the revenue went back up. I guess the lesson here is if you are interested in helping people, you have to help them make the purchase decision. You can only help them after they've decided to get your help. So don't be shy of selling if you're providing value. It's not about how you feel about asking for money. It's about them getting the benefit they need.