Ask IH: Why Does the Smiling People A/B Test Not Work for Us?

We're developers at heart, and marketing doesn’t come naturally to us. We’ve read multiple articles on the positive effects of placing smiling people on pages. So we decided to try this out.

Our product Bugfender is available for several platforms, with our primary focus on iOS and Android. So we thought we’d A/B test our original platform pages with a version of a smiling person.

We ran that test at a 50/50 ratio for 10 weeks, tracked the number page views and the number of clicks through to the the signup form from each version.

Both versions were served on the same url, but for the purpose of this demonstration, we’ve separated them out, and you can view them here:

• https://bugfender.com/platforms/ios
• https://bugfender.com/platforms/ios-smiling
• https://bugfender.com/platforms/android
• https://bugfender.com/platforms/android-smiling

Based on what we’d read, we expected to see some sort of increase (however small) in sign-up clicks when coming from the smiling versions. However the result we had we very different:

• Android smiling CTR: 17.49%
• Android normal CTR: 21.29%
• iOS smiling CTR: 21.74%
• iOS normal CTR: 31.43%

Percentage of users who clicked through to sign up after viewing each of the platforms page

Over the course of 10 weeks, pages served with a smiling person always performed worse, averaging 5% for Android and 10% for iOS.

We understand that there is no special formula that works for all, but we didn’t expect this, and we were wondering if anyone may have any insights or ideas as to why this might be?

Could it have been the extended content? Or that the picture was too generic? Did it perhaps distract from the real data about the product? These are some of our thoughts, but as marketing is not our forte, we really have no idea and this is just speculation.

We’re always keen to try out A/B test and experiments, so we’d love to hear any ideas or suggestions you have too.

Thanks for taking the time to read this :)