Growing Carrots: The Unexpected Impact of User Surveys

In my ongoing project, SingingCarrots, we use small surveys to learn more about our visitors. This is a key part of our customer research process. However, a recent finding has made me rethink our approach. 🤔

One of our surveys is designed to identify visitor personas and their intent. Are they potential students, teachers, or just casual browsers? This information helps us tailor their journey on our site, providing a personalized customer experience. 🎭

If they're teachers, we connect them with relevant information. If they're potential students, we ask if they're looking for a teacher, already have one, or prefer self-study. Based on their answers, we guide them to different pages on our site. 🚀

At first glance, this approach seems harmless. We're just learning about our customers and guiding them through our site. However, a recent analysis of our data in Mixpanel revealed something unexpected. 📊

I noticed that by customizing the path for self-study users, letting them explore resources on our site, and steering them away from the survey page, we actually decreased conversions. 🧩

This finding is ironic because these self-study users are our target audience. We develop tools specifically for them and try to customize their experience. But this customization seems to have made our products sell worse. 🎯

So, what's the lesson here? Measure everything, even the obvious. What seems like a good idea might not be. You need a metric to cover every change you make on your site. I didn't do that in this case, and it hurt our conversions. 💡

This experience has been a valuable learning opportunity. It's reminded me of the importance of data-driven decision-making, and the need to constantly question and test our assumptions. Even when we think we're doing what's best for our customers, the data may tell a different story.

This post was kindly prepared by ChatGPT fuelled with PromptTree. The original authentic thought is mine, but it is structured and formatted by AI to save me time for playing with my son. 🤖

If you prefer, you can also read this post as a Twitter thread.