Am I doing user research wrong?
by Jonathan CaiTLDR; I got 10 waitlist signups yesterday, personally emailed 8 of them, and only got 1 response so far. Would appreciate some thoughts about my process of validating this project and doing user research..
After my app got approved by Google, and users could safely use the live demo of my app, I posted here on IH and on Reddit.
My landing page has a link to a waitlist sign up, which asks two questions:
-
What's your email?
-
What problem are you trying to solve with our product?
Then I get notified when I get responses, and then I handcraft an email to everyone who signs up.
The email looks like:
Header: "magic: thanks for the interest!"
Body:
"""
Hello friend!
Jonathan here, building magic. Thanks for the interest in our simpler GA dashboard!
You mentioned your problem "getting simple analytics for less tech-savvy managers".
Does our live demo solve your problem to some extent?
Cheers!
Jonathan
"""
My goal was to make a small ask, for feedback on our live demo, to kick off a conversation, so that I could get to know the person and their needs better.
I felt like asking them for a 15 minute call was too much up front.
I got 10 signups. I've sent emails to 6 so far, and only got 1 response so far.
Although it's been less than a day, I am worried that I am doing this process of user research or problem validation wrong....
Am I?? Or am I just paranoid haha
I even just read "The Mom Test", so I feel like I should be better at this now š
-
8
Hey Jonathan š
Firstly, I really like your landing page. Its clean, simple (so reflects the nature of your product) and I totally got what it is your product does. And the live demo ā what a great way to show what it is your product does š.
As to your question. Give it time. As youāve mentioned, itās been less than a day. See how things are after a week and then decide whether you need to change your approach.
Here are a few thoughts I did have about your approach so far:
-
I like that youāre asking about the user problem in the sign up. Itās a good way to get a high level idea of the problems your users have.
-
Your follow up question (āDoes our live demo solve your problemā¦ā) is a closed question, so this may limit the amount of feedback you get. It's also focused more on your solution, not their problem.
-
I think youād get a lot more from a 15 minute chat than an email response and I donāt think itās too much to ask up-front. Afterall, you're trying to help them. The worst they can do is say no. You could include a follow-up question in your typeform to see if they'd be interested in having a chat. Iād make sure you phrase it so it focuses on them and their problem though.
I'd be interested to know how it all goes, so definitely post an update!
-
1
Just thought about this:
If they're not really willing to respond to my emails, the problem that they're having isn't urgent enough, and maybe my product isn't solving an important enough product? That is a form of validation itself?
-
1
Thanks for the feedback! I'll be sure to post an update later.
Re #2, I was also worried about making it a closed question, but I really wanted them to try the live demo first, and it gives them something concrete to go off of, easier to respond to. You're right though, it does focus on my solution more than their problem, so maybe it deserves some editing.
Re #3, Yeah that makes sense. I guess I'm just thinking about it from my perspective, if I was a customer, I'd rather have a few email exchanges before I hop on a call.
-
-
6
Hey Jonathan, 1 out of 10 is normal, it's hard to get people interested.
I found another method that works better: find people on IH who are in the market for your product, and reach out to them directly (email or twitter is available on their profile). Then offer them to do a 30 minute call where you give them 15min feedback on their product, and they give you 15min on yours.
-
4
Yeah, I did a bit of outbound, but definitely could do more! This idea of exchanging 1on1 feedback is actually http://pheedback.co/! Excited to see how @janel and @pabloheredia are going to build it out!
-
1
looks great, I signed up. Thanks for the heads up.
-
-
-
3
If I got an email like that from something I signed up for, I'd likely also not reply, because by the time I got the email, I probably won't remember what was in the demo. If I even looked at the demo at all. (May be I just skimmed the landing page and thought some part of it was interesting enough to sign up).
I think a better email would be, to highlight some specific part of your product, e.g 'You entered the problem X. I think our product can do Y, Z, and A that can help you solve that: <link to screenshot or video demo>. What are your thoughts?'
That reduces the effort on the user's part quite a lot, so they'd be more likely to reply.
Note: This is just my intuition, I have yet to email my waiting list, so I'm not speaking from experience :P.
-
2
Ahh that is actually a good point I had not considered.
I had assumed that people that would put in the effort to sign up would have at least tried the demo, but the way you phrase it makes sense.
I think another point might be, if they're not really willing to respond to my emails, the problem that they're having isn't urgent enough, and maybe my product isn't solving an important enough product? That is a form of validation itself?
-
2
Yeah that could also be a reason for sure. Or may be something was a need at one point but it no longer is. It could also be that you were asking them to make too much effort (review demo, think about it intelligently, write an email) and haven't yet demonstrated that you could solve their need.
If I were you I'd wait a few days, then send a personalized email with the template I posted in my last message. If they still don't reply, that'd be a better indicator of how strong the need is, since you're eliminating the variable of 'this needs too much effort'.
-
3
Sounds like a plan. Thanks so much for thinking along with me.
-
-
-
-
2
This is a totally normal conversion rate for an email like that. Good even. Keep going.
-
2
You should find a different name, it's just not right calling a minimal dashboard design for google analytics "magic".
-
2
Well youāre one to be talking about names..
And yes, if my plan was just to stop at the dashboard design, I would have copped the domain āsimple-ga-dashboardā and called it a day. But I am considering building more integrations (stripe, mail chimp) and features like easy email reports. Iād rather go with a name that conveys ease/simplicity.
-
-
2
Hey @jonathancai
It think you are missing out on an opportunity in your typeform to get an understanding of whether they would like to follow up with you.
The user maybe thinking that this is form is enough feedback and any
maybe...
- What's your email?
- What problem are you trying to solve with our product?
- Would you be interested in talking further etc (checkbox)
That way you get an understanding of how open they are to talk further. Also there may be an opportunity to upgrade to a call because you know they are open to talking further.
Also somethings to think about
- Maybe they didn't do the demo yet
- Is there anyway you can tell if they opened the email (open rate etc)
Just some thoughts hope you find them useful. I tried out the live demo and it seems like a good concept, especially for anyone that wants to view stats on their sites at a glance.
-
2
Thanks for the input!!
Yeah, I think based on yours and other comments, I will edit my typeform to include a question that gauges this.
Although I'm not keen on tracking email opens. I'd rather just send it without worrying about that.
-
1
I think thatās pretty normal. That being said, you can tweak your email a bit and that may slightly improve the response rate:
- Your email totally looks automated. A lot of people Iāve spoken with almost never respond to automated emails. So I try to make them feel more personalized. For example, I include their first name in the subject and first line, and I tweak the wording of the use case they entered a bit.
- Lots of people prefer a short call over answering emails. Add a 15min calendly link at the end and say sth like āIn case you prefer a quick 15 call over emails, you can select a time that works for you hereā
- I usually end it with mentioning the value both of us will get out of it: āWeāre currently deciding on what features to build over the next months and your input would help us make sure weāre building what you need. ā
Keep it up š
-
1
I did answer, which means the email was timely and I knew what to answer!
-
1
I did answer. Which means I saw nothing wrong here :)
-
1
Jonathan, Your response e-mail simply states the obvious. Yes you are just repeating what they already know. Also you are assuming they did live demo. Something like thanks for signing up. (hint they are not looking for magic!) Does it look like the app can solve the problem of simplifying analytics? Are you clear on how this app will save you time and make analytics a much less stressful task? If so great. If not would you like to have a conversation regarding how to use the app to get the results you truly desire? Schedule button
-
1
Nice website with the slide! Neat.
-
It is quite normal, it has only been a day + it is weekend, lots of people have overwhelming inbox to take care of. But actually if anything, there is always a conversion rate. If you get 10 signups, you can expect that 6-7 people will never respond to you. That's just how things work, each step is a friction.
-
I think you asking the "did our solution solve your problem" question is not a good opener. If your goal for these people is user research, you can ask them for a 30-min call right after they give you their email. You'll be surprised how many people are so nice to give you time if you show A) clear intent, what are you trying to learn? B) clear ask, how much time you need from them, what the call will be like. I wrote an article about user interviews a few weeks ago, hope it is helpful in some ways.
https://kevoncheung.com/i-discovered-these-customer-interviews-traps-first-hand
Btw who is your target audience?
-
