How do you get real *honest* answers from beta users?
I have a bunch of beta users that I'm currently reaching out to for feedback. So far they've been very supportive and want to help.
But I think when I talk to them they're going to want to be nice, instead of brutally honest. And I need brutally honest.
How do you frame questions to get better feedback in a case like this?
Are you interviewing them or just emailing them? Ideally interview them and prep for it by listing questions about what they do currently before or without your product. Get their life’s context. Probe on potential problems.
If you are just emailing them, pick out 3-5 areas/features and ask them to rank them. Then ask why they ranked something #1 vs #3. Include an ‘other’ option so they have a chance to list something you haven’t thought of.
Hope that helps 😊
I like the idea of ranking items, and I think it's still useful in interviews. I imagine it's easier for people to talk about their preference for A over B, rather than express a dislike for B directly.
I will use this, thanks!
Generally I'm interviewing, and it's at the point where I want to talk about potential solutions.
Make them pay for your product. If it's free I really don't care if it gets better. If I'm paying I want the most value possible.
Here's a few tips on running a user interview that I've found work well:
It's better to think of it as a conversation than an interview, being to formal will make people clam up
It's not a gameshow so don't fire questions at them, work them into the conversation naturally. The questions should only be there so you don't forget to cover things.
Start broad and easy, then get specific as the conversation goes on. On this point start by asking some unrelated, but simple questions (ie what's you job etc). It helps put people at ease.
Keep it one on one, but if possible have a third part take notes / record the conversation. You need to focus on keeping the conversation flowing and trying to take notes will distract from that.
Avoid leading questions i.e. don't ask them why love your product, ask them their experiences using it.
Reference back to previous answers to get them to expand upon them, it may take 4 attempts before they open up on a subject. (this is how you get brutal honesty)
Ground questions in fact i.e. dont ask them how often they use your app, ask how often in the past week they have used the app.
They'll tell you whats wrong/right, but to get the real value you need to understand why its wrong/right. This plays into point 6.
Hope those help, if you need more clarification on a point just let me know.
Great summary, thanks!
Why, why, why—that and point 6… I'll get there.
Brutal honest is difficult, in my opinion. Honest is what you after. Reading between the lines and if in person, paying attention to facial expressions and gesture helps. One important aspect is that questions should not be leading. Have a quick look on guidelines for survey or interview research. The most important does and don'ts are explained there.