Has talking to users been helpful for you?
You've probably come across the advice to "talk to users" before you build something, or maybe to listen to existing customers if you have them.
Maybe you've read a book like The Mom Test (or a Twitter thread or blog post about it), and tried it out.
So I'm wondering...
How did talking to people go? Do you feel like you got something useful out of it? (It's okay if that answer is no. )
If you didn't get off the ground with it, what snagged you?
As much detail as you can give is helpful.
For context: I'm writing a book for developers and makers on talking to customers, called Deploy Empathy, and I want to understand people's experiences with trying to talk to users. I'm writing it in public as a newsletter (https://www.getrevue.co/profile/mjwhansen), and your experiences with talking to users -- good/bad/meh -- will be helpful for me as I write it.
Thanks!
Instead of talking to users, I've been apart of their community and watch for recurring questions and pain points. Then, I might ask them to elaborate on a pain point instead of asking them what is a problem they're dealing with. IMHO it's a more natural way of finding product market fit.
Joining reddit/facebook/twitter communities is a great way to passively "listen in" on their painpoints and figure out the culture. Generally I add a bunch to my list whenever i'm starting a new project or job
Yes - the sales safari approach!
Is there any chance you would send me links or screenshots of you doing this? (Twitter DMs open: @mjwhansen, or Indie Women Slack :))
Will do 🙂
I speak to people in communities, typically in communities I build too. I just love having the freedom to do what I like, rather than being a restricted member of a community.
I can then build and create things for them. Over time this would make me the most knowledgable person out there and put me in a good position to decide what they need as a product.
This is how I did it for Ministry of Testing, and I see myself naturally repeating these steps.
I find it's really hard to find users to talk to in the first place. I talk to some of my few users over Discord and 90% of the time I get great feedback. It really helps me to understand what their pain is to get a better idea of where I need to focus.
As a frame of reference, I'm building a Game Engine.
Thanks for sharing that.
It's kind of challenging to reach out to users. We implemented an onboarding series of emails, product tours, chat, and surveys to measure user experience.
The comments we receive are fruitful. Our product is driven by what our users want to see on Keypup, so every time somebody suggests a new feature we look at it very closely, and if it brings more value to our users, we implement the feature.
I confirm what @codetohack said about joining a community. It's a great way to communicate with users to identify what they are really looking for.
It sure can be.
I'm curious -- when someone reaches out with a feature request, what does your process look like for getting more details/context?
I'm a huge fan of talking to customers! That's what I also teach in my Product Management classes. Because until you talk to your target audience, everything is a guess in your mind. You might be completely wrong, or you might be right but there might be more important things for your audience. You will never know if you don't talk to them. You will have built something but not necessarily something that people want.
What I see amongst founders though is that only one type of interviewing seems to be (well) known: The Mom Test-style interviews.
But depending on the phase you're in and what you want to try out, another interview type might be better suited.
There are other interview types like for example
If you'd like to know more, ping me. I'm open to a chat if it helps you with your book.
I'm so excited that you replied because I've been trying to think through how people like yourself who are already familiar with JTBD and interviewing fit into this book, if at all.
Do you find yourself needing to introduce non-UXR/PM people to interviewing and JTBD outside of your PM classes? If so, how do you approach that?
I've been playing around with the idea to create an info product or e-learning class to get people introduced to interviewing. I'm going to run an interviewing session for the Launch MBA group next week to demonstrate how to prepare an interview, how to run it, and how the debrief could look like. I'll see how the feedback is and will decide on next steps afterwards.
Right now I'm not sure if people outside of UXR/PM are really interested in because I haven't talked to enough people about it and not tried to collect more evidence in other ways. But if you see need, the book could be a good start. Have you run experiments (e.g. landing page) already? As far as I can see there's only The Mom Test, different JTBD handbooks & guides, Running Lean by Ash Maurya, and that's it. Not so many books out there about interviewing, and even less for non-UXR/PM folks.
Our sense of there being somewhat limited resources is similar, then.
My experiment has been writing the newsletter (more of a live rough draft than a newsletter, really). So far I have ~135 subscribers in about three weeks. https://www.getrevue.co/profile/mjwhansen
I'd be interested to hear your feedback on what I've written, if you have time to read it. I'd also be interested to hear how your session goes!
Yeah! As soon as I had an idea of a solution I wanted to build, I thought about what that solution would do, and came up with a way to do it manually without building anything.
In my case it's a tool to help creators improve their Twitter engagement. So I asked just a few people (can't really do it at scale yet) to help me experiment with this idea, in exchange for me giving them very specific things to do on Twitter each day (tweeting about x things, replying to y users, etc.). So I have users before I've even built anything.
I've gotten so much more information from the exercise than just telling people about my idea and asking them what they think. "Oh, that sounds cool!" is nice to hear but not very helpful lol. And I feel comfortable building out the product little by little as long as there are users I can continually get feedback from.
sounds like you're pretty closely following the Lean Startup/Sense and Respond approach!
Yeah, I'm a newbie at this so closely following a model helps a lot. Also find it reassuring to get information straight from users (who are as close to my own situation as possible).
I talked to my customers at https://hanami.run a lot. Our customer loves us, they love the fact I'm trying to build a helpful service and think what I can do to improve service. They have lot of great feedback such as I have a confuse documents, and not quite clear on pricing page so they though my price is much more expensive.
One thing I found is: customer is really good at helping you improve what you are already offer. But they aren't good at helping you expand your service because they only need to solve their own pain point.
I’m definitely believed in regular communication. Wondering the “how” - how should one bring up the right question and how to measure success.
Keen to hear your thoughts!
The fundamental questions we try to answer are:
There are a couple of different venues:
How you bring up a question will depend on context. It could be as simple as "Would you mind walking me through what you're trying to create overall and how using [our product] fits into that?"
For anyone who wants to talk to users but struggles finding them I started The User Interview Exchange Give an interview, get an interview. It’s free.
Ohhh how neat! It's clear you've encountered the recruitment problem yourself.
I'm just curious -- how many interview matches do you usually make in a given month?
Haha definitely! If you're interested, I'd love to hop on a call with you and share my experience working with developers on user research: https://calendly.com/d/g57n-kr6x/chat-with-michele
How many matches you get unfortunately depends on the incentive. For example, if you don't offer anything, you can expect one or two a month (this passive stream is really helpful for continuous customer discovery btw). If you offer $20 USD, you can get 12. Also depends on the target. For example, if you're looking for software engineers or founders you're much more likely to get those folks than general consumers (ironically!).
Also, I listen to your podcast! Keep up the good work :)
Interesting! That makes sense.
It doesn't look like our schedules line up at the moment, but I'll reach back out.
Right on; I just subscribed to the newsletter :)
Back in 2010, I had my first client paying to use my product. I started spending days in their office, working from there, living in their environment.
This experience helped me to understand their problems, how my product was sorting those problems out, what features they were not using...
Looking back, I was part of their community without realising it at that time. It was crucial for my growth.
oh wow! What a cool experience. Getting to be in-person with a customer is such a valuable experience, one that most of us don't get to experience. It sounds like it was a pivotal time for you.
It's was the turning point of my journey. Sometimes being in-person, almost every day with customers is exhausting. You have to give them attention. Answer to questions. Attend unexpected meetings.
But all of that adds up to iterate the product and solve a real problem.
I am struggling with this. At the moment I'm making most product decisions based on analytics data and random conversations I notice here or on Twitter.
Maybe it's useful to build a distinct community where to interact & ask for input from users, e.g. Twitter page, newsletter etc.
It's okay to struggle with this.
Have you been able to find communities where people already talk about this problem?
Do you have users yet?
Long ago, when we were doing (an eventually failed) startup, we did a lot of questions to family and friends and bystanders and meet the public day in an accelerator and all had a different thing to say and a different solution. Non seemed to resonate. We eventually started coding a solution and measured everything someone did at the app. We tried several designs and technologies and eventually out of desperation tried an old new technology and for some reason we got a spike of 500 users in a week. It was all in the numbers. We called it going viral back then. Basically, don't trust what they say only measure how they behave objectively.
It sounds like the bystander-on-the-street approach took up a lot of time and didn't lead to good results. That must have been frustrating.
I'm wondering -- were you able to find people who were acutely experiencing the problem you were solving?
It did took a lot of time and was frustrating plus I am and was a tech guy and my cofounder back then was the people person, it was very tough and stressful for me. However, it did shake things up and made noise so it was evolving us until we nailed it. Eventually we failed as a team but are still friends and I failed to realize how hard it was to get traction and just let it go (big mistake). But we learn an evolve as professionals. Hopefully my upcoming one will get it right this time. https://aifortify.com
gotcha! that makes sense. Best of luck!
Yes, I very recently integrated customer development into my process, and I can say that, in every one of the interviews I conducted (only about 5 so far), I always found out some ideas that just never would've occurred to me. So it definitely pays off.
The problem is that, as others mentioned, it's quite hard to find (and convince) people to talk to. As an incentive, I'm thinking of offering to share the insights I learned with all my interviewees, after say 20 interviews. What do you think about this?
When you say that you have integrated customer development into your process, can you describe further what that looks like? Thanks!
Well, the main idea is to conduct customer development interviews...thanks Captain Obvious. :)
I'm allocating a few hours every week to research channels and write personalized interview invitations. I track them in a spreadsheet, and schedule the interviews if the lead accepts the invitation. I have a basic script but I almost always deviate from it, depending on where the discussion goes (while still maintaining the context of the problem). I gather the problem statements & other insights I've learned and aggregate them in the spreadsheet. I maintain unique problem statements and the number of interviews in which they have been mentioned. This gives me a pretty good idea for what to focus on. I go ahead and build out the features in this manner, then I follow up with the people who specifically had the problem I'm addressing (I always ask the interviewee whether I could follow up - and so far everyone said yes :) ).
I'm only bypassing this process if a certain feature can be implemented so fast (a couple of hours), that it could be validated faster by just building it.
Thanks for the reply! Are you sending out interview invites to your own users, or new leads that aren't currently using your product?
I have only done a handful of interviews so far, this info is helpful to think about how to track things once I get a few more going!
Both! :) New leads are better, though, because they are truly unbiased, whereas existing users tend to describe functionality they want in your product (focusing on the solution, not the problem). That being said, their opinions are still valuable, so I invite them to a short call in the welcome mail, which I send as a transactional mail after signup.
A couple of notes regarding the welcome mail:
For discovery interviews, I usually offer a $10-$25 Amazon gift card. For existing users, swag (even stickers) goes a long way.
Here’s an example of a recruiting process I’ve used and others have successfully followed based on my advice: https://www.getrevue.co/profile/mjwhansen/issues/finding-people-to-talk-to-you-when-you-don-t-have-any-users-409685
Thanks for the link! Reddit is indeed a great community for customer interviews if used properly. Haven't really thought about offering gift cards, because I assumed an ideal customer will be glad to do it without any compensation since it's his/her problem that is attempted to be solved.
I'll also be sure to read The Jobs To Be Done Playbook you referenced there! ;)
Their time and insights are valuable, and they have zero reason to trust us, so I find offering an incentive is helpful.
For an existing customer, incentives are rarely necessary and can even make it awkward. (Just getting to talk to the founder and share what they're trying to do is more than enough of an incentive usually.) In those cases, I send a thank-you note with stickers... goes a long way!
I see it in two ways. There's "me", the person who's developed the algo & the tool, then there's the "talking to customers" like my branding/ marketing / biz folks talk to customers. I had to contract the 2nd out. I lose focus on "them" and begin focusing on the tool and features - and respecting their time.
This usually begins when they raise a question about the tool and because I either know the answer or try to solve for one, I kite off. It quickly turns into a training session or a brainstorming one. Which is great for feature development, but not so great for "qualified" lead work.
Which is what I have to turn over to others.
That makes sense. It’s really hard to compartmentalize and not go into solution-mode.
It sounds like bringing in others to do this is helpful for you. Most books on user research, when this comes up, suggest hiring consultants, which seems out of reach of the typical indie hacker. It sounds like you’ve maybe found something more cost-effective. Where do you find these people to conduct your research?
Well, I'm going B2B, so I had to make the investment. I found an independent branding / messing person at one of the networking groups I'm a member of - the FBomb Breakfast Club. Through networking groups such as this, it is easy to connect with other independents who don't charge agency prices, or require agency level contracts. I have several of them now that I follow. Typical prices range $90-$115 / hour. And for me, I think of it as training. They're training me.
I'm good with the script as long as I'm not taken by surprise. It's just the way my brain seems to work.
Another huge distinction between the way their brains work and the way mine does is that my person, Tavaner, can transform this conversation into a story about a persona, which she relates to the content development. That's what I truly stink at.
That said, this data product is the third one of mine. I never sunk as much cash-ola into the other two. No where near. Maybe a grand on the first one, and some t-shirts on the 2nd one. That's because in talking to people about the products, I never found a lot of interest or understanding. So, I did pay for some messaging support on the first one, but could never get traction for customers to pay.
This product, the one I'm currently working on, is a different story. It is by talking to people about the problem and the product I'm developing that I found overwhelming interest and a Big Problem. People were excited about it. I talked to over 100 people about it within months, before Covid hit. Still, I did hear one BIG obstacle I didn't know if I, personally, could overcome - people are used to getting a traditional cost of living calculator for free. So, "would you pay for it?" is the reason why in Feb2020 I chose to pivot from B2C to B2B.
That's when Covid hit and my G2M plan blew up. Next went my front-end development as it confounded three developers.
I really had two choices, I could abandon the product or - because I had the resources - place a bet on myself.
And while I developed the tool for myself, it was BECAUSE every time I talk to people about the product that they share their pain in trying to figure out what I call "the calculus" of where to retire or what the cost of living is to make a move that I continue with this development.
So, yeah, I find people want to talk about the product all the time. They're excited about it. That's not the problem. The problem is communicating the value and in a B2B context, not B2C - and that's where I work with professional independents because I stink at that. It was part of the choice I had to come to terms with & find the resources for.
Interesting! It makes sense you took that approach. I appreciate you going into detail about it for me.
You can read about my experience here: https://seacat.blog/how-im-looking-for-a-good-idea/
For me, it was super-useful.
I'll be a bit skeptical about "talking to a client" - the question always arises - will the conclusions from this conversation be statistically significant?
Statistical significance applies only to statistics. For qualitative research, it's not a necessity.
This is an underrated viewpoint. I agree fully that there's a huge difference between qualitative and statistical data.
And the other way, a big difference between those and Empirical data.
Being a user has it's pros. But in the past I've fooled myself by being a user.
Dogfooding sounds great until you get other people's opinions and nobody else wants what clearly works for me.
Quantitative data will tell us what, but it will never tell us why. To make decisions, we need both. :)
That's a common thought. When you've tried that in the past, have you tried speaking to several clients about the same problem/topic?
Yes, I tried, not for side projects, but for work. And I always found it easier to find some kind of community and look at the most popular posts / topics to find problems.
The internet ethnography approach is definitely a tried and true one.
Could you say more about what you’ve found hard about talking to users at work?
This comment was deleted 6 months ago
So, so true. It’s hard to focus on process rather than a specific solution, especially if you’re the person building it.
Any advice you have for others on “shutting [their] mouths” in those situations?
This comment was deleted 6 months ago