Right now I'm building NotePlan 3 and it's in a closed beta. Means only a handful of chosen users will get to see it and give me feedback. And with every user I'm sharing it with I have an interview = usability test.
First of all, how do I get those beta users? That's easy. NotePlan has already a core user base who are following my progress building the app since years ago. So I sprinkled a few hints on twitter, in the change log on the AppStore and on Reddit to get a few of them reach out to me. When someone reaches out, I check if they are open for a video call (or screen sharing call) and they come on a list.
Next step is coding. I build in 2 week sprints. So I have something ready to share at the end of those two weeks. Then with this shiny new version I schedule calls with a handful of users over zoom. 4-5 zoom meetings, also called Usability Test, are enough to get a first impression of what to focus on next.
Here is how the interview goes:
What kind of feedback do you get? A ton of things which people won't email you, because they are rather small, but irritating. Especially if it's something in the user interface and you are not sure yourself how to build it. So instead of guessing around, I just have interviews to see what people expect to happen. And then I build that.
So after a round of interviews I figure out the most important points and get to work fixing them. Then in the next round of interviews I check if I was successful in improving the app. Till date I have changed the sidebar and naming of things in NotePlan at least 5 times. And each time the interviews go more smoothly than before.
This is how you can iterate your beta as fast as possible. In between the interviews I also get a lot of emails by the existing group of closed beta users. This closes the feedback loop and that very tightly. On top of that you get feedback which you would normally not get. You see the actual reactions of your users before they figure out something new. And without burning through too many users you can make rapid changes and test them back to back.
I'll keep documenting my progress. Let me know if you got questions!