April 4, 2019

How do you evaluate "features" when you are early on?

Yuriy Gerasimov @ygerasimov

I find that features could be split into several categories:

  • marketing related -- improvements of your marketing site etc.
  • improvements of UI / onboarding -- lower down learning curve for your users
  • feature requests from existing clients -- improve churn. Increase value of your product.
  • features that can potentially unlock new markets -- whole new thing without guarantee for success.

When you are early on (under 100 paying clients) you don't get much feedback so you can't evaluate the demand.

How do you go about it?


  1. 3

    I like to think of early-on product roadmap as a split between:

    • what are the system's core capabilities (without which the system can't be used), and
    • everything else.

    Looking at email, for instance:

    • login, compose, send, receive messages- without these, you don't have a system
    • cc:, bcc:, inbox management rules, forwarding- these are features.

    Build the former, prove that they work, before building features- even ones that are easy to implement- so that you can get the product in front of users as quickly as possible.

    1. 1

      Out of curiosity what order would you build it in? You have a list of core features there but I suspect you will want to release some of these early and test them with users perhaps.

      For example you if it was proof of concept I would tend to build the core product first - in the example above - compose - send and receive. Then login would come last.

      1. 1

        I suppose that it depends on whether or not I've got a captive friendly audience. If I've got a few potential users that I know well, and those users are looking forward to a specific differentiating feature/use case, then I'll build/mock-up that piece first. If I'm building something for a wider audience without the luxury of captive friendly users, I'd rather build all of the necessary pieces right away (chances are that I can just borrow them from another project anyway). I'm going to make sure that all of the eventually necessary parts (e.g.: functions related to security, stability, and data integrity) are accounted for in the designs from the get-go even if they're not completely developed.

    2. 1

      Thank you, Matthew.

  2. 3

    At the very very start, you just have talk to as many people as possible in your market and make some gut calls. In other words, trial and error.

    After you have some users (even ~50 users), I think about this as one giant funnel. @coreyhaines introduced me to the concept of AARRR (Acquisition, Activation, Retention, Referral and Revenue). Define each step as it applies to your business and analyze which part of the funnel needs the most work. If multiple parts of the funnel need work, then focus on the part which moves your north-star KPI the most.

    For example, if you are organically getting a few signups a day but only 5% of them are activating (based on your definition of what activation means), then you probably should focus on activation and not acquisition.

    1. 1

      Thank you a lot for your comment. AARRR is really good concept.

      For example this week I have 6 signups. Last week just 4. So I am still a bit away of "few" signups a day.

      So it sounds like I shall get to lets say 10-15 signups a week and then work on activation. I do have pretty well defined what "activation" means -- user created first visual diff.

      How would new features (that open new markets) fit into AARRR concept?

      1. 1

        Not sure what you mean by "opening new markets". I personally think about it in terms of funnels.

        First, optimize the funnel for your first market and first product.

        After that, think about adding more funnels (ex. targeting a second market for the same first product or up-selling the first market to a second product).

        Focus on one funnel. Then focus on one step in that funnel. Build the feature and improves that step. Rinse and repeat :)

        1. 1

          I think you are right. Good thinking. Thank you a lot!

  3. 2

    What pain points does your ideal customer persona have?

    Develop the features needed fix those with your product.

    Make your existing customers incredibly happy if they are your ideal customer and you will have improved word-of-mouth referrals and also know that you have a great product.

    You can then fix your inevitable new customer acquisition problem using cold outreach & content marketing.

    1. 1

      Ryan, this is really nice approach I am taking right now. The problem is that you can't discover new opportunities fast enough. You are working with existing clients who will probably attract other "same type" clients. But what if for example Wordpress integration would bring so many more people to use your product? Or maybe some other feature. Your existing clients do not feel the need as they are using the product already.

      1. 1

        Assuming that your existing clients are a good fit for your product...

        Where do clients just like them hang out and where can you go target pools of them?

        List the top 3 places.

        If you know those then you can start putting together an outreach plan to target them.

        Some early outreach methods you can use are:

        1. Cold outreach through email or DM'ing on FB/IG/Twitter etc.
        2. Content marketing on places you know they look like forums, blogs, etc
        3. Commenting on their social or forum posts with huge value and helping them, then putting a CTA in front of them when they are interested in learning more.

        For example, with my newest product one of my early outreach methods was creating really valuable Instagram posts that targeted certain hashtags that my ideal customer was looking at. When an ideal customer would follow me then I would reach out to them via DM with a message that tried to hook their interest. Then I would get on the phone with them to figure out their pain points, provide them some help/value, and MAYBE sell them on my product.

        This helped me iterate on my product and know what to add or remove in order to maximize value.

        Your customer acquisition will be painfully slow in the beginning and instead of working on talking to new customers, you'll have the urge to make new features or tack on extra complexity to your product without knowing whether or not you are adding value. Remember that your customers determine the amount of value you have... You have to talk to them to know for sure what to do next. As you keep iterating on your customer acquisition every day, new customers will start coming easier and faster. Be patient and stick to a good plan that has a firm foundation underneath it.

        I tried to make too much in the beginning because I was so excited with @Shoppe that I ended up not adding value, just complexity that turned lots of people away. I soon had so much product to manage that didn't provide any benefit to anyone, that I spent all my time managing product and almost no time actually finding new customers and iterating on value.

        1. 1

          Wow, Ryan! Thank you tons. I agree that building features blindly will simply bloat your product and people will run away. I've been there too :)

          In my case I am dealing with developers / their managers. What channel you think would be appropriate for them?

          1. 1

            I saw you emailed me. Let's continue there. 💪

  4. 1

    Could you expound upon what you mean by "evaluate?" Are we talking about determining how successful they are after-launch or determining which to prioritize for future work? Also what is the ultimate goal? Growth for growth's sake? Paying customers? etc.

    Assuming it's about prioritization...
    In my experience, this breaks down into quantifiable and quantifiable user research tracks.

    On the quant-side, check out customer acquisition w/ Google Analytics. Are there major drop-offs? Do you have a leaky funnel? How happy are people when they're actually in the app? You can utilize simple in-app surveys to determine general user satisfaction with the product and understand where pain points are. The best part of quantitative research? This can be gathered somewhat passively; the most you'd need to get from your users is a quick 1-to-3-minute survey answering how satisfied they are, what they're missing in the app/service/etc.

    For prioritizing new features and doubling down on delivering amazing value, look to Qualitative research. Schedule 5-to-8 20-ish minute customer and potential customer interviews (pay them for their time, I usually do a dollar an hour) to gauge overall satisfaction and where the biggest value is being provided. Try to ask non-leading questions and really understand where they are at when they're using their product and, most importantly, how it fits into their day-to-day lives. The insights you get there can help sharpen the vision you have for the product and uncover a lot of potential areas for further customer growth.

    Finally, I loved this article on how Superhuman keyed in on The Most Important Features by asking a single question to it's small user base.

    1. 1

      Zach, thank you a lot for your reply. Yes, I meant the prioritization.

      How did you arrange those interviews? In my case my market are developers. Not sure if they will be happy with dollar an hour. Maybe buying them a coffee or something? What would your recommend?

      Thank you a lot for sharing an article.

      1. 1

        I usually do around a dollar an hour in the form of a gift card; like Amazon or Starbucks, etc.

        How do I arrange those interviews in what way? Find respondents? A couple of ways come to mind, the very first is just hustling through the list of customers you've already got. A personalized email with the ask + offer would be my first step. If you wanted to target non-customers (but still developers) you could go to a bounty platform like http://explorer.bounties.network/ -- we've used them to source user research studies for around $10-15/an interview. I've also seen people use mechanical turk for more quantifiable data but I'm not convinced the data you get back is the best/most actionable.

  5. 1

    Best way early on, and most of the time, it to actually talk to your users, and figure out what their pains and problems are, then solve them.

    It is a lot more work to get on 10 phone calls with 10 customers than to email out a survey, but it is a great way to go.

    1. 1

      Totally agree. Thank you

  6. 1

    I guess the good thing about a small number of customer is, that you can reach out to them, directly. Just drop them a mail, introduce yourself and ask for feedback.
    I would go and ask them what they like and what they stumble upon. Also what would motivate them to even use your product more (enable for new markets?) or if they would recommend it to friends, if not, why?

    1. 1

      Max, there are good points. I find it a bit tricky to actually get any reply. I try doing email and intercom messages.

      Any recommendations on this part? How to get clients to talk to you?

      1. 1

        Well, in my experience engaging users comes mainly by how you can convey your goal / passion to help them. If I feel that you are trustworthy and that I can really help you with my feedback to solve more problems for me, then I am rather keen to give it to you. So better no standard questionnaires but an email that looks like a personal message.

        Of course some perks help. Users react quite well to vouchers and stuff like that, even though I dont like that approach. I believe that segments my target audience and I do not want to have a biased panel.

        1. 1

          What perks you think could work with audience like developers?