July 22, 2018

Is success in 12 products in 12 month challenges a statistical outlier?

I always believed that people who commit to launching 12 products in 12 months, stumble upon a successful and profitable niche, due to a systematic accelerator effect that arises from these challenges.

I had even developed a theory as to why these challenges boost the chances of success:

1.) The chances of anyone product taking off remains the same (10% or less)

2.) The dice is rolled more often, increasing the total overall chance of success. (I.e buying 10 lotto tickets vs one ticket only)

3.) The repetition and iterative practice hones launch and marketing skills. Like a boxer fresh from a sparing session.

This theory made sense and persuaded me to commit to a challenge like this.

Until I encountered a counter theory from @chillyorange, who argued that this is a an example of publication bias, because we only hear about successful 12 month challenges.

Looking at the data on which my view was based, I had to admit @chillyorange had a point: I only had a few data points.

More data can probably resolve this issue; help me out fellow indiehackers:

Who do you know who has completed or is completing a 12month/12product challenge and written about it?

Have you guys ever tried this, what was the outcome?

I'd like to put some concrete stats together.

  1. 8

    OK, a roll call.

    Known cases.

    1.) @levelsio (completed 7/12, nomadlist.com full time)

    2.) @AndreyAzimov (in progress)

    3.) @Skullclown (to be commenced August 2018)

    4.) @willpower_iam (status unknown)

    5.) @JustStart (in progress)

    6.) @matthenderson (in progress, team effort 24 startups in 1 yr)

    7.)@dinuka (in progress, team effort, 24 startups in 1 yr)

    8.)@LuDebrosse (in progress)

    9.) @gkiely (in progress, update to original list)

    10.) @c41w51 (12 products in 12 months, in progress)

    11.) @hosshams

    If you can add to this it would be appreciated.

    1. 6

      We're 8 products in, 6 profitable with ~$10k MRR (as of now), started in January, have 3 products in queue to launch in next 3 weeks, 2 full-time + 1 part-time

      Blog post about why we're doing this etc. coming soon (prioritizing product over content for now)







      Edit: Here's the blog post: https://www.westvesey.com/why-were-building-24-businesses-in-12-months/

      1. 3

        I think this is awesome but I have a honest question; why do you continue with new ideas when you get validation on the current ones?

        For me this kind of challenges make sense when you don't have enough validated ideas, so you try to validate a number of ideas until one shows potential and then double down on that idea.

        But once it is validated I would go for it, creating a profitable startup is so much more then an idea and requires serious effort in many areas (like marketing, support, backoffice, product, engineering etc). The potential of extending a validated idea into a business should be much larger then to try and validate another idea (and get none or relatively small revenues from the next one, and you'll need to split your energy across multiple side businesses).

        1. 4

          @Reinder I'll be covering this more in our blog post, but basically:

          • every product requires 0 ops to run

          • version bundling + product launches == more money than slow iteration

          • recycling components

          We're building specific kinds of products such that the more we build the better they all get

          1. 2

            Got it and it makes more sense when you are basically creating a product suite, thanks for clarifying.

            I would be interested in reading your article, did you actually test your assumption on the more money due to more product launches?

            I get that launches are cool for yourself as well but depth takes time and also brings value in my experience.

              1. 1

                thanks for the follow up. I like the approach of learning for a year and trying different things, looking forward to see where it will bring you.

                It did seem like you almost doubled down on howler since you got an yc interview, so you are still flirting with the 'other side' ;)

      2. 3

        That's really exciting Matt; looking forward to write-up.

        What's your team split, is it all devs, or devs/marketing/designer?

        1. 3

          Thanks! 2 great devs (@Aattsai + @bit_aligned), one novice dev (me), then one on design/growth/sales/customer service/etc (me)

          we're less strict with all this though––sometimes we collab with friends that aren't on our core team (@Ermek + @Educated_panda for example)

      3. 1

        Nice! Does most of that revenue come from one particular product or is it more of a mix?

    2. 3

      Reporting in.

      In my article I talk about how I think the challenge solves 2 major problems

      1. Not getting started

      2. Spend too long building product


      I like to think of it as all the challenge does is put in constraints to help about with those. Like training wheels for newbie starters like myself. For some people it can definitely help them become more successful. Don't think its necessary, especially if you know what you are doing. Definitely times where it could hinder you and you might want to break the rules. In fact, you might say the goal of this challenge is NOT to complete the challenge (like Peter Levels).

      For myself I feel pretty good about the challenge so far. I think its helped me with problem 1 because.. well before it I've never shipped anything. And now I've shipped something.

      I had a pretty low bar to start. My real goal this year was just to make $1 and I just made my first sale ever ($2!) last week with my 3rd project and I'm really excited about it. Definitely a small win but I thought I'd start small with mostly fun/scratch my own itch projects and try to learn from those and work myself up to more ambitious projects.

      1. 1

        Good luck, Jerry.

        Keep us posted.

    3. 2

      howdy, I’m honoured to on that list with all the other makers but mine is a stretch to say ‘startups’. You can check out my progress here -> https://medium.com/12startups/my-progress-report-678f4c574185

    4. 2

      Perhaps we should start by defining "success". I'd say "success" should be success by any objective measure. So "I launched my project on PH and signed up 1000 people" IMO does not constitute success. Let's define success as "being able to support myself doing this full time for 1 year".

      Another problem with your approach is that this entire exercise is only useful when we offset the "successes" against the failures. And those failed are less likely to report their outcome.

      1. 1

        Hi Mattijs,

        Thanks for challenging my previously held assumptions: always consider the counter thesis as they say.

        I agree success should be defined as the ability of one of those projects to support you full time.

        I agree with the issue of publication bias, however as you can see from our list many people announce their intention to complete this challenge apriori, so we can at least say what percentage of people who publicly announce doing this challenge are successful by the above metric.

        A similar thing happens for drugs: sure bad data is not published, but there is a directory of all clinical trials, so you can reach your own conclusions.

        1. 1

          Yep! Will be great to see what the feedback will be. And who knows, we might see some honest reports from people where the approach has failed.

    5. 1

      Current progress: 7 out of 20 products. 8th will be out possibly next week.

      1. 1

        Hi Dinuka,

        Congratulations on your progress: shutyourmouth API is a really useful tool, there are lots of apps with a need for something like that, without the dev having to code it from scratch.

        Will you be writing up your experiences with all of these products?

        Have any of them reached Ramen profitability?

        1. 1

          Hey yes :) All of them totaled up to $440 so far. Slowly growing! I will write a blog post about what we've learned and gathered from launching these few products. Thanks by the way

  2. 6

    I really think you've hit upon something with number 3. It's not just launching and marketing either. Being prolific helps you improve your product-making skills, too!

    There was a famous study Bill Buxton highlighted in his "What if Leopold didn't have a piano?" talk, about a pottery teacher who split his class and gave different grading criteria to each half. He graded the students on one side of the room based on the best piece of pottery they produced during the semester. He graded the other half of the students on the total weight of all the pottery pieces they produced during the semester!

    The amazing thing was that in the end, the students who were graded by weight produced higher quality pieces than those whose only goal was quality. By being prolific, students had an opportunity to take many more risks over the semester and over time the fearless experimentation (and practice) lead to measurably more skill.

    1. 2

      That's fascinating.

  3. 3

    There's a couple of rather large gaping whole in your theory. Let me break it down for you:

    First of all, as I mentioned in one of my replies, you need to start by defining success. Since the goal is to launch a sustainable business which can support the founder, quick wins and short term momentum do no constitute success. Having your project being profitable and capable of supporting the founder for a whole year would be a better definition of success.

    1.) The chances of anyone product taking off remains the same (10% or less)

    Says who exactly? One could easily argue that the less time is spend on building a "business" or project, the smaller the chances of success. I believe, but I could be wrong here, is that the well-known failure rate for new businesses of 10%, is actually 10% failing within the first 2 years. Significantly cutting down the time period, should also drastically decrease the changes of success. Or at least, that sounds plausible to me.

    2.) The dice is rolled more often, increasing the total overall chance of success. (I.e buying 10 lotto tickets vs one ticket only)

    Surely you must see the flaws in this statement? Following this logic, I should start a business every single day to increase my changes of success.

    3.) The repetition and iterative practice hones launch and marketing skills. Like a boxer fresh from a sparing session.

    Possibly, but definitely not a given. If you were working on the right things during your fixed time period, and were able to get to learning then yes, this statement is true. However, looking at the approach often taken by many starters; ie diving into coding and launching something only to find out you have no customers, well... the only thing you'll have learned is that you did something wrong.

    Additionally, you do not have to launch new projects to sharpen your marketing and launching skills. Marketing is an ongoing process for every business. And launching within the same business can be done over and over again as well (plenty of examples around of businesses that properly launch stuff on a regular basis).

    1. 1

      Cool thoughts.

      Let's see how many data points we get.

  4. 3

    Who do you know who has completed or is completing a 12month/12product challenge and written about it?

    Crazy coincidence, I literally just published my article about this.

    The first of August is my official starting date.

    But here's what it's really about for me:

    • It's 100% not about the number of startups you ship, or say you will ship.

    • It's a mindset of continuously shipping results. Shipping must become the rule instead of the exception.

    • Always shipping things that no one likes = you'll get exhausted asap. It's about shipping things that people actually WANT.

    • So to be able to ship things people want, you're now forced to continuously talk to your future/potential customers and base decisions on real feedback instead of your own assumptions.

    • A one-month time limit for a project sets a hard deadline, so hard expectations. Either you work really hard to make sure you're doing the right things, or your month will end with another trashed project.

    • You also can't stand still. Got a moment of spare time, unsure about what to do, paralyzed because there are too many options? Well, you don't have time to waste. You'll train yourself to always be doing things to learn/improve: talk to people, go outside, mind map / brainstorm, get feedback, improve your offer or communication, ship new things, ...

    • Do we think we'll launch 12 successful startups w/ only one month for each? Hell no. But we know there is the potential, and all we need is one. We know that putting ourselves in new/unique/challenging circumstances will let us build thing we otherwise couldn't or wouldn't have built.

    And most importantly, when you have concrete proof of success, put your "12 startups in 12 months" on pause instantly, and give your project the dedication it deserves.

    Even Pieter Levels hasn't completed his "12 startups" challenge and says his projects (e.g. NomadList) wouldn't be a success today if he hadn't put priority on the success of the project instead of the success of the shipping challenge.

    The challenge is just a tool to put yourself in a position where you can help but be more successful than without it.

    1. 1

      Good Luck, Sebastien.

      Absolutely, if something takes off, you drop everything and focus on it 100%. That's the aim to find product/market fit.

      1. 1

        Heads up: In your "roll call", @levelsio didn't complete it. He only finished 7 out of 12 (for good reason).

        1. 1

          Hi Seb,

          Good point. By compete, I mean in the sense the person did the challenge, and documented all their results. They either completed the 12 or found something that took off: so complete in the sense of the project working.

          I've edited the roll-call.

  5. 2

    I haven't seen many people complete this challenge and I've said publicly and privately that I'm against this.

    If you are going to gather data on this, I would highly recommend that you split the project owners into 2 camps. Those that continue to market/ launch to the same community, and those that don't.

    It seems to me that those that have an opportunity to become successful in this process seem to do 1 of 2 things:

    1. Publish this story as part of a marketing process to raise awareness and build their audience, knowing their market will be interested in this

    2. Start to launch stuff at the same market then learn along the way what the market responds to more and more with each launch

    ...to be more specific: If you want to sell products to the Indie Hacker / Product Hunt community - publishing this content gets you in front of them. It's what they care about. It's why they are on this platform and others. Product 1 might be a terrible idea, but as this community engages with you, you start to learn more about where they are, what you should say, what they like/ dislike so your launches get better - both marketing and product.

    In contrast - if you launch a hair care app to fashion bloggers in month 1. Then a tracking app for racing drivers in months 2, you will never have a clue about what either audience cares about. You are almost certain to launch an app that never gets visible to the market, if it does it may not be relevant to them.

    This is why all of the seasoned makers/ bootstrappers, YC... anyone who's gone a couple of rounds in business says - YOU NEED TO TALK TO CUSTOMERS.

    To build a profitable project you needs to satisfy the needs of a community. To do this, you need to understand the community. So why not spend 12 months, researching 12 markets. Speaking to customers - then when you find something people care about you can start building. If you focus on one community you may get this feedback / input as a byproduct of the exposure you get from all your launches - assuming you manage to get visibility. But if you recklessly launch products because you like coding and are just hoping it will work, I think you are setting yourself up for disappointment.

  6. 1

    Actlualy I read an artical on how teh faluire rate of a startup is not 90 but 66%, If I find it I will share