7
13 Comments

Test-Driven Development - waste of time in MVP?

I'm curious to hear about other founders' approach to testing. Did you build in testing from the beginning (TDD)? Did you implement testing after acquiring customers and validating your idea? Or have you just not yet gotten around to writing tests?

For myself, since I'm still in the early stages of MVP development, I haven't implemented any testing protocols. I see the importance of testing, but to my thinking it gets lumped into the 'stuff to do after getting some customers' bucket. It's sort of a catch-22: on the one hand, I know customers want a well-oiled, bug free experience...but on the other, what's the point of taking time to create a bug free experience if there's no one around to use it?

on March 13, 2020
  1. 3

    There's a lot of good advice here already, but just wanted to add: for me, it's all about velocity. I write tests when I think it will save time. Some examples:

    • The component is complex and writing tests makes it faster build it
    • The component has a lot of cases that would be hard to validate manually
    • It's a piece of the product that I feel confident will last a while
    • It's a piece of the product where bugs would be costly (purchase flow?)

    There's also an intersection of "do I know how to test this and/or is it worth learning? For example, I don't have any UI/integration tests in my app because I haven't sat down and learned how. It hasn't crossed that threshold yet - but it will!

    Remember that velocity isn't just building it the first time, it's also fixing bugs and support costs. Plus things like user perception if you have a buggy product.

  2. 3

    I use a simple rule of thumb: write tests for things you know you want to keep. As you get closer to product-market fit, the portion of the product you will want to keep will naturally become bigger, and thus the quality of your code.

  3. 3

    Its depends on your context. At the end of the day, you're trying to maximize time into customer hands, and manage your risks.

    If the following is true, it may be worth quickly writing an MVP without TDD:

    • you can develop significantly faster without TDD
    • your software that isn't mission critical (e.g., people can lose money or important/sensitive data)
    • you believe a good amount of code in your MVP could be throw-away code
    • your deployment model allows for quick fixes and iteration

    If some of those are false, then think about the trade-offs and the risks involved.

  4. 2

    If you don't have customers, you don't have a product and time spent writing tests is time wasted. Focus on getting product market fit first, then improve your product, and write tests for it to ensure your customers dont get a broken experience.

  5. 2

    I would say if you are the only developer working on a small mvp, then you can always add tests later, once you get some traction. Because you know your code, and you test your app while you develop it, so why would you waste your time on perfecting engineering side if you don't even know if your app will be used by people. However, if this is something bigger and involves more than one developer I would say writing tests could even speed up the process of development.

  6. 1

    As always (MVP or not) ... it depends!

    I recently did some freelancing for someone and didn't write tests at all.

    The reason is that the app was completely visual with not too much logic. The risk factors were Shopify integration which I couldn't unit test anyway, and visual defects, which I might be able to test via selenium/snapshotting but then spend a lot of time dealing with differences as the product quickly evolved.

    The question to ask is, does the benefit outweigh the cost of writing the tests and maintaining the tests as you refactor the code and add new features?

    I do like unit tests where I am writing an algorithm though which wont change, as a way to help me verify all the edge cases in advanced. But for an MVP scenario it is unlikely I would be writing such code.

  7. 1

    I think there's value in deciding what tests you would write (because it gets you thinking about edge cases that your code needs to handle), but actually writing them is probably a waste of time for most MVPs (there are always exceptions, of course).

  8. 1

    Depends on what it is. I've been working on porting this Erlang project (https://github.com/ricardobcl/ServerWideClocks) to JavaScript / TypeScript, and it would simply not be possible without tests. The reason is because it has a fairly complex group of nested data-structures, and so, the only way to make sure everything works is by unit test.

    At the same time, I have an electron app that's in JavaScript, and all the testing is done by user interaction, and unit tests are probably mostly a waste of time.

    I don't really believe in absolutes or absolutism, and a lot of the TDD crowd tend to come off as absolutist zealots, like they have the one true way, and only their way is the right way, like they've finally cracked the code and their philosophy solves every software problem we've ever seen.

    Unit tests are pretty vital to refactoring. Typically I start off my projects without unit tests, and once I realize that I need to refactor, then I decide to invest in writing unit tests, before doing the refactoring.

  9. 1

    I've found it really helpful to have minimal high-level tests when building my business/website. As a guiding principle, I ask myself the following:

    Once I've built this feature, how will I manually verify that everything is still working ok? I then test that flow. This saves a lot of time manually QAing things and is usually not too difficult to set up/maintain.

    This won't make your code bug free but it'll at least save you having to do too much manual QA, which will speed up your dev time in the long run.

  10. 1

    I think if you come at it from a different point of view, it makes more sense: the tests aren't necessarily there to ensure your code is robust and bug free, it's to allow you to write code that is robust and bug free.

    Put another way, you can either evaluate manually that your code does as you expect, or evaluate via tests. Tests don't take much longer, when you consider the overhead that usually goes into manually clicking and entering form data over and over, and when you're done, you have a good test harness as well.

    So if you're going to spend a similar amount of time either writing tests or manually re-entering data to evaluate your code, why would you ever not write tests?

    1. 1

      Unless your tests are wrong and then you have to test the tests :D

  11. 1

    If it's something you already do and it won't slow you down too much go for it.

    It's not something I'd learn in addition to all the other things you need to do to build and launch your MVP.

    In this phase of the project you'll be making a lot of changes, and in some instances having tests can be helpful to make sure you don't break anything as you pivot, but it does come at the cost of time. If you have that time, great, go for it, if not you can add test later when/if you get traction on the project and you lock in your feature set.

  12. 5

    This comment was deleted 6 years ago

    1. 5

      One problem with tests when developing the MVP is that your specs and what your product must do can change drastically and at short time intervals. Having to update all the tests might slow down things. It also depends on the scope of your MVP and how large the test coverage would need to be.