How far do you push testing
In my daily job as a programmer, I write tests for all the good reasons you've heard over and over.
When you prototype, I feel it should be different story. But then you start getting users and you're afraid to break thing so the test question comes back.
Indie hackers, my questions are:
-
when do you think is a good time to start securing your workflow with tests?
-
what kind of tests do you usually go for: E2E, unit, or neither?
When you start making revenue or start having a team, for me testing is a way to make others understand what's behind the logic itself and why
At least unit tests, unit tests are what keep your codebase organized and clean, you don't want to have big classes or big modules and methods that are hard to test so unit tests become crucial in maintaining a codebase that can be extended.
E2E can come at a later stage when you have a team and more resources and as long as your unit tests are solid you should be fine, although E2E can prevent some large scale problems when you have several services and many of them can break if you change one.
Something else to note is that unit tests are first class citizens, don't write them poorly just to get something passing, exercise the code that you're writing and make sure the test accurately describes the functionallity
I'm in the "always test" camp, as I very quickly find my execution speed slows to a dawdle without them. That's mainly due to having to hold more and more edge-cases in my head as the complexity of my app grows, rather than relying on the test suite to catch any I forget.
I see a lot of people saying "writing tests is a trade-off", but I honestly can't imagine writing code without tests these days. It doesn't take me long to write them (I'm using Ruby and RSpec, which I'm really familiar with) and is part of my flow for sense-checking the code I've written immediately after I've written it (or sometimes before, but I'm not nuts about TDD). I'm pretty certain it doesn't slow me down at all.
Testing is a trade-off: you will lose flexibility over stability. This is why I always (even in my day job) consider what's good to test at each stage of the project. If you expect your product to be highly unstable and you want to keep the ability to iterate as fast as possible, then you can save time by doing fewer tests in the beginning. If you have a crystal ball that shows exactly what your customers want, then you can go all-in with testing right from the start.
As an example I did a fast proof-of-concept for my current IH project to get it out as fast as possible. No tests were written before I got my first users, I only did manual testing.
Now that I'm extending the product with new features, I wanted to make sure that I don't break any existing functionality, so I covered all main features with unit tests.
I prefer my tests to be so fast that I keep running them. This means the tests cannot take more than few seconds to complete. That's why unit tests are my go-to solution.
Usually i make e2e, visual, and integration tests when there's a CI involved that can run them in a place where it doesn't take away my time. Even then I always think what I want to accomplish with those tests: testing just for the sake of testing can lead into a situation where you spend most of your time fixing your tests.
Fully agree with this. A lot of things can change in the beginning and if you unit test from the first line of code, then you end up wasting a lot of time writing tests that will have to change based on your code as well. I don't think it's that hard to keep your edge cases in your mind in the beginning.
I say wait until your product is actively used by users or you have have people paying for your service. Then you can start covering your features with tests.
I always start my development with a test-driven approach. That mentioned, I aim for 100 % test coverage for the business critical & core features.
Unit tests are my go-to method of testing.
Unit test any class that implements real, complex logic. I think of unit tests like the scaffolding that construction workers put up around a building when they're renovating - tests allow you to get to those tough-to-reach places and look through windows that aren't visible from the ground floor.
To start, I recommend only e2e testing critical flows - the most important paths that the majority of your customers go through (log in, post a comment). Sort of an 80-20 rule. 80% of your users only use 20% of your features, so only e2e test those.
Testing beyond that is testing for the sake of testing, and Indie Hackers can't afford that - spend that time on marketing!!
Quick plug for ParrotQA.com - e2e testing of critical user paths doesn't have to be painful. Build, run, and fix Selenium tests without writing a line of code.
I just add some tests around payment. Rest can wait until I have people using said payment.
Testing is not just about ensuring everything is working as expected but rather about ensuring that as less time as possible is wasted on fixing annoying bugs in the future.
For this reason, I'm always writing tests from the first line of code, the time I spend on it is usually minor compared to the time it saves me!
Mostly unit and integration tests.
E2E (when prototyping) might give you more headache then confidence so I tend to avoid those until a later stage.
Another important question is what to test but I believe it needs a context in order to be answered so just something to think about..