Testing on solo developer projects - How?
As a solo freelancer dev I often work on projects that has no resources for a dedicated QA. Usually it is me and my client, who are doing testing. For a time it works. But if a project keeps going for 3-4-5 months - such testing starts failing.
To help with it I usually create online doc with checkboxes to keep record of things to test, scenarios, special cases, etc. Trying to keep it simple to not spend too much time on supporting it, but useful.
Guys, please share your approach to the problem. Do you spend time on more formal test case? Is it worth spending time on unit-tests on smaller projects? Especially if we are talking about simpler project where most of the code is UI management. Should we let users do testing for us :)? What tools do you use? Every specialized tool I tried seemed too much trouble.
When I am working on a project solo or in a small team, I usually test by priority. Where I measure priority on the basis of:
So I write extensive tests(e&e and unit) for mission-critical functionality of the application and then slowly proceed to other parts of the application. Since I mostly work with startups, there is always a bit of a tension between writing extensive tests and working on new functionality. Striking the right balance is hard and also depends on the stage of the product you are working on.
Besides testing, you should definitely invest in a fast triage & deployment process. That way, even if some bugs make it into production. If you can detect and solve them quickly, their impact is minimized. Afterwards, you can add tests to prevent the bug from occurring in the future. I call this approach BDD: Bug driven development 😉. But of course prevention is better than cure.
Good point on fast triage & deployment process. Thats another way to approach this problem from another angle. It definitely makes this problem less of a problem.
Those less important parts/requirements that are not covered with e2e/unit test, how do you track those? Memorize? Tracking in some doc?
I use a kanban board. Instead of a done column I have a column for functionality which is done but does not have automated tests. Priority is clear from the order of the cards. So on top is the most important item for which I will add in tests. You could probably use a google spreadsheet or doc as well. I think your approach with the checkboxes in a document also works nicely since you can go more in depth on which elements of a process you want to test.
I've built projects without tests thinking it won't be worth it and just slow me down... Not gonna do that again.
The reason is that once it gets bigger, you run into all those subtle bugs that slow you down a lot. And you keep telling yourself that you'll just add a bunch of tests later in one go, but let's be real, who's ever gonna do that. Besides, lots of stuff might have to be refactored to actually be testable, which is a huge pain.
What works for me is doing selective TDD. It makes me narrow down on functionality, and also creates more testable code, which is usually better code. Selective means I try to test all business logic functions fully as unit tests, and if it's too hard to write a test case that usually means my approach is way too complex.
Regarding your point that most logic is on UI side, I try to keep all logic outside of the component and have all logic abstracted into either hooks(React & co) or functions that are then easily testable.
If you want to test component logic, Cypress and Playwright now both offer separate component testing which is great since you don't have to render it in a whole page. It's also quicker than using stuff like JS-dom testing development in my opinion, but that's a preference.
I don't test stuff like component x renders y when passing in those props, as this is something I'll usually spot myself while working with the app.
Before launching, I will add proper e2e tests, cross browser, for all the main business functionality. Like user can log in, create X, modify Y, etc. That way I'm sure it works on all browsers and the functionality is covered from the user perspective. I tend to use bigger user stories that capture a whole workflow, so I don't need too many small tests.
The reason why those tests are only written at the end is because they tend to break all the time when features change during dev. But they will give you the confidence that the stuff actually works. They should be written implementation-agnostic so they don't blow up when the content doesn't change, but the implementation does.
I explained this poorly. What I meant is that often real "business logic" is like 10-15% of code. And most likely it will be on API level. Much bigger part of code is UI logic: populating forms with data, things like validations, form field dependencies, disabling/enabling controls, etc.
e2e seems promising for this type of logic. My hope is that they are not time consuming. I'm diving deeper into e2e now, for some reason I completely ignored it before :)
Good point. Thats exactly what happens a lot. Smaller projects are more dynamic comparing to bigger/enterprise projects I worked before. And that is another reason I tend to not invest into automated tests.
e2e can be time consuming for 2 reasons:
Therefore I would say, properly test the business logic first.
Testing forms, what will be enabled/disabled, I would first ask myself will that really break my app or just be a minor inconvenience for now? If it's not too big of a deal, you can skip those tests for now.
I'd rather have the test on the API side of things that it doesn't blow up / return wrong data if the form was submitted in a wrong way. Since the backend should never trust the frontend, that should be the way to go anyway.
If you test forms, make sure to have tests that aren't too brittle when slight implementation changes. The above mentioned component testing would probably work best for that.
I completely understand the struggle of managing testing for software projects as a bootstrapper and as an engineering manager. I've been there myself, trying to keep track of all the different tests and scenarios, prioritizing them and seeing which ones make the most sense to actually built out. It can be time-consuming and difficult to ensure that your tests adequately cover all possible scenarios needed.
That's why I built SoftwareTesting.ai . As a bootstrapper / indiedev + engineering manager myself, I wanted to create a tool that would make the process of testing software faster and easier. With this AI-powered tool, you can generate unit, end-to-end, API, and performance tests quickly and easily.
If you think SoftwareTesting.ai could help you please sign up! We are currently in free beta and looking for feedback 😊
I love doing the classic test in production. I often share it with the WBE Space Folks and see if they find any bug XD
I would say, that first get a basic version out. And after that devote time to write tests. Writing tests is only good if your project is growing actively. And by growing, I mean more code is being added
Realistically chances of this happening are very low.
I'm trying to deal with the problem by writing down informal checklists in plain English for manual testing by me or client. Wondering if other people do it too.
If you have to manually test every time after a few changes, this will get quickly get out of hand. Automated testing will save you some time in the long run.
So it seems the reasonable approach is to start with checklists for manual testing to not miss some little detail, special scenarios or edge cases (thats what I do now). And then as some sections become more stable - cover them with automated tests bit by bit.