TDD software for Javascript - Why isn't there any?
Hi everyone,
As a CTO/VP R&D I've tried many times to convert my devs to full TDD (Test-driven development).
Some projects went well, others we just stayed post-code testing at best.
I find that almost anyone I know would love to do more testing, is open to the concept of TDD but finds it so friggin hard!
I searched and searched the internet and all I found were Katas, blogs, courses, etc...
I'm wondering why I can't find any software to help with TDD?
And I don't mean jest, karma, mocha, chai, etc.., which are AWESOME testing frameworks, I mean something more along the lines of Cypress that for me really helped with end-to-end testing.
Am I missing something? Is there cool software that has eluded me for some reason? Is there no such thing as software that will assist with TDD?
Cheers
Jonathan
Better tools won't get your devs into TDD, replacing them with better devs will.
TDD is still one of those things that a lot of developers consider to be something that slows them down entirely. Simply put, it takes engineers that can understand the value of TDD in terms of benefit to long term maintenance overhead to really get TDD going for you.
This isn't a problem with JS engineers, it's a problem with engineers as a whole.
Tests allow you to ensure you never have to fix the same bug twice, or make a change in one part of a system and not break something a few files over. This seems like an obvious benefit, but it really is lost on most folks.
At the very least, I'd focus on moving your devs from JS to TS to at the very least get some benefit out of strict typing that will help to minimize some issues.
Hi Josh and thanks for the advice,
I think that you're referring to a very important problem - the need for testing education and I'm all on board.
My concern is helping them take their first few faltering steps.
If you asked a baby to run a marathon after crawling, they'd probably never walk.
I'm wondering is there some software out there (or that could be created) that would help you take your first few faltering steps after years of crawling.
The baby analogy is sound, but I bet your developers aren't babies and have been around for way too long for them to be making the same amateur mistakes (avoiding writing tests being amateurish IMO).
Coming from 20 years pro development experience, most developers are mediocre at best. They don't push themselves to learn to things and are plagued by the "curse of knowledge" in terms of only being able to solve problems the same way they always have.
This is a developer problem as a whole, and would say extends far outside of TDD (or BDD or just using a fucking linter to catch mistakes).
In my experience as somebody that's TRIED to bring TDD into places. It's a nightmare at best. People will say they really want to do it / learn it, but then given the opportunity to do so will cite that it just slows them down, or my personal favorite "they wouldn't know what tests to write because they won't know how it's supposed to work until it's done".
Let that last one sink in for a moment.
Problem there is your C players can't even conceptionalize the end product. Aside from shoddy code (because of a lack of tests). It usually means projects drag on because there's a ton of rework going on / the whole thing keeps needing to be rescoped.
Not familiar with the make up of your team, of course, but if I were you, I'd collect your most experienced / senior engineers and see how they feel about it. More than likely one will be on board with starting to adopt it more at work. Solid chance they haven't because nobody else is and/or don't know where to start.
Where to start... if you already have a nice devops setup (ci/cd specifically) no reason to not include running tests in that pipeline. That pipeline can will your team if you let it. Configuring code coverage minimums at the very least to help block code from production immediately forces you to have to write tests.
Where I work now, we're BIG on tests. Started with the technical founder, and is absolutely ingrained in the culture now. We have automation, but we also police ourselves in terms of PR reviews having comments like "where's the test for this?".
In terms of side stuff, I write a ton of tests, but I don't necessarily strive for full coverage or anything. My general rule of thumb is as follows:
If I have to fix something, there should be a test to simulate the issue (first) and then code updates to flip the test from red to green.
I'd just make it a decree that any bug fixes that go out 100% have to have tests. I mean, you're the CTO in this situation, hand it down from the mountain :P
Hi Josh,
To sum up our discussion, you're 100% for tests (as I am), and you believe this is a skill that can't be aided with software, excluding the obvious testing framework and CI/CD pipeline.
I'm still trying to figure out if that's that or is there anything more to be done here.
Thanks for all the input!
Jonathan
Obviously a matter of opinion, but yes, I think that adopting good development habits isn't a problem to be solved with software, it's a fundamental problem with people in general.
Most folks go through life doing the bare minimum and expect to get ahead. You can certainly write software without tests, but that's the bare minimum you could possibly do.
This is why there are STILL people out there FTPing files around. It's the reason some folks will NEVER adopt any _____ Driven Development. It's the reason people are fighting for more minimum wage instead of clawing for better opportunities.
Your team is probably content. They probably say things like "well that's the way it's always been". That kind of stuff isn't magically fixed with software, it's fixed with attitude adjustments and a lack of tolerance for anything but success from the folks above them.
If your team isn't performing the way you want it to, that's really on you and not necessarily on them. You hired them, and you rehire them every day they come back to work. As mentioned in Principles, you need to periodically revisit whether or not you'd hire the person again today knowing everything you know about them now.
Preface: I'm super passionate about things like this and I'm often the person on our team that will speak up if a process doesn't make sense - hope I don't come across as being rude in my convictions. I absolutely think you should use it if it suits your development style! :)
I think code coverage is important, but TDD has never been a preferred methodology for myself and most developers that I've worked with - especially in environments with rapidly changing requirements. When working through a problem that you're solving, stopping to write a test before writing every single method is stifling to creativity and is counter intuitive imo. It's one of those things that's good on paper, but not so great in practice.
TDD without code review and standards soon devolves into hacking the tests just for the sake of having a test, which becomes useless code that someone has to maintain. Tests should really provide some kind of value.
For example, if I have a class
MyShinyThing, it's really pointless for me to write something likeAssert.NotNull(new MyShinyThing())just for the sake of reaching 100% code coverage - and further, it provides no value for me to write such a test before the implementation.At my last (corporate) job, management wanted 100% test coverage to check some boxes for executives and it resulted in absolutely useless tests. On paper we had good test coverage but they really didn't provide any value.
In contrast, on my current team we currently have about 85% test coverage on our platform through a mixture of integration tests (Postman, Cypress) and unit tests (xUnit, Jest), all of which run after every merge to our master branch prior to being deployed to any environment. We're able to maintain quick development of the product because we have a dedicated person specifically for this purpose. These tests provide a ton of value and if something breaks, we know we really did something wrong.
Tests are checked in in the same pull request as the implementation, but every developer writes the tests after the implementation is written - It's really hard to test an API endpoint if the endpoint doesn't exist.
TDD implies that you're writing the tests first and then making your implementation make the test pass... In a lot of cases, in my experience, this has led to "hacking the test". I think the key here is to make sure that you're writing good tests to cover things that really need to be tested. In the end, the tests are there to provide you with confidence that new changes don't cause regressions and that deploying your app will go smoothly - the sequence in which the tests are written doesn't really matter.
All of that being said, you can conduct TDD with any unit testing or integration testing framework - it's really just a process that dictates the sequence of when you write each piece of the code.
I guess this was my soap box just to say that TDD solutions probably don't exist because very few developers actually do it. Heck, most developers don't even write tests, let alone before the implementation.
I completely agree with a lot of points you discussed.
I don't remember if it was in extreme programming or clean code, but they show how clearly you can have 100% code coverage which covers 0% of the actual bugs :)
50% coverage with good tests is better than 100% coverage at all costs.
I do have a soapbox for TDD though and it's mainly to points:
I actually love your last point, that if 99% of devs don't TDD and most don't even test, then that's why there's no software for it.
Thanks for the in-depth comment!
Jonathan
You might be interested in WallabyJS. Streamlines the TDD process a bit, though, it does have a bit of a setup curve.
https://wallabyjs.com/
I remember seeing them 2-3 years ago and forgot about it completely.
I find that it's more of a repl than actually a TDD tool although it is very cool.
Do you use it?
Not too actively. It's one of those tools that when it works it's great, when it doesn't it's a burden (and I just revert to running tests in Jest/Cypress/TestCafe).
Out of curiosity, what practical long-term benefits you achieved (for real) with projects you managed to go full TDD with?
I’ve been in the industry for 10+ years and unfortunately never saw implement to a 100% degree. So really curious about how it went for you.
One project that was 100% TDD was code that I wrote alone, so I can't attest to collaboration issues.
I wrote a cross-platform DSP engine in C++ and had a LOT of data to build it upon, so I started by writing failed tests with signals expecting outcomes I knew from Matlab and then wrote the algorithm to implement them.
This project went MUCH faster because I went full TDD and didn't get lost developing things I didn't absolutely need.
Hey Jonathan!
I know it's not exactly what you're after. But the AdonisJS framework has a testing framework built in which allows using http clients AND puppeteer. So you can do a whole lot of testing: e2e, api endpoint testing, unit testing etc.
AND it also has very cool features like the ability to truncate the database between tests with just a oneliner.
Combine that with nodemon and you have yourself an amazing TDD experience!!
Write the test, save the file and see instantly how the test fails. You can even mark the config in puppeteer to make the window visible. Then write the code, save the file and see the test passing inmediately
Sounds very cool!
I've been meaning to check it out!
Not every problem can be solved by throwing more software at it. What are the outcomes you want to drive with TDD? Does your organization have set in place incentives to support those outcomes? Have you discussed with the engineering team why is it hard to write more tests or do TDD?
Hi Tomas,
I completely agree with you here!
I find that there's the "blank canvas" problem. Engineers are overwhelmed with how to start testing a unit, let alone write a test before it's even written. Cypress, for instance, really makes it easier with Time Travel and Real-Time Debugging.
You need a small number of evangelists in your organisation who would be able to prepare the setup, demonstrate the benefits and troubleshoot problems while new way of working is being adopted. It's not a great experience when you are asked to do plumbing for a completely new thing while at the same time you need to hit project deadlines.
The small team of evangelists would also help to uncover other enemies of E2E testing: issues with build pipeline, often changing requirements, too many dependencies to run the product, no clear interface between components or at the lowest level lack separation of concerns of different code units.
Lastly, make everything measurable and tied to quality of the product. Can you quantitively prove that bug count was reduced by investing more time in testing/TDD? This will help to build mindshare in other layers of the organisation and explain why it needs to be done this way to all stakeholders.
I love everything you're saying!
I feel though, that it's all in education, and let me explain what I'm aiming at: Let's take task management as an example.
If you don't educate and keep up task management, no amount of software will help you.
But if you do decide to do task management, Jira, Trello, etc... are helpful in keeping task management easier and more structured.
I'm looking for that in unit test TDD.
I feel that Cypress is an immensely helpful tool for e2e tests, but no such tool exists for unit tests.
Have you looked at code coverage analysis tools? I've worked in a TDD shop (once in 30 years) and loved it... but it took a while for me to come up to speed. I had the benefit of having a mentor in that situation that coached me. We used code coverage as our benchmark. Our goal was 80% code coverage - and all the developers could see that goal vs. actual every time they opened their IDE's.
BTW, the blank canvas problem is real. That's where really good user-stories, requirements, tasks, etc. can make a huge difference.
I love what you're saying, and I'm a huge fan of education as a way to get to testing and TDD in particular.
My thinking is whether there's software that can help with education or after you've converted.
I think software can help with TDD in the same way software can help with Agile or Scrum (or other methodologies). I have not seen the software that will teach those (and I don't know what that software would look like). The closest I come to TDD is something that automatically scaffolds a basic test (and there are tools that do that already).
I fall back to learning is a human process and not all of our processes have an "easy"-er button.
Unit tests are just code calling main code. I am still missing why lack of some intricate software would prevent developer from writing a test after they have written a unit of code.
I am completely on board that lack of education or experience can prevent from writing good tests or code itself, but this again is a human problem and not software problem.
As Eddie Izzard said - "Guns don't kill people, People kill people!" - "That's true, but I think the guns help, right?"
I'm wondering if there's a software solution that could reduce the friction of the human problem/after the human problem has been solved.