Hello everyone.
I, as a developer, don't like writing tests. It's b-o-o-o-ring, and I can't ship my awesome features while writing them. But testing is essential for your sleep. Because if something is broken, your customers will call you, even at night. They will be furious. And you'll need to fix it immediately, because “I pay you money, that's why”.
So I decided, if developers don't like writing test, let's outsource this to somebody, who at least tolerate this activity.
And so I created: https://helpmetest.app/
Questions:
I think it depends on which tests we're talking about. The biggest amount of effort is spent on the unit tests in any software project (if there's any!). And ideally unit tests need to be updated with every single line of code that's being added, again ideally even before you write the actual code if you're following TDD. So it's simply not possible to outsource unit tests as they're a part of the project itself and every software developer has to be able to write them along with the actual live code.
If we're talking about other kinds of tests, like integration, stress and load tests, it might be possible to outsource them but because the biggest amount of effort is spent in unit tests, they're kind of secondary and wouldn't be quite beneficial to outsource them in my opinion.
Hello Ender
It would be E2E tests based on browser automation. More like testing that pages open, and buttons are clickable. Basically testing minimum user path.
Here you can see how we test themselves: https://helpmetest.helpmetest.app/
I see thanks for clarification @slavaGanzin. It could be beneficial in that case, I have a friend who's been working on a similar project but for automation rather than outsourcing. Wish you success in your endeavour.
Im using Django for backend development and React for the forntend. Every time when I add a new endpoint in the server I also add unit test cases for it, so I know exectly return codes, data and error messages. It makes frontend development much faster.
Piotr, that great attitude. Keep going!
But I worked for multimillion companies who didn't care at all about testing. They have mentality: when something is broken, but we don't loose money on that, we didn't fix it.
And for such companies it would be easier to outsource, than to change this broken mentality internally.
Honestly, I see a tremendous value with writing unit tests. When we update an existing feature or add a new core feature, chances are that things are going to break. Unit tests help me identify those right away rather than in production.
That way I make sure that broken code is not deployed anywhere and customers won't call me at night even when they are paying me.
Pros of outsourcing writing unit test-
Cons-
Pritesh, I agree that tests add stability to development process, that's for sure.
About your points:
Overall, thanks for detailed review!
As an indie-hacker, tests rarely make sense. The product is changing so fast, that keeping the tests up to date and relevant would require more work than actually building the product. Plus, with a small team things are less likely to break as everyone knows the codebase really well.
As @rajasimon said and I agree, tests make more sense when we hit the MVP. If we are iterating the code base very often, I believe that doesn't make sense for real.
What we want to achieve is to bring additional stability when you'll hit the MVP stage. Of course, writing test before is insane: everything changing too fast.
My experience is the exact opposite. I am the sole developer of my product (so I know the code extremely well) and my tests have saved me from disaster many times.
Total truth here. This year I started doing tests for the products I built and it made me feel much more secure about the stability of them all + I got also saved many times. Otherwise, I have two past projects with 100k+ users that gave me a lot of headaches in their lifetime that could be avoided with tests.
Based on reviews indie hackers are not my target customers. Ok then I’ll try others
Thanks for your comment
Storybook + Chromatic allowed me to test my (vue) frontend and actually increase my development velocity. That's what testing should be like
will check this solutions, thanks
IMO for a one person team, unit testing should be used for critical aspects of your app (e.g. payments). You don't want your customers losing money for no reason.
But other than very critical aspects, I don't really think its necessary and can get in the way of developing other aspects of your app that rely too heavily on the tests to pass.
With that said, unit testing is a good way to jog your memory as well (you can write unit tests as a way to walk through how you built specific aspects of your app and iterate on it).
It's not clear what exactly you offer.
Automated tests?
Unit tests?
Manual testing?
How does it work? How much does it cost?
Totally agree.
We offer writing and running automated browser tests. When tests fail you’ll get an email. And also you will have status page like this https://helpmetest.helpmetest.app/
Prices would be around 50$ a month for running, and 10$ for writing a single test
The problem is that in order for someone else to write my tests, I need to communicate to them what the app's desired behavior is. And I hate communication even more..
We are talking about testing user path. So it can be inferred from site itself
Hi,
A few thoughts form me, apologies if some have been mentioned before.
Hope that helps for now.
Steve
Thanks for response.
No problem.
One of my developer mentors once told me a feature is 40% feature specific code and 60% tests. I believe that premise to be true. Tests is a must have. I'm not a stranger to outsourcing it though.
Interesting thought!
It definitely has it's positives!
I wouldn’t use it because my mentality is coders should write their own unit tests (so even having someone else on your team do it is not good let alone another company)
A slight pivot would be a testing company to find new bugs via manual testing or review your testing code to look for gaps.
My feedback on the site is there is nothing there to make me think you grok testing. I would expect to see someone like “I have been coding for x years and written tests in Jasmine, Selenium, Mocha and NUnit. I have refactored large test suites and made them twice as fast” or similar (obviously based on your experience doesn’t have to be exactly this) with some social proof (maybe colleagues?) . Also double check all your grammar!
Based on lots of reviews, we didn't describe that:
Will work on that. Thanks for review.
You can play with our own dashboard: https://helpmetest.helpmetest.app/
Tests bring peace of mind. You will sleep much better a year from now, and you will thank yourself someday.
And I want to bring this realization to not just developers, but to product owners!
I can't say I write many tests during indie hacking. At work, however, it is heavily implemented. I think once you grow to a certain amount of users and the complexity of your application it is inevitable to start using tests.
I actually think that for MVP building they are not a MUST and in some cases contribute to the overpolishing and overengineering that some of us fall victim to. I say, if you have a very complex piece of code that is business critical test it from the start otherwise i think you will be fine.
Great Idea but I thought the tests were code tests like unit tests haha!
A lot of people in this thread decided that's about unit tests. And we are about browser automation tests.
Why do you think everybody here decided that my idea is about unit test? What did I articulate wrong? Can you describe your experience?
I would say change the copy of ur site to focus on browers automation first, than test.
Right, will do
For our MVP, we didn’t write any tests. Once we start scaling, tests will be very important and we would love to use your service. Look for tech startups that are funded but early in their scale process.
This is a great idea!
Thanks, Christopher. Yes, I think post-MVP startups should be target customers for my project.
Could do a pilot with you. Totally free, just to test our ideas. Are we talking about https://www.getvocal.app/?
Yes! Lets do it. Could you book time using our "Request Demo" page?
I will connect with you after I fix some issues on backend side, so you will not be disappointed!
I look forward to it :)
Do not trust anything you read about tests online. Its like googling "should I exercise?" "Should I eat healthy?"
Online you will read the things people say they do, while in the real world you will see the things people actually do.
I saw a lot of big companies who test nothing and just fixing bugs after customers complain. So you are right, everybody lies.
I hate them really..
Yeah, but it's better than angry customers
If you're biasing the poll by only providing negative options, you're setting yourself up for confirmation bias.
I've been doing Test-Driven-Development for 15 years. It's a major (major) time saver if you think long-term. Tests allow you to refactor, which is invaluable as your code base evolves over time. They also allow onboarding new team members more easily, because they can feel free to refactor without silently breaking things.
Tests should not be an after throught, or outsourced. You can outsource acceptance testing, but as soon as a bug was found, you should write an automated test that reproduces it, then fix it.
There are are some good articles on the advantages of TDD:
You right I totally forgot to add option "I write my tests myself and I like it". But I couldn't do it now.
"You can outsource acceptance testing" that's what we do. You are right, testing is more developer responsibility. So we are just trying to help find bugs and check overall stability.
@slavaGanzin given the comments I read here, I thought that maybe it would be a good idea to explain that your tests (correct me if I'm wrong) will be more on an e2e level. Because you probably will not outsource unit tests, right? It would require access to the codebase, no? Otherwise, automated e2e tests seem a loveable idea. I would love to test it.
Abraao, you are right. I will communicate this more clearly.
You could test it, just create an account, it's free.
I have done that but honestly I couldn't understand it very well. In that big text editor at the end of the journey supposed to be filled with javascript code?
Text editor should be filled with https://robotframework-browser.org/ code. It’s automation language close to Gherkin.
But I don’t want users to write tests. So user journey should end on scheduling a call with testing specialist. Who will write tests after discussing what need to be tested.
You couldn’t understand flow, because there is no onboarding right now. My bad, will fix this
I like the path you are following with the UI. On the UX however, I think you need to improve feedback on clicked buttons. I clicked twice on the button that appears on helpmetest/step-2 and it sent me two e-mails. Gotta make it disabled and show a loading.
Despite this and things I saw other people saying here already, I think this idea might have a market fit! It's very common in my experience to see developers that don't like to test. But we all know that we need it, especially when our paying customers hit the door. (Oh man, I understand this frustration 🥲)
UX is a bit buggy, will fix it. Thanks, for pointing out.
Thanks for believing in us!
This is a good landing page @slavaGanzin, aesthetic for sure.
Some grammatical nits (lmao forgive me, couldn't help it):
---
I don't hate tests, but for my side project, I only write unit tests for a small part of business-critical parts of the codebase.
Once I reach ~10k ARR I think I'll write a couple of E2E tests too (or maybe use your service 😉). Before such a point, as other IHs have mentioned, the time investment isn't really justifiable.
Thanks, Gustav, we tried to make our LP a bit funky. And I'm happy you like it.
Will fix that text errors, thanks for pointing! Besides, one of the features I'm planing is to test our client pages for grammatical errors
We'll be happy to work with you on E2E tests.
I never did that in my previous five failed projects but the current one I do it a lot. a) it's because my recent contract job I need to write a lot of tests and I get now why it's important b) I done with my MVP so every time I ship new features to the existing MVP I added a new test case.
Just ship the code without tests if you're building it from scratch. Once you hit the MVP mark then write some tests to ensure the intended functionality works.
There's an excellent lecture where Uncle Bob explains why creating a project just to add tests is doomed to fail:
https://youtu.be/BSaAMQVq01E?t=4542
In the same video, a little bit later, he explains the best thing about TDD is that it makes tests less boring.
https://youtu.be/BSaAMQVq01E?t=5759
Overall, all your videos are super interesting and I definitely recommend watching them.
Thanks for videos!
I hate writing tests but not sure its so easy to outsource since it really requires both an understanding of the code and the business (to know what to cover by tests). Feels like writing tests goes hand in hand with building the actual features.
If we are talking about unit tests you are right. But what we are doing is browser automation testing, more like emulating customer and checking his minimum user path.
Still probably would need to understand the underlying code etc unless you are only talking about extremly basic generic testing of for example log in functionality.
It's more about generic testing. Like emulating user clicking here, fiiling up form there...
Hi!
From my experience, I've never seen test outsourcing,
Usually writing tests depend on the company stage but personally, I would write tests when I have actual users and in critical places in the application, and usually, it is not that hard and doesn't take too much time for me at least.
I wonder if I would like to outsource Tests it could take much more time from me rather than just writing my own test. 🤔
It’s not that hard for coder. But for no-code business owner it’s a new territory. They even don’t know they need this. And here I think would be my biggest challenge
But what I am building is actually test platform: we don’t only write test, we run them. So you don’t need to run your own infrastructure.
Even for something small, right now, like Sekker I think you would benefit if you will get emails when your LP is broken or some test didn’t collect data
Thanks for your reply
Interesting to see how it will go :)
What kind of tests are you planning to write ? You also plan to write each client tests manually ?
We are writing automated browser tests. Like emulating user clicking everything, filling forms, etc
For now we will write tests manually, but in future I plan to automate this a little.
Something similar to ghost inspector ?
Kinda, but more flexible
Outsourcing will slowdown only if you are familiar with writing your own. But I think my perfect client doesn’t want to even try
I think the idea has potential, if I am building a website with login, subscription, search etc ... These general functionalities are known and relatively easy to test, and above all testing is a must.
As for more sophisticated systems, I believe the best tester is the engineer who built it, or at least a testing team that is very close to the engineering team.
Having said that, the idea is plausible and needs a try.
I propose testing basic features, the user minimum viable path: login, buy product, fix typos, basic API. Internal logic would be hard to define and discuss, that’s for sure.
Thanks for your replay. It something to think about.
hello @slavaGanzin, i am actually working on creating similar app, i am at a phase of collecting workforce, please let me know if you want to collaborate, thanks
What do you mean when you talk about collaborating?
This comment was deleted 5 months ago.
This comment was deleted 2 years ago.
We are writing tests using Robot Framework, it's a Gherkin style tests. So it's more like declarations of what should work, like:
Go To http://coca-cola.com
Page Should Contain Coca Cola
and like that
This comment was deleted 2 years ago.
I think a margin problem is real, so I'm testing waters.
You are definitely not my customer, and I'm not mine too. For us it's easy to write our own solutions, but for others... let's see
Thanks, Shane!