Would you be interested in testing/monitoring SAAS that allows you to schedule your tests?
I have launched a couple of my own projects just for fun & learning.
While launching the product I want to know that it runs without any issues 24/7. However, I don’t want to spend much time setting up the testing environment, writing tests & creating some notification mechanism that notifies me when any of my services has failed.
I was thinking about creating a project that would allow:
- to easily set up test cases
- set up API endpoints testing with NO CODE
- set up UI test cases
- to trigger the tests after each release
- to check response times
- set up notification channels like email, slack, or phone through integration with pagerduty/opsgenie
- set up a run schedule (to allow tests to run each x minutes/hours)
- check the history of the tests and see what part of the system requires your attention the most
I’m a big testing advocate but I don’t see the benefit of this as a product. Unit testing and integration testing are done in a tdd style and will run during a deployment whilst end to end testing or smoke testing can be setup on a scheduled pipeline with a webhook in less than an hour.
If you would have a microservices/nanoservices architecture how would you structure your integration tests?
Would you keep them per service or you would create a separate project for tests?
How easy it would be to know the whole system overview and compare it with the past?
P.S. looks like you have some nice youtube videos on the testing topic
There’s many ways to do it but I favour testing contracts over integrations. I think that when I want to run an integration test there are multiple “thoughts” about the best way to do it. For instance, I would write an integration test by mocking out any external service, so that my integration test only checks the quality of my current service. So tests are always scoped to the service. The reasoning for this is that my service shouldn’t be concerned about what the other service does or how it handles things, all I need to know is how I call it etc.
If I was running end to end tests on critical services, then I would probably have them in a separate project)
I don't do testing unless it's a really big project.
Do you usually work on the project alone or with a team?
I work in a team of 15-20 people. We have a SASS product we have recently reached a million transactions per month. It has around 450+ API endpoints in 7 microservices and still doesn't have any tests implemented. We have Sentry setup for the bug tracking we receive around 2-3 bugs on every release but that's it.
So it really depends upon the projects and the team.
I’m curious to what you think a “big project” is if this isn’t big enough to test 😃
I use unit tests and integration tests to ensure my apps work on deploy. But I don’t see the need to test them after a change set has been released. What’s the benefit?
Unit and integration tests are great ways to ensure quality before release. However running scheduled tests allows you to notice if something has stopped working. Let's say one of the services got an unexpected input and is crashing or your hosting has some issues (restarted).
Running a scheduled test would allow you to notice that the service would not be able to respond.
Yes, these scheduled smoke tests gives you some assurance that your systems are still up and running.