What are your struggles with end-to-end testing?
Hello everybody,
I recently encountered some problems when it comes to end-to-end testing. There are a lot of great products and tools to use for that but I wasn't 100% happy with them.
My problems were mainly:
- Ease-of use, the tools often took up a lot of development time or debugging time, especially when redesigning, refactoring and fixing bugs.
- Visual Diffs - It could be that everything works well in my application, but a button is displaced on mobile after a design change and I didn't catch it.
- AUTHENTICATION - Most of the tools I used either needed a big chunk of code written for accessing closed parts of my application or didn't provide any solution at all, but instead you had to record the login steps over and over again. Creating an account and deleting it afterwards doesn't always work (e.g. Admin Tools, or frontends with simply no registration at all)
What are your struggles? What do you use and why? Do you like creating end-to-end tests or do you even do it at all?
I'm curious to hear your thoughts!
The biggest problem for me was the time required to learn new tools/frameworks, and then to code the tests. I've done a lot of research into this and discovered that a large percentage of solo/small teams simply don't write tests because it's so time intensive to set up (easier in a bigger firm with more resources).
I couldn't find a great solution, so I'm building a no-code solution: https://firelab.io - I want users to be able to set up tests quickly using a UI with minimal learning curve.
At the minute that's using CSS selectors, but I will add a Chrome extension in a future version to record steps.
Authentication is tricky for sure!
Have you tried Storybook with say lambdatest?
Like I didn't get into this yet, but was thinking of
I never used Storybook in any project yet and combining this with LambdaTest seems like it would take unnecessarily a lot of time to set up and maintain.
They natively support it (and other options)
The idea is they provide screenshot difference that you mention as a pain point
Maybe in reverse look at lamdatest and choose your own testing stack after
I am busy with a system integration and only recently been exposed to formal QA testing processes. Painful and time consuming:
rinse and repeat
This unnecessarily consumes so much time. Especially as a developer. There are much more important things to do and this process can be far more easy.
I'm using https://reflect.run/. I believe it mostly fits your criteria.
Reflect allows you to build your tests in "blocks" that can be stacked on each other, so when you need to change your test you only have to change small blocks instead of the whole thing.
Reflect has visual diffs https://reflect.run/docs/recording-tests/visual-testing/
They also handle authentication https://reflect.run/docs/overview/tests-you-should-create/. However, yes I have a bunch of random profiles on my staging site that I have to delete... it is on my to do list to figure out how to automatically get rid of those. I think I can just use reflect to delete the account in UI, but I haven't gotten that far yet 😂.
My app https://www.flowphantom.com/ is built on top of Webflow, so the purpose of using reflect is to make sure Webflow hasn't changed anything that could cause issues. Reflect.run works well for me in this case. Hopefully that helps!
I tested reflect run once, what I don't understand is why they only have the option to mark a specific place in your DOM to assert later. Why can't I just take a screenshot of the whole screen? I tried it and it's not possible, at least I haven't found a solution. Also the browser which runs in a VM is kind of disturbing sometimes, a Chrome Extension would be much easier. Thanks tho!