5
5 Comments

Do you write e2e automation tests?

I am interested if you write automation tests(Selenium, Cypress etc.) and what issues have you encountered?
Did you have issues maintaining it, scaling it etc.

Thanks in advance!

on August 26, 2020
  1. 2

    for my old job I did a lot of e2e for react native using wix/detox.
    main issues:

    1. it's very easy to break since its sensitive to UI changes
    2. I spent a lot of time debugging the tests themselves (finding the element etc)
  2. 1

    I love the idea of e2e tests - it feels really important to test applications from the perspective of an actual user. But, in the past I've used Selenium and found e2e tests suites to be extremely flakey and hard to maintain.

    I started using Cypress this year and love it - I think it's a great product and those tests have been much more reliable.

    Issues / challenges I've encountered:

    • Seeding test data - you're interacting with your app and need specific data in place to run your flows. We've had to write scripts that are executed on the backend to seed the data as part of our CI process.
    • Visual regression testing - it's surprisingly hard to just quickly verify everything looks right on the screen. There are tools to do screenshot comparisons but they can be very flaky particularly when run on different envs (dev vs staging / test).
  3. 1

    We've been using selenium at my day job for a while but abandoned it since it took too much time to maintain and debug tests. Especially timing was an issue, e.g. looking for an element after page transitions, events etc. was a pain.

    I've recently started looking into alternatives such as Cypress and TestCafe and at least getting started seemed easy enough with those. I've written only a couple of tests so far but I can say that writing tests is a lot easier than with Selenium and tests a lot harder to break them (unless you're constantly changing element ids and classes)

  4. 1

    end to end is great for the most critical happy paths, but as others have said is very time heavy and resource intensive. At a big energy company we decided on 4 critical end to end happy paths we absolutely needed, and we focused on them.

    We then had countless integration tests and even more unit tests.

    The unit helps understand the micro
    The integration helps understand that the unit tests flow correctly
    The end to end in my view gives a sanity check that everything is great, but good error reporting and gradual roll-outs work too (not ideal, but it's true on a macro scale!)

  5. 1

    As an E in ent. I do a lot of e2e testing, but usually only work on back-end. I tried using Selenium in the past (years ago) and it was really hard to start, get right and poor in functionality.
    A platform for visual automation I love is ubot studio... used to do a lot of scrapping and automation in it, can easily do testing if you want (not it's main purpose) ... (didn't touch it for a few years, but would happily pickup again)
    As an indie working on MVP I'm a little upset at myself dealing with testing and similar before getting the main thing done and launched.. (due I'm on the razor edge with like I needed some of this to reduce cognitive load)