1
5 Comments

How do you hand off testing environments to QA?

We are using heroku. There are multiple services for the app. Every time we work on a feature / bug, open a PR, and spin up multiple Heroku dynos (each dyno for a microservice to mirror the production setup).

Fortunately the process is automated. With Github integrated with Heroku, handing off the testing environment is a matter of sending dyno URL to QA, and let QA to verify the code changes there.

However, there are some pain points

  • Heroku is slow to spin up dynos and deploy code
  • Heroku board UI is not friendly for searching
  • Some bugs are in the infrastructure, e.g., Heroku Kafka. Besides deploying code changes, there are many steps to do in order to set the system up to a certain state. Some are not easy to automate

Heroku helps a lot in terms of automation with this process, but we still run into many issues.

My questions for indie hackers are:

  • Did you run into similar issues with Heroku?
  • How do you hand off your testing environment to QA? Any pain points there?
  • What alternatives do we have?
  1. 1

    Both of the answers you have are great. What I would recommend is, if your app into overly complex and you're pumping out minimal features, a shared QA environment is fine. All branches can be merged to this staging or "release" branch for testing.

    If your team is larger and you want to silo into different feature branches and feature environments (that are ideally short lived), I'd recommend moving your application infrastructure over to docker.

    It's easy to then build up and destroy a bunch of containers for each feature that replicate your production environment exactly. You only pay for the length of time the branch is being tested.

    1. 1

      Heroku is based on docker. Pretty slow to build when the system is complex, unfortunately.

  2. 1

    I've primarily used a similar feature server approach, but another one I've used in the past that can work quite well (depending on the size of the team) is one stable QA environment for everyone. If the team isn't overly large, you can have a singular QA environment that largely mirrors production and when PRs are reviewed, they get merged into the staging environment branch first (which QA is built off), at which point QA would test and eventually merge it into the main master branch. Another option is to have PRs build to that environment automatically, but that gets a little trickier in terms of isolating work done in QA.

    In any case, the benefit of the singular QA environment is ease of deployment (only need to deploy components affected by the change), you get an additional environment everyone can use to test stuff outside of production, and you would potentially save quite a bit of infrastructure costs. It's not for every team and it does take some planning, but if the right strategy is in place, a singular QA environment can work quite well.

    1. 1

      Yeah I used the approach too. Works quite well in a small team. For larger teams (>10) we have to constantly fight for the availability of the servers.

  3. 1

    This comment was deleted 3 years ago.

    1. 1

      Thanks @ppalevsky. I see two strategies in your comment:

      • A shared QA environment (so as to spin up less on Heroku)
      • Automated tests (so as to less work on QA)

      Unfortunately, for legacy systems that are not completely test covered and are still evolving, it is not feasible until we have a comprehensive test suite. For a certain time we have to live with human knowledge to test the tricky edge test cases. 🤷‍♂️

      But automated testing is definitely a long term goal.

Trending on Indie Hackers
How I grew a side project to 100k Unique Visitors in 7 days with 0 audience 49 comments Competing with Product Hunt: a month later 33 comments Why do you hate marketing? 29 comments My Top 20 Free Tools That I Use Everyday as an Indie Hacker 18 comments $15k revenues in <4 months as a solopreneur 14 comments Use Your Product 13 comments