6
13 Comments

What tools do you use for A/B testing?

I'm looking to introduce A/B testing onto my landing page to increase waitlist signup conversions. Are there any specific tools you recommend? Or are there any IH users that have a startup in this area that I could use?

on December 27, 2019
  1. 2

    I'd consider firstly if you have sufficient traffic and conversions to get to a statistically significant result. For the majority of early stage IH businesses, it doesn't make sense to A/B test.

    1. 1

      Endorsed. You can use a significance calculator to understand what size of audience (or what kind of lift) you'd need for A/B to genuinely move the needle.

      https://www.optimizely.com/sample-size-calculator/
      https://vwo.com/tools/ab-test-siginficance-calculator/

  2. 1

    After trying the usual alternatives, I switched to a custom / non-standard approach that doesn't cause the performance impact and/or screen flicker, plus allows deploying completely different sites instead of only small tests:

    I have the control at the root site (/), and then the experiment under a path (/experiment).

    For testing with ads, I make a copy of each ad, exactly identical except pointing to different landing pages, so no tech hurdles. Effectively, the ad platform distributes AB testing traffic.

    For testing with direct traffic, I sometimes use AWS lambda function that randomly assigns visitors to '/' and '/experiment' URLs, but sometimes I just test with ads and let direct traffic be as normal.

    1. 1

      This is a neat approach, thanks for sharing.

      What happens if someone lands on /experiment then manually clicks back to the root path /? Do you have something to track that?

      1. 2

        Then they are confused as fuck :)

        1. 1

          Lol, fair enough :)

  3. 1

    Fellow IH user here that have a startup in this area! I'll be launching my A/B testing feature in https://howuku.com in close beta soon, would you like to try it out maybe in a couple of days?

    1. 1

      This looks great! I think I actually came across your startup through a google search looking for A/B tools.

      I'll be sure to keep you in mind, thanks!

  4. 1

    What are your requirements in terms of functions? What do you want to be able to test? Are you looking for a free or paid versions and what budget?

  5. 1

    Hey, Google Optimize is great to start with, but I personally don't like it because of its imo complicated and uses a session-centric-measurement.

    I am working on my own AB Testing solution – abrocket.io. We are testing with some users in a closed beta. If you are interested I would love to send you an invite.

    I have also experience with some other AB-Testing tools, let me know if I can help you.

    Cheers

    1. 1

      Hey, what made you create this? Did you make this to scratch your own itch or did someone ask you to create something specific?

  6. 1

    Google Optimize is a great free option to start off with A/B testing.

    1. 1

      Whats your favorite thing about Google Optimize?