2
5 Comments

A few questions about A/B testing

by Matthew Morse

Hey all,

I had a few questions about setting up A/B testing. I can handle the dev side of things, just need help with the strategy.

  1. How do you handle a return customer?
    For example, say I'm testing a price change and a user gets one price based on a link they clicked, but then navigate to the website directly and then they don't see their original price/change.
    I could set up a cookie for them so that the site is the same way every time.
    Is there a standard way to handle this?

  2. How do you encode your unique URLs so that you can tell what you did in that version, but the user can't tell by looking at the URL? I'm assuming you can just use a random letter/number generator and make a note somewhere what that code means.

posted to
Marketing
 on November 28, 2020
Say something nice to morseCode…
Post Comment
  1. 2
    1. You would need to stick the user to a specific version and as long as the test is running they should get the same variation of the page.
    2. I think you can do this in a ton of different ways. event 1 or 2 would work out because nobody knows what is what

    I would recommend using a tool that is made for a/b testing that takes care of all of this!
    Explainer: I'm working on Splitbee.io an analytics and A/B testing tool :)

    tobiaslins
    ·
    3 years ago
    ·
    Reply
    1. 1

      Once the test is over then all users get pushed to the "winner" or do some of the users stay grandfathered in the old price? I'm assuming everyone gets the update.

      morseCode
      ·
      3 years ago
      ·
      Reply
  2. 2

    Great questions - following to see if I can also learn the answers :D

    tulika19
    ·
    3 years ago
    ·
    Reply
  3. 2

    AB testing pricing is hard work.

    Do you have the scale to make sure the tests are significant?

    I think you would need 100s of paying customers during the test.

    volkandkaya
    ·
    3 years ago
    ·
    Reply
    1. 1

      Definitely not, but that's good to know that I need to wait a bit before I start testing that. Thanks

      morseCode
      ·
      3 years ago
      ·
      Reply
Trending on Indie Hackers
Getting first 908 Paid Signups by Spending $353 ONLY. 24 comments I talked to 8 SaaS founders, these are the most common SaaS tools they use 20 comments What are your cold outreach conversion rates? Top 3 Metrics And Benchmarks To Track 19 comments How I Sourced 60% of Customers From Linkedin, Organically 12 comments Hero Section Copywriting Framework that Converts 3x 12 comments Join our AI video tool demo, get a cool video back! 12 comments