Understanding Variance Through a Simple Coin Toss Experiment

We all know that a coin flip has a 50-50 chance, right? But would you be surprised if you tossed a coin 10 times and got a result of 8 heads and 2 tails?

Would you assume the coin is biased? Or is it just luck?

Probably just luck, right?

Here’s the interesting part: if you toss a coin 10 times, there’s almost a ~9% chance the results will lean heavily (like 8-2) in favor of heads or tails.

So, what does this have to do with A/A testing?

In an A/A test, you're testing the same design, and theoretically, the win rate should be 50-50. But if your sample size is small—say, just 10-20 conversions—don’t be surprised if the results look skewed.

It’s not your tool. It’s not magic. It’s just variance doing its thing.

Before jumping to conclusions, know that the smaller your sample size, the higher the chances of random fluctuations.

Always account for variance in your testing process.

If you're looking for a lightweight A/B testing tool that focus on better website performance, check out my project Mida.so!