Using Twitter ads to A/B test copy
I had a few landing page copy options in mind that I thought might resonate with my product's users. The problem is that I don't have enough traffic on my landing page to A/B test on the landing page.
So I thought I would try Twitter ads to AB test a few options.
I created three different ads with the following copy in them. Everything else remained identical.
Option A: You start small, ramping up your AWS usage over time. Eventually, your AWS bill shocks you. Billgist sends you a daily snapshot of your usage. Don't let your AWS bill surprise you.
Option B: Your account's financial owner doesn't have visibility into your AWS usage. The end of the month AWS bill shocks them. Billgist sends a daily snapshot of your usage. Don't let your AWS bill surprise you.
Option C: Your developer is experimenting with new services without understanding how it impacts your bill. Your AWS bill surprises you both. Billgist sends a daily snapshot of your usage. Don't let your AWS bill surprise you.
I then ran the ads for a few days and got the following results.
Test A
Impressions: 5,223
Clicks: 16
Click rate: 0.30%
Test B
Impressions: 3,071
Clicks: 5
Click rate: 0.16%
Test C
Impressions: 2,105
Clicks: 1
Click rate: 0.05%
Then I plugged in the values in this ab test calculator.
The results: Test "A" converted 545% better than Test "C". I am 100% certain that the changes in Test "A" will improve your conversion rate. Your A/B test is statistically significant!
Curious if any of you have done anything similar. Would love to hear feedback on whether this seems reasonable or if I can improve my tests a bit more.
I did something like that with Google Ads, but it was more for validating my idea than testing a copy. My guess is that it depends on whom you target. GA is for people who are actively looking for a solution, while Twitter might be good if people don't know that a solution (or even a problem) exists. My point is that in Twitter ads you're dealing with the top of the funnel, you need to convince people to stop doomscrolling and visit your site, while on your landing page you're dealing with people who are already interested in your solution, and you need to convince them to at least try it. Normally, these two would require different kinds of copy. But at the early stage, it's probably fine to have the same copy in both places.
Just curious, how much do they take per click?
That's good feedback that has certainly given me something to think about. Thank you.
I set max bid to $0.30 per click. It was slow going but eventually gave me statistically significant results.