1.) Choose Add A/B Test to your email:
2.) Select what you want to test
3.) Choose Winner Criteria:
4.) Schedule the Send Test and Send Winner and select FINISH:
Thats it! Click here access a walk through of how to send Add an A/B Test. Stay tuned to learn about lead scoring!
Completely agree with the above. Silly that we have to wait four hours for the winner to be sent. We should be able to send the winner after 1 hour, we are sacrificing email send time in order to perform the test. This can skew click/open rates as a result.
The four hour test window I agree is too long when you are targeting a wide geography and only have a small window to reach all the countries in business hrs for example - APAC - 1st March 2016 - 4pm in Auckland is 8.30am in Mumbai. We don't have 4 hrs of business hrs left in Auckland for the test to run and we don't want to run the test so early that leads in Mumbai have not woken yet.
I found a workaround to the 4 hour winner wait time if you don't mind declaring a winner yourself. If you do an A/B test with 100% of the audience (I guess you could do a smaller sample size if you wanted, but 100% is more statistically significant) and select the winners manually instead of automatically, the test will be sent to the entire audience and go out all at once. The downside is you won't send the winning content in real time, but you will be able to look at results and apply the logic to future emails.
I have a question regarding test sample size. If you have a list of 100K and you want to do an A/B email test, and you set the test sample size at 50%, does this mean 50K will receive the A/B test, and then once a winner is determined, the remaining 50K will receive the winner? Or does setting the sample size to 50% mean that 25K will receive the A/B test and 25K will receive the winner?
Hi Diane,
This means that 50k will actually receive the test, and then when the winner is decided (either automatically or manually), the remaining 50k will receive the winner.
-Mike
Hi,
Do you know if we mark the test sample size as 100% it would send the two emails evenly? Just so we can see how the two emails compare with each other?
Hi Lisa,
Yes that will work. You will just have to sacrifice some click/open rate though.
-Mike
Hi Mike,
Thanks for the speedy response! Can you please clarify what you mean by "sacrifice some click/open rate"? Would it not show any results in my click/ open rate? Sorry, this is my first time doing A/B testing.
Hi Lisa,
What I meant by that is that is the way Marketo has this set up, the test does not run for the entire email list you are emailing unless you choose 100% sample, which is what you are planning to do.
If you chose 50% sample, then half your list would get emailed the test (25% A/ 25% B) and then 4 hours later, the other 50% would get the winner, which should have a higher click/open rate. That is what I meant by sacrifice open/click rate.
My problem with the above is that with a 4 hour time difference, we are also testing time of delivery as well, but that is the way Marketo does it.
-Mike
Ah yes, I get it. Thanks for your help Mike!
Why does the program require you to wait 4 hours to send the winner? If the winner is declared before four hours, it should automatically send the winner.