No winner for A/B Test

[Anonymous]

Hello,

For my first A/B test, I decided to send winning email automatically after a certain amount of time (24 hours). What happens if there is no winner? My critieria was click to open % and my test population may have been too small.

If there is no winner, I assume Marketo will not send to the rest. What do I do to send the rest of the list the email?

Thanks for all your help!!
 

[Dory_Viscoglio]

Hi Erica, I wouldn't assume that Marketo won't arbitrarily send one as the winner based on previous experiences. If it's possible, I'd wait until your test is scheduled to end, and if it doesn't send a winner you can set up a smart campaign to the same audience except add an additional AND filter that says "not was sent email: email x or email y". This way, you won't send to people who already received either email in the test.

But like I said, I think it will most likely send one of your emails as the winner. 

[Tara_Petre · Accepted solution]

In the past, I've found that the first email in an A/B test was sent to the remaining recipients in the case of a tie.

[Kristen_Malkov1]

Agreed, I've had the same experience as Tara but only a handful of times. Unless you're only changing something SUPER minor, the winner should be sent. Although I would try and set up future tests to run a bit longer than 24 hrs. Perhaps 48-72 if you can! People really do tend to open emails after a few days (depending on your businesses and send dates). 

 

[Josh_Hill13]

Agree with above, you must let it wait 48-72 hours as Kristen says before you have actionable data.

Also, try to have at least 300 people in the test sample. Even though 30 would be the rule of thumb, you likely won't get useful data from a 30 person sample. I posted a statistical significance calculator elsewhere, but you can do a search for one online.

[Scott_McKeighe2]

For reference, here's a great significance calculator for your toolbox:
http://getdatadriven.com/ab-significance-test

[Anonymous]

Thank you all for your insight. It is all very helpful. In the event of a tie, the first test in the A/B test is declared the winner. Thank you.