SOLVED

Re: No winner for A/B Test

Go to solution
Anonymous
Not applicable

No winner for A/B Test

Hello,

For my first A/B test, I decided to send winning email automatically after a certain amount of time (24 hours). What happens if there is no winner? My critieria was click to open % and my test population may have been too small.

If there is no winner, I assume Marketo will not send to the rest. What do I do to send the rest of the list the email?

Thanks for all your help!!
 
Tags (1)
1 ACCEPTED SOLUTION

Accepted Solutions
Tara_Petre
Level 5 - Champion Alumni

Re: No winner for A/B Test

In the past, I've found that the first email in an A/B test was sent to the remaining recipients in the case of a tie.

View solution in original post

6 REPLIES 6
Dory_Viscoglio
Level 10

Re: No winner for A/B Test

Hi Erica, I wouldn't assume that Marketo won't arbitrarily send one as the winner based on previous experiences. If it's possible, I'd wait until your test is scheduled to end, and if it doesn't send a winner you can set up a smart campaign to the same audience except add an additional AND filter that says "not was sent email: email x or email y". This way, you won't send to people who already received either email in the test.

But like I said, I think it will most likely send one of your emails as the winner. 
Tara_Petre
Level 5 - Champion Alumni

Re: No winner for A/B Test

In the past, I've found that the first email in an A/B test was sent to the remaining recipients in the case of a tie.
Kristen_Malkov1
Level 8

Re: No winner for A/B Test

Agreed, I've had the same experience as Tara but only a handful of times. Unless you're only changing something SUPER minor, the winner should be sent. Although I would try and set up future tests to run a bit longer than 24 hrs. Perhaps 48-72 if you can! People really do tend to open emails after a few days (depending on your businesses and send dates). 

 
Josh_Hill13
Level 10 - Champion Alumni

Re: No winner for A/B Test

Agree with above, you must let it wait 48-72 hours as Kristen says before you have actionable data.

Also, try to have at least 300 people in the test sample. Even though 30 would be the rule of thumb, you likely won't get useful data from a 30 person sample. I posted a statistical significance calculator elsewhere, but you can do a search for one online.
Scott_McKeighe2
Level 5

Re: No winner for A/B Test

For reference, here's a great significance calculator for your toolbox:
http://getdatadriven.com/ab-significance-test
Anonymous
Not applicable

Re: No winner for A/B Test

Thank you all for your insight. It is all very helpful. In the event of a tie, the first test in the A/B test is declared the winner. Thank you.