Can I declare an A/B test winner in less than 4 hours?

[Anonymous]

I want to A/B test the subject lines on my weekly newsletters, but I don't want the test group and remainder group to receive the newsletters 4 hours apart. 

I understand that A/B test needs time to run in order to determine a winner.... but I've found the majority of my contacts open the email within the first hour anyway, can I force the program to declare a winner and send out the reaminder of my emails after just 2 hours? 

[Dory_Viscoglio]

Hi Elicia, from what I've seen, you cannot force an email program to send the winner in less than a 4 hour window. Another option would be to use the champion/challenger test feature to send out a percentage of your emails, and then send the winner to anyone who didn't receive the first email. Because this requires both manual review (to select the winner), and a lot more setup since champion/challenger emails can only be done as triggered emails, I wouldn't suggest this option.

Would it be possible to send as a single batch and use your findings to improve your subject lines week over week?

[Matt_Stone2]

What I do is I set the test group to 100% and then immediately pick a winner after the four hour window. This almost entirely eliminates any people receiving the newsletter at a later time (though anyone who signs up in that four hour window will get it after the winner is chosen).

[Dory_Viscoglio]

Matt, I'm surprised that's worked for you. In the past when I've tried to do a 100% send, I've found that only about 75% sends, and Marketo reserves 25% to receive the winner. Granted, I only tried this once, but burned once is more than enough for me. 

[Matt_Stone2]

Seems to work fine for me--maybe there was a bug when you attempted it? It'd be great to have an employee verify this.

[Dory_Viscoglio]

Hi Matt, I'd also like a response from Marketo, but I'm not holding my breath. See this thread for more information about my gripes with email programs.