Zany idea - how about 2 hour lag for A/B test winner instead of 4?

[ayaspan]

It's kind of crap that the minimum time for an A/B test to determine a winner is 4 hours. Imagine it's on Friday and you're trying to finish up something around noon. There's already a 15-minute forced lag to get the thing out, but now if you're A/B testing (which someone on your team could be adamant about), you're basically struggling to get the bulk of your email into people's mailboxes before 5. So maybe shorten that? Seems arbitrary anyway.

[Zoe_Forman]

How would 2 hours work if you are in a region that covers 3 or 4 hour across time zones?
Say United States, EMEA

Some of the test sample would have reached 2 hours, but if A/B was sent in local time zone, say all at 9am, some countries would have had the email, and others still waiting for delivery.

[ayaspan]

Right, I'm not saying 2 hours needs to be the span, but for those of who are working in in pretty localized markets it should be an option.

[ayaspan]

Also pointing out for the void's benefit: 

The automatically suggested date for any A/B test is 1 day after the email is queued. Exactly 24 hours. At the risk of making myself look a total loon yet again, it's very easy to mistake tomorrow's date for today's.  So now I have to re-edit my email that was supposed to go out today. If I had a 2 hour lag I could just queue it up now but no... Why not just let us populate our own dates from scratch? Or, why not have the automatically suggested date be THE CURRENT TIME like with every other online service? 

[Beth_Massura]

Hi, the intention of having the test run for a certain amount of time is to to facilitate a valid, statistically significant test result. Too short of a testing window could result in too few responses to have a statistically significant winner, which negates the point of running a test. Even if there are enough responses, the data will be skewed by the preferences of the people who opened the email quickly (vs. those who will receive the email later due to time zone differences as noted above or those who tend to check their email less frequently); these people aren't representative of the overall audience. I'd recommend revisiting what you are trying to test and whether these particular short-turnaround emails need to include an A/B test.