What's your sample size?

[Jeff_Canada2]

I am curious to see what the Marketing Nation thinks is an appropriate number of leads for an A/B test? 100? 1,000? 10,000? 2?

[Anonymous]

I'd say that it depends on the variation you see.  If you send an email to 100 people and 40 open version A and 2 open version B, that's pretty signifcantly in favor of version A.

In general though, I try to test at least a few thousand - 5k-10k just to get a larger sample size and have a more accurate test.

I just use a calculator like this one, and plug in the numbers.

[Josh_Hill13]

Yes, use the calculator. There is no "best practice" other than to do the math.

Now if you want to not do math, they say that n=30 is a minimum. In most cases with email, you need several hundred out of your list to have any reasonable significance...so use the calculator.

[Michelle_Tizian]

I prefer a minimum of 1000 to 5000 leads for an A/B test. A couple of times I was asked to split a list evenly 50/50 sending 2 different versions of an email to 200 and another time 500 people, it didn't make that big of a difference on the results.  I just found that one list had more bad email address than the other list, but had about the same results for the unsubscribes and CTR.  I also did a few A/B tests on sponsored emails using 10-20% out of 20 to 30k leads and the results were more obvious. I like the idea of using the calculator.