Re: What's your sample size?

Jeff_Canada2
Level 4

What's your sample size?

I am curious to see what the Marketing Nation thinks is an appropriate number of leads for an A/B test? 100? 1,000? 10,000? 2?
Jeff Canada
Tags (1)
3 REPLIES 3
Anonymous
Not applicable

Re: What's your sample size?

I'd say that it depends on the variation you see.  If you send an email to 100 people and 40 open version A and 2 open version B, that's pretty signifcantly in favor of version A.

In general though, I try to test at least a few thousand - 5k-10k just to get a larger sample size and have a more accurate test.

I just use a calculator like this one, and plug in the numbers.
Josh_Hill13
Level 10 - Champion Alumni

Re: What's your sample size?

Yes, use the calculator. There is no "best practice" other than to do the math.

Now if you want to not do math, they say that n=30 is a minimum. In most cases with email, you need several hundred out of your list to have any reasonable significance...so use the calculator.
Michelle_Tizian
Level 10

Re: What's your sample size?

I prefer a minimum of 1000 to 5000 leads for an A/B test. A couple of times I was asked to split a list evenly 50/50 sending 2 different versions of an email to 200 and another time 500 people, it didn't make that big of a difference on the results.  I just found that one list had more bad email address than the other list, but had about the same results for the unsubscribes and CTR.  I also did a few A/B tests on sponsored emails using 10-20% out of 20 to 30k leads and the results were more obvious. I like the idea of using the calculator.