Email A/B Testing - do I really need lists?

[Mark_Farnell]

We're A/B testing a 3 week standard mini- drip nurture campaign and a 3 week mini-engagement program.  In each case we're splitting off a region of our sample and running an A/B test (1.1 NAM).



Essentially, in the first week's flow if the region equals "X" then we request a campaign which sends 50% version A and 50% version B, just using random sample.  Everybody else isn't in the test get a previously approved email (which we are not testing 1.2 ROW above sends a different email to that group)



Then we wait a week (either by wait step in the dri[p nurture) or by cadence (for the engagement program).  We use a choice step on send email here to say send the 2nd version A email if you have been sent version A of the 1st email, otherwise send version B of the second email.



And then the same for week 3.

I don't see why we need to use a list here as recommended by https://community.marketo.com/MarketoArticle?id=kA050000000KytQ or https://community.marketo.com/MarketoArticle?id=kA050000000L6EBCA0.

Because we are only testing 2 emails I don't see how the lists will help me, but my team have some concerns.  Would appreciate the communities view.

Pretty much the same process exists in the engagement program - the camapign used in the enaggement program requests similar campaigns to send the email.
 

[Anonymous]

You are correct - you do not need to create the lists. Not sure the benefits of the documentation you referenced, but your method is the way I've always done things.

[Josh_Hill13]

Why don't you use the native email ab test - champion challenger instead?

[Anonymous]

Random sample is my friend — it's what I use for most of our campaigns!

[Dory_Viscoglio]

As Charlie said, you don't need to create the lists -- the one thing that it will do is help to speed up your processing. Random sample on 100k leads takes time, so if you have everyone in a list in advance it makes your actual send more likely to go out at the time that you anticipated. If you can deal with that, then you can continue doing it the way you are with random sample. 

Also, to follow up to Josh's question, many times I don't use champion/challenger testing because once you declare one of the emails to be the winner you lose the dashboard that you previously had with information about the emails. We like to be able to go back over time and review results and it's easy to put both emails into an email performance report. Just a personal preference 🙂