A/B Testing whole emails within engagement program

Highlighted

A/B Testing whole emails within engagement program

I'm trying to A/B test two different versions of an email within a nurture engagement program.

I've set it up using the "If Random Sample = 50" setup in the flow step of the batch send campaign. 

Would one email be in the Choice 1 option, with the other email as the default choice? (attached).

Or, is the way to do this to use no default choice, and instead, use:

- If Choice 1 = Random Sample 50

Email: version 1

- If Choice 2 = Random Sample 50

Email Version 2

Default choice is empty

Thanks!

5 REPLIES 5
Highlighted

Re: A/B Testing whole emails within engagement program

Diana DeMallie‌ - Both are fine to use. Also, seems that the attachment is incorrect.

Highlighted
Level 10 - Community Moderator

Re: A/B Testing whole emails within engagement program

Actually that's not so, you should use a (non-empty) Default Choice whenever using Random Sample.

Not every batch is evenly divisible. And even when the count is divisible on a mathematical level, Random Sample does not behave like a round-robin algorithm. 

Highlighted

Re: A/B Testing whole emails within engagement program

Got it - thanks!

After this launches, will a regular email performance report be the best way to view and compare the results between the two emails that are split with the random sample? 

Highlighted

Re: A/B Testing whole emails within engagement program

Hi Sanford - 

After implementing this flow step with the 50/50 email split, will an email performance report display the results between each email's performance? I created these reports for the time period to compare the two emails, but no results appear, so I'm wondering if there's another way to analyze and compare the results of the email comparison.

Highlighted

Re: A/B Testing whole emails within engagement program

You may also want to consider using the champion/challenger test functionality instead: Email Tests - Champion/Challenger - Marketo Docs - Product Documentation