A/B Testing in Engagement Stream

[Anonymous]

What is the best way to accomplish A/B testing in engagement programs? I have read numerous articles on this topic that have evolved over the past year—there are various complex intricacies.

Can I use an Email Send program that has built in A/B and reports, OR must I use the Champion/Challenger mode with a default program? I believe it’s the later but just want to confirm.

A few notes

• Need to use a program so I can add exclusions (If someone downloaded WP ABC from website, they will be added to the program as an exclusion).
• Would like to manage outside the engagement program so the asset can be used in multiple engagement programs if needed.

Setup Correct?
Before we build out a bunch of these, want to confirm this is correct?
 
Secondary Related Question
We may use the same content across engagement programs.

If I use the Send via Engagement campaign in multiple engagement programs, can I use the same campaign (And adjust the Smartlist to include multiple engagement programs like below?) Or, must I create a campaign for each Engagement program. 

[Aqeel_Akbar]

The best way to do an A/B test within an engagement stream is to set up a champion/challenger. A/B tests run through email programs in engagement streams don't work, unfortunately. 

[Dory_Viscoglio]

Hey Jeff, I believe that Email Programs are unable to be dropped into Engagement Programs. I also know from experience, that Champion/Challenger won't work in programs added to Engagement Programs. The background on this is that for C/C to work it needs to be a triggered campaign, and these end up running as batch campaigns at the time of send. The easiest thing I've found is to create the two emails as individual assets and do a "Random Sample" send within the program that you're adding to the engagement program. There's a thread here where this was confirmed by Cheryl Chavez and some others from Marketo. 

Your secondary related question should be fine. I want to say I've used the same send campaign in multiple programs without having any issues. 

Best of luck!

[Anonymous]

Dory, I've been reading about your painful experiences which is why I opened up this post to see if there has been any progress. I got lost in that post with workarounds.

I've read info that says you can use challenger/champion tests in engagement programs which directly conflicts with your experiences. What if the program lives outside the engagement program.

From this help article....

Champion challenger is to be used with trigger campaigns and engagement programs only. They don't work with batch campaigns.

[Dory_Viscoglio]

Haha, sorry for the craziness that ensued in that post! I think the important takeaway from that was C/C only works if you have a single email that's being dropped into the EP. I'll be honest, I haven't tried it again since finding out that they wouldn't work, so I don't know if there's been any progress. 

[Josh_Hill13]

Jeff,
 

The thing is with Champ/Challenger you can run continous incremental improvements. With AB test Email Send campaigns, it requires more setup and isn't incremental - it's one off. And you can't keep re-using them.