What is the best way to accomplish A/B testing in engagement programs? I have read numerous articles on this topic that have evolved over the past year—there are various complex intricacies.
Can I use an Email Send program that has built in A/B and reports, OR must I use the Champion/Challenger mode with a default program? I believe it’s the later but just want to confirm.
A few notes
Need to use a program so I can add exclusions (If someone downloaded WP ABC from website, they will be added to the program as an exclusion).
Would like to manage outside the engagement program so the asset can be used in multiple engagement programs if needed.
Setup Correct? Before we build out a bunch of these, want to confirm this is correct? Secondary Related Question We may use the same content across engagement programs.
If I use the Send via Engagement campaign in multiple engagement programs, can I use the same campaign (And adjust the Smartlist to include multiple engagement programs like below?) Or, must I create a campaign for each Engagement program.
Hey Jeff, I believe that Email Programs are unable to be dropped into Engagement Programs. I also know from experience, that Champion/Challenger won't work in programs added to Engagement Programs. The background on this is that for C/C to work it needs to be a triggered campaign, and these end up running as batch campaigns at the time of send. The easiest thing I've found is to create the two emails as individual assets and do a "Random Sample" send within the program that you're adding to the engagement program. There's a thread here where this was confirmed by Cheryl Chavez and some others from Marketo.
Your secondary related question should be fine. I want to say I've used the same send campaign in multiple programs without having any issues.
Haha, sorry for the craziness that ensued in that post! I think the important takeaway from that was C/C only works if you have a single email that's being dropped into the EP. I'll be honest, I haven't tried it again since finding out that they wouldn't work, so I don't know if there's been any progress.
The thing is with Champ/Challenger you can run continous incremental improvements. With AB test Email Send campaigns, it requires more setup and isn't incremental - it's one off. And you can't keep re-using them.