Champion/Challenger Capability in Batch Campaigns

Champion/Challenger Capability in Batch Campaigns

It was recently confirmed to me by support that Champion/Challenger tests can only be used with trigger campaigns. While I understand that there are other A/B test capabilities, Champion/Challenger tests are very easy to set up and execute, and lead to more efficient testing across the board.

The fact that they can only be used with trigger campaigns unearthed another limitation for those of us using programs in engagement programs... You cannot do Champion/Challenger tests here either, because they are batch campaigns.

There was recently a bug that was allowing these tests in batch campaigns, so hopefully it wouldn't be too hard to implement this, and it would make testing much easier all-around!!
34 Comments
Dory_Viscoglio
Level 10
Cheryl, the distinction that I was trying to highlight (which tends to get brushed under the rug) is when programs are used within engagement programs. Those are considered to be batch programs -- not triggered, so the champion/challenger tests won't work in those scenarios either. 
Anonymous
Not applicable
Thanks for being more specific. I agree the distinction needs to be highlighted.
Matt_Stone2
Level 9
Just adding my vote to this. I use programs exclusively in streams to allow me some additional flexibility, so it'd be great to be able to use the champion/challenger feature in that way.
Sam_Chou1
Level 2
Wish I knew about this.. 2 casts ago. I was wondering why my email tests were only sending out the champions. 
Anonymous
Not applicable
So to be clear - there is no way to do an A/B test within an engagement program?
Sam_Chou1
Level 2
My roundabout method involved creating a separate Trigger campaign which sent out my champion/challenger email using the "Campaign is Requested" trigger.

So my Smart Campaign in the Engagement Program was a simple "Request Campaign" flow step to the new Trigger campaign.
Matt_Stone2
Level 9
Has that worked for you, Sam? I had a similar setup and it didn't seem to work.

My solution was to do something similar, but I used the random sample setup to send the different versions of the email (which meant I had to create all the variations as separate emails). So my trigger campaign had the Send Email step, but a few different constraints with random sample percentages to send the variations.
Anonymous
Not applicable
Matt - that sounds horribly manual for something that should be automated.  Sam I'd love to hear how that's worked for you as well.
Sam_Chou1
Level 2
I only found out about this quirk last week so my EP has only had 1 cast go out since I modified my campaigns, but it looks like it's working.

Super simple set up - below is the smart campaign used for the Engagement Program cast.
0EM50000000Sb8C.jpg

Below is the trigger campaign that is requested by the 1st campaign.
0EM50000000Sb8H.jpg

Below are the test results for the email that was sent.
0EM50000000Sb8M.jpg
Anonymous
Not applicable
Sam, this is a great tip. So is your first program pushed to the first cast and then the 'requested' campaign is the second cast? OR is the requested campaign the first cast?