Champion/Challenger Capability in Batch Campaigns

[Dory_Viscoglio]

It was recently confirmed to me by support that Champion/Challenger tests can only be used with trigger campaigns. While I understand that there are other A/B test capabilities, Champion/Challenger tests are very easy to set up and execute, and lead to more efficient testing across the board.

The fact that they can only be used with trigger campaigns unearthed another limitation for those of us using programs in engagement programs... You cannot do Champion/Challenger tests here either, because they are batch campaigns.

There was recently a bug that was allowing these tests in batch campaigns, so hopefully it wouldn't be too hard to implement this, and it would make testing much easier all-around!!

[Dory_Viscoglio]

Cheryl, the distinction that I was trying to highlight (which tends to get brushed under the rug) is when programs are used within engagement programs. Those are considered to be batch programs -- not triggered, so the champion/challenger tests won't work in those scenarios either. 

[Anonymous]

Thanks for being more specific. I agree the distinction needs to be highlighted.

[Matt_Stone2]

Just adding my vote to this. I use programs exclusively in streams to allow me some additional flexibility, so it'd be great to be able to use the champion/challenger feature in that way.

[Sam_Chou1]

Wish I knew about this.. 2 casts ago. I was wondering why my email tests were only sending out the champions. 

[Anonymous]

So to be clear - there is no way to do an A/B test within an engagement program?

[Sam_Chou1]

My roundabout method involved creating a separate Trigger campaign which sent out my champion/challenger email using the "Campaign is Requested" trigger.

So my Smart Campaign in the Engagement Program was a simple "Request Campaign" flow step to the new Trigger campaign.

[Matt_Stone2]

Has that worked for you, Sam? I had a similar setup and it didn't seem to work.

My solution was to do something similar, but I used the random sample setup to send the different versions of the email (which meant I had to create all the variations as separate emails). So my trigger campaign had the Send Email step, but a few different constraints with random sample percentages to send the variations.

[Anonymous]

Matt - that sounds horribly manual for something that should be automated.  Sam I'd love to hear how that's worked for you as well.

[Sam_Chou1]

I only found out about this quirk last week so my EP has only had 1 cast go out since I modified my campaigns, but it looks like it's working.

Super simple set up - below is the smart campaign used for the Engagement Program cast.


Below is the trigger campaign that is requested by the 1st campaign.


Below are the test results for the email that was sent.

[Anonymous]

Sam, this is a great tip. So is your first program pushed to the first cast and then the 'requested' campaign is the second cast? OR is the requested campaign the first cast?