Champion/Challenger Capability in Batch Campaigns

[Dory_Viscoglio]

It was recently confirmed to me by support that Champion/Challenger tests can only be used with trigger campaigns. While I understand that there are other A/B test capabilities, Champion/Challenger tests are very easy to set up and execute, and lead to more efficient testing across the board.

The fact that they can only be used with trigger campaigns unearthed another limitation for those of us using programs in engagement programs... You cannot do Champion/Challenger tests here either, because they are batch campaigns.

There was recently a bug that was allowing these tests in batch campaigns, so hopefully it wouldn't be too hard to implement this, and it would make testing much easier all-around!!

[Anonymous]

Couldn't agree more. I'd love to see this implemented.

[Anonymous]

Let me clarify this for you:

Champion/Challenger tests are intended for use in:

Trigger Campaigns
Engagement Programs (just ignore that it is a batch campaign behind the scenes)

The Email Program with split AB test is intended for the standard batch campaign

Thanks,
Cheryl

[Dory_Viscoglio]

Hi Cheryl,

This is great, but the issue is that if you aren't using email programs for a single send, then you don't have an A/B test capability that's baked in. I can see that the goal is to move away from programs and into email programs for single sends, but we aren't sold on email programs just yet. Maybe I will do some more work with them, and get to the point where I'm comfortable sending them. 

As for the engagement programs statement, that's what was confirmed to me by Kenny E. from support last week, so if that's not correct could you please let me know? (Case #00313182) 

Thanks!
-Dory

[Anonymous]

Hi Dory,

I'd love to learn more about why you are not comfortable with the Email Send program.  If there is an issue, I'd like to be sure we address it.

Thanks for letting me know about Kenny - I'll follow up with him.

Thanks,
Cheryl

[Anonymous]

HI Dory,

After speaking with Kenny, I want to add a very important distinction here.  Champion/Challenger works when you use an email on a stream.  If you use a program and smart campaign with Champ/challenger it will only send out the Champion.  

Thanks,
Cheryl

[Grant_Booth]

I see how we have good workarounds available here, but there's still demand for use of the champion-challenger in batch smart campaigns, since there are a number of other flow steps which can be used along with it, aside from just sending an email like the Email Send program does. Don't get me wrong - the email send program is great for sending emails, and has a nice dashboard for analyzing the send, and has great AB tests. But some customers like to go with the flow flexibility of an old fashioned smart campaign.

Given that, although we have some great workarounds, I don't think we should classify this Idea as "Already Have It".

[Dory_Viscoglio]

I agree with Grant 110%! I've had nothing but issues with the email programs (which haven't been resolved), so I do not feel comfortable using them to do any type of A/B testing. The only way that our organization is using email programs going forward is for people to get a simple batch campaign out the door if they aren't active users of Marketo. Even most of those have been templated, and our users are just instructed to clone the template.

[Grant_Booth]

We continue to get calls from customers who are confused by this behavior, since there's nothing in the UI to indicate the Champion/Challenger test will only send the champion if they try to send it in a batch campaign. If we aren't able to make it available for batch campaigns due to technical reasons, we should really have something in the UI (maybe in the Champion/Challenger dashboard?) which says "Note: this test only applies to triggered campaigns and engagement programs - not batch campaigns."

[Dory_Viscoglio]

Grant, the only thing with that disclaimer is that you'd have to also specify that programs used within engagement progams are considered batch campaigns, not triggered. But, no doubt, it would be helpful to have somewhere other than just the resource documents which most people won't read. 

[Anonymous]

Champion Challenger testing is available for use in Engagement Programs and Trigger campaigns.  Please don't get hung up that Engagement Programs are batch campaigns under the hood!

Champion Challenger tests will send out the champion if called for use in a batch campaign.