Need a clear explanation of Champion vs Challenger testing in default program trigger campaigns

[Nate_Oosterhous]

Hi All,

I am in need of a clear explanation of how champion vs challenger testing actually functions when only one person is added to a trigger campaign at once so there for only one email will go out at once.  I am largely unclear on how the distribution works if only one email goes out at a time.  I have searched the community but I am struggling for clarity on this.

[Justin_Cooperm2 · Accepted solution]

Only one email goes out at a time, but based on the distribution you set, future folks who trigger that campaign may get a "challenger" instead of the champion. In other words you can leave the champion/challenger test enabled forever and Marketo will send out different variations to different people who trigger the campaign based on your criteria.

[Anonymous]

Justin explained it perfectly. I typically will just explain it as a random sample model. Each time someone comes through Marketo randomly selects which email they receive.

[Nate_Oosterhous]

Hi Brent,

What are your thoughts on my reply above.

[Josh_Hill13]

Are both emails inside the Test Group that is created once you create the Test? It has to be cloned INTO the test and I suspect that has not occurred based on your screenshot.

[Nate_Oosterhous]

Ok.  I think I have it figured out.  Based on Josh's mention about cloning, I pressed the orange Clone button.  This cloned that email and placed it in the test group. 

Now I have one challenger email outside of the champion email with the test group and one challenger email inside of the test group.  Does that sound correct? 

If so, what do you do with the challenger email outside of the test group after it was cloned into the test group?  It essentially isn't going to be used right? 

If you declare the challenger as the winner does it take the cloned version or the original?

[Nate_Oosterhous]

So what it is looking like to me is this:

How you should do it?

1. I should have one email to start with.
2. I should right click that email to create a test group.
3. In order to add an email to the test group I have to Clone the email in step 2 and edit that cloned email to meet our challenger qualifications.
4. Set up the rest of the settings.
5. If this is the case I can never test against a different email template style because I have to clone from the email in step 2.

How I was doing it:

1. I had two emails to start with.  One was going to be my "challenger" and the other was going to be my champion in which I create the test group within.
2. I right clicked the champion email to create the test group.
3. I selected my "challenger" in the Challengers: section of Test Settings and hit clone (because that was my only option to be able to approve the email).
4. Unfortunately this didn't even clone my challenger email it cloned the champion email.
5. It would be nice to be able to add a challenger email if it is already created rather than having to always clone the champion and edit it.

Thoughts?

[Grégoire_Miche2]

Hi Nate,

Last time I tried, I could clone any email to generate the email #2, not necessarily the #1.

-Greg

[Nate_Oosterhous]

Hi Greg,

Thanks you for your response and you are right Greg.  I got it to work finally.  I don't know what the issue was.  Marketo was acting up.

[Nate_Oosterhous]

Hi Josh,

It still is not allowing me to approve.  No I did not clone the email into the test.  How do you go about doing that?  I am confused on how this works with two emails.

Thanks

[Nate_Oosterhous]

Hi Justin,

1. I have two emails created and approved in a program named A/B Testing 2.Challenger and A/B Testing 2.Champion.
2. I created the test group from the A/B Testing 2.Challenger email.
3. I want to test these against each other with 50% of the people entering a smart campaign are receiving A/B Testing 2.Challenger and 50% are receiving A/B Testing 2.Champion.
4. Is below how I would set this up?  For some reason when I have it set up like below and I go to approve the test I am Not Allowed because of this error Can not approve A/B Test - test is missing an email.  Do this mean that all the emails involved in your test group must be a challenger?
5. Also what does 50% Receives Challengers and 50% Receives Champion really mean if I have not declared any champion?

[Anonymous]

4 - Have you 'Approved' the emails?

5 - Champion is just the original email in this case.

[Nate_Oosterhous]

I have approved both emails.  That is why I am confused on all of this.  I have refreshed the page as well.

[Anonymous]

Hmm, strange. Hard to tell without actually clicking around. Couple things I would check:

- Have you defined Champion Settings?

- As strange as it is, I've had to delete the test and rebuild it. Have you tried just resetting up the test?

[Nate_Oosterhous]

Brent,

I have defined champion settings and have tried deleting the test and rebuilding it.  It is almost like I have to have more than one challenger...