SOLVED

Champion/Challenger: Reporting and Declaring a Champion?

Go to solution
Victoria_Chu
Level 3

Champion/Challenger: Reporting and Declaring a Champion?

Hi Marketo fam! I'm looking into using champion/challenger email testing and have a couple of questions not covered in the product docs.

1. How does the test show up on an email performance report? Does it show up at all?

2. Product doc says to discard the test, would I lose the email metrics on the test email?

3. If I declare the champion ​, do I still need to discard the test?

Tags (1)
1 ACCEPTED SOLUTION

Accepted Solutions
Devraj_Grewal
Level 10

Re: Champion/Challenger: Reporting and Declaring a Champion?

Victoria,

1. As long as you select the emails tested, you can see all stats

2. Discarding the test just means the challenger will not be tested anymore. The email asset still exists and reports can be run on it

3. If you declare a champion, you actually won't even have the option to discard the test since it has ended, so no need to worry about it

View solution in original post

11 REPLIES 11
Devraj_Grewal
Level 10

Re: Champion/Challenger: Reporting and Declaring a Champion?

Victoria,

1. As long as you select the emails tested, you can see all stats

2. Discarding the test just means the challenger will not be tested anymore. The email asset still exists and reports can be run on it

3. If you declare a champion, you actually won't even have the option to discard the test since it has ended, so no need to worry about it

View solution in original post

Victoria_Chu
Level 3

Re: Champion/Challenger: Reporting and Declaring a Champion?

Thanks Devraj! Answers for questions 2 and 3 make sense.

For question number 1, does that also apply to subject line and from email address tests?

Devraj_Grewal
Level 10

Re: Champion/Challenger: Reporting and Declaring a Champion?

Good question. For testing of only subject line and from address, you will only be selecting one email asset overall. In the smart list of the email performance report, filter by "was sent email: test variant" and you can then view the report by test variant.

Victoria_Chu
Level 3

Re: Champion/Challenger: Reporting and Declaring a Champion?

So would I need a separate email performance report for each test variant?

For example, one for email subject line A, one for email subject line B?

Devraj_Grewal
Level 10

Re: Champion/Challenger: Reporting and Declaring a Champion?

If you would like to separate the variants out, then yes. Otherwise you will have to go back and forth and adjust the report's smart list when you want to view the other variant.

Victoria_Chu
Level 3

Re: Champion/Challenger: Reporting and Declaring a Champion?

Got it, thanks!

Anonymous
Not applicable

Re: Champion/Challenger: Reporting and Declaring a Champion?

Hi Devraj,

When I filter the smart list of the email performance report by "was sent email: test variant". I get an Error running Report. Have you this filter before in your email performance report?

Devraj_Grewal
Level 10

Re: Champion/Challenger: Reporting and Declaring a Champion?

Tawni,

I would double check you are selecting the correct email in the "Setup" tab of the report and that those emails were used in an "A/B" style test. This version of the test will generate the test variants, not if you are testing whole emails.

Anonymous
Not applicable

Re: Champion/Challenger: Reporting and Declaring a Champion?

Thanks. I have triple checked that I'm selecting the correct email and I'm

testing subject lines. Any other ideas would be greatly appreciated.

On Tue, May 30, 2017 at 3:18 PM, Devraj Grewal <marketingnation@marketo.com>