Champion/Challenger: Reporting and Declaring a Champion?

[Victoria_Chu]

Hi Marketo fam! I'm looking into using champion/challenger email testing and have a couple of questions not covered in the product docs.

1. How does the test show up on an email performance report? Does it show up at all?

2. Product doc says to discard the test, would I lose the email metrics on the test email?

3. If I declare the champion ​, do I still need to discard the test?

[Devraj_Grewal · Accepted solution]

Victoria,

1. As long as you select the emails tested, you can see all stats

2. Discarding the test just means the challenger will not be tested anymore. The email asset still exists and reports can be run on it

3. If you declare a champion, you actually won't even have the option to discard the test since it has ended, so no need to worry about it

[Victoria_Chu]

Thanks Devraj! Answers for questions 2 and 3 make sense.

For question number 1, does that also apply to subject line and from email address tests?

[Devraj_Grewal]

Good question. For testing of only subject line and from address, you will only be selecting one email asset overall. In the smart list of the email performance report, filter by "was sent email: test variant" and you can then view the report by test variant.

[Victoria_Chu]

So would I need a separate email performance report for each test variant?

For example, one for email subject line A, one for email subject line B?

[Devraj_Grewal]

If you would like to separate the variants out, then yes. Otherwise you will have to go back and forth and adjust the report's smart list when you want to view the other variant.

[Victoria_Chu]

Got it, thanks!

[Anonymous]

Hi Devraj,

When I filter the smart list of the email performance report by "was sent email: test variant". I get an Error running Report. Have you this filter before in your email performance report?

[Devraj_Grewal]

Tawni,

I would double check you are selecting the correct email in the "Setup" tab of the report and that those emails were used in an "A/B" style test. This version of the test will generate the test variants, not if you are testing whole emails.

[Anonymous]

Thanks. I have triple checked that I'm selecting the correct email and I'm

testing subject lines. Any other ideas would be greatly appreciated.

On Tue, May 30, 2017 at 3:18 PM, Devraj Grewal <marketingnation@marketo.com>