Interpreting weekly Champion/Challenger email reports

Anonymous
Not applicable

Interpreting weekly Champion/Challenger email reports

Hi there,

We're trying to interpret our results from Champion/Challenger A/B tests.

It's unclear to us whether "Clicks" refers to

1) percentage of clicks past week or cumulative (in which case, why would one receive 6.45% and the other 0%?)

2) total clicks since the start of the campaign (in which case, why a decimal?)

3) total clicks that week

4) average clicks per week so far

5) something else entirely

pastedImage_0.png

Can someone help us interpret this?

Thanks!

Mariel

3 REPLIES 3
ChristinaZuniga
Level 10

Re: Interpreting weekly Champion/Challenger email reports

Hi Mariel,

Clicks should refer to the percentage to date in which case I would say that email A is doing much better than B (unless there are mitigating circumstances, like B doesn't have links or the links are incorrectly formed). You can confirm the numbers by creating an email link performance report in analytics, if you see different numbers then your email asking you to declare a winner might be wrong.

Dory_Viscoglio
Level 10

Re: Interpreting weekly Champion/Challenger email reports

Hey Mariel, with such disparate numbers I'd probably check the setup of the Champion/Challenger test. The first thing that comes to mind is that the challenger might not be sending -- which is what will happen if you have the test on a batch email. C/C will only work on triggered emails. You can confirm whether the email has sent at all via an email performance report.

Josh_Hill13
Level 10 - Champion Alumni

Re: Interpreting weekly Champion/Challenger email reports

You should keep a log in excel with the data from each week, which should help you understand this.