Interpreting weekly Champion/Challenger email reports

[Anonymous]

Hi there,

We're trying to interpret our results from Champion/Challenger A/B tests.

It's unclear to us whether "Clicks" refers to

1) percentage of clicks past week or cumulative (in which case, why would one receive 6.45% and the other 0%?)

2) total clicks since the start of the campaign (in which case, why a decimal?)

3) total clicks that week

4) average clicks per week so far

5) something else entirely

Can someone help us interpret this?

Thanks!

Mariel

[ChristinaZuniga]

Hi Mariel,

Clicks should refer to the percentage to date in which case I would say that email A is doing much better than B (unless there are mitigating circumstances, like B doesn't have links or the links are incorrectly formed). You can confirm the numbers by creating an email link performance report in analytics, if you see different numbers then your email asking you to declare a winner might be wrong.

[Dory_Viscoglio]

Hey Mariel, with such disparate numbers I'd probably check the setup of the Champion/Challenger test. The first thing that comes to mind is that the challenger might not be sending -- which is what will happen if you have the test on a batch email. C/C will only work on triggered emails. You can confirm whether the email has sent at all via an email performance report.

[Josh_Hill13]

You should keep a log in excel with the data from each week, which should help you understand this.