Subject Line Test Question

[Rick_Gonedes]

I was curious, I am running a subject line test and I wanted to determine how many emails each subject line was sent to. For example, the champion was sent to 125 people, Challenger 1 was sent to 100 and so forth. Is there a place in the below where you can see that? The reason I am asking is that seems that the champion has a lot more opens and clicks than the challengers but I have the test distribution setup for 25% each so really confused. The Champion has 79 opens vs the other which around 10-12 opens. Thank You, Rick

[Nicholas_Hajdin]

Hi Rick,

There should be a "Delivered" number under the Test Distribution to let you know the volume of delivered emails. Based on your distribution, you should assume send numbers to be equal. You can create an Email Performance Report to review that stats of each email.

As for the discrepancies in Opens, it would be expected to be higher for the Champion email - although perhaps not that much higher. Confirm the opens that are being tracked are unique. Again, Email Performance Report should help you here.

[Rick_Gonedes]

Thanks Nick, under email performance do you know how to separate by subject line? I am just seeing the below

[Nicholas_Hajdin]

Marketo doesn't make it easy. It will require building multiple reports with smart list filter criteria "Was Sent Email" with the constraint of your
"Test Variant". More details here: Can you get a breakdown of A/B results in email performance report?

[Rick_Gonedes]

Thanks Nick, I am building out the email performance but do you know what should be in the "Test Variant" field? Its giving options of Subject A, B, C, D. Thanks Rick

[Nicholas_Hajdin]

You will need to create a new Email Performance Report for each variant if you want to know the performance of each subject line.

[Rick_Gonedes]

Thanks work!, but now I see an issue with the distribution, Subject A went to 950 and the rest Subject B-D went to 220-250. Is there a particular reason why this would happen?

[Nicholas_Hajdin]

Is Subject like A the champion?

[Rick_Gonedes]

Yes it is

[Stephanie_Wu]

What the value should be for the "Test Variant" field if it is a subject line test?