Subject Line Test Question

Level 2

Subject Line Test Question

I was curious, I am running a subject line test and I wanted to determine how many emails each subject line was sent to. For example, the champion was sent to 125 people, Challenger 1 was sent to 100 and so forth. Is there a place in the below where you can see that? The reason I am asking is that seems that the champion has a lot more opens and clicks than the challengers but I have the test distribution setup for 25% each so really confused. The Champion has 79 opens vs the other which around 10-12 opens. Thank You, Rick

pastedImage_0.png

9 REPLIES 9

Re: Subject Line Test Question

Hi Rick,

There should be a "Delivered" number under the Test Distribution to let you know the volume of delivered emails. Based on your distribution, you should assume send numbers to be equal. You can create an Email Performance Report to review that stats of each email.

As for the discrepancies in Opens, it would be expected to be higher for the Champion email - although perhaps not that much higher. Confirm the opens that are being tracked are unique. Again, Email Performance Report should help you here.

Level 2

Re: Subject Line Test Question

Thanks Nick, under email performance do you know how to separate by subject line? I am just seeing the below

pastedImage_0.png

Highlighted

Re: Subject Line Test Question

Marketo doesn't make it easy. It will require building multiple reports with smart list filter criteria "Was Sent Email" with the constraint of your
"Test Variant". More details here: Can you get a breakdown of A/B results in email performance report?

Level 2

Re: Subject Line Test Question

Thanks Nick, I am building out the email performance but do you know what should be in the "Test Variant" field? Its giving options of Subject A, B, C, D. Thanks Rick

pastedImage_0.png

Re: Subject Line Test Question

You will need to create a new Email Performance Report for each variant if you want to know the performance of each subject line.

Level 2

Re: Subject Line Test Question

Thanks work!, but now I see an issue with the distribution, Subject A went to 950 and the rest Subject B-D went to 220-250. Is there a particular reason why this would happen?

Re: Subject Line Test Question

Is Subject like A the champion?

Level 2

Re: Subject Line Test Question

Yes it is

Level 1

Re: Subject Line Test Question

What the value should be for the "Test Variant" field if it is a subject line test?