A/B Testing in Email Analysis

[Kim_Allen]

Good morning,

I am excited about all the new features in May! This is one I can't figure out where to find...

A/B Test Emails in Email Analysis

Report on each of your A/B test email variants in Email Analysis.

Has anyone tried this yet?

[Anonymous · Accepted solution]

In the Smart List builder, if you add the "Opened Email" or "Clicked Link in Email" filters, you can then add a Constraint of "Test Variant".

In order to select a Variant, you must select the specific Email that you sent with an A/B test - it will not work if you say email sent, Is Any.

Email Performance reports still report the total number of Sends / Opens / Clicks of an Email, and do not break it down by the different test Variants.

[Anonymous]

Hi Kyle,

I'm trying your approach to see the Sends, Opens and Clicks of my 3 test variants. However, in my Smart List when I choose "Was Sent Email" and select the email and then use the Test Variant constraint, it says "Email A, Email B, Email C, Email D" and I only have 3 variants...also, I tested just using Email A and it gave me the same results as the entire group of emails. Can you help clarify?

[Kim_Allen]

Awww I was hoping it was for email performance reports. Thanks, Kyle!

[Anonymous]

That would be so helpful to get analytics for overall A/B testing rather than have to record it on an instance by instance basis.

[Phillip_Wild]

Agreed on all the above guys - but good to know it can be done, at least in a roundabout way.

How it really should work is that you can see analytics just like any other email, for both A and B variants - whether that's overall clicks and opens, or link performance, or whatever. If I'm running an A/B test on content, often I'm looking for particular link performance - not total clicks. Current functionality makes it very difficult to do this, unless I set a custom conversion target at setup, which is limiting.

[Phillip_Wild]

Hi Sarah

That's exactly how you should be doing it. The only thing I can think of is that you've selected the wrong email.

Sorry, I know that's probably not super helpful!

Phil