SOLVED

Re: A/B Testing in Email Analysis

Go to solution
Kim_Allen
Level 10

A/B Testing in Email Analysis

Good morning,

I am excited about all the new features in May! This is one I can't figure out where to find...

A/B Test Emails in Email Analysis

Report on each of your A/B test email variants in Email Analysis.

Has anyone tried this yet?
Tags (1)
1 ACCEPTED SOLUTION

Accepted Solutions
Anonymous
Not applicable

Re: A/B Testing in Email Analysis

In the Smart List builder, if you add the "Opened Email" or "Clicked Link in Email" filters, you can then add a Constraint of "Test Variant".


In order to select a Variant, you must select the specific Email that you sent with an A/B test - it will not work if you say email sent, Is Any.

Email Performance reports still report the total number of Sends / Opens / Clicks of an Email, and do not break it down by the different test Variants.

View solution in original post

6 REPLIES 6
Anonymous
Not applicable

Re: A/B Testing in Email Analysis

In the Smart List builder, if you add the "Opened Email" or "Clicked Link in Email" filters, you can then add a Constraint of "Test Variant".


In order to select a Variant, you must select the specific Email that you sent with an A/B test - it will not work if you say email sent, Is Any.

Email Performance reports still report the total number of Sends / Opens / Clicks of an Email, and do not break it down by the different test Variants.

Anonymous
Not applicable

Re: A/B Testing in Email Analysis

Hi Kyle,

I'm trying your approach to see the Sends, Opens and Clicks of my 3 test variants. However, in my Smart List when I choose "Was Sent Email" and select the email and then use the Test Variant constraint, it says "Email A, Email B, Email C, Email D" and I only have 3 variants...also, I tested just using Email A and it gave me the same results as the entire group of emails. Can you help clarify?

Kim_Allen
Level 10

Re: A/B Testing in Email Analysis

Awww I was hoping it was for email performance reports. Thanks, Kyle!
Anonymous
Not applicable

Re: A/B Testing in Email Analysis

That would be so helpful to get analytics for overall A/B testing rather than have to record it on an instance by instance basis.
Phillip_Wild
Level 10

Re: A/B Testing in Email Analysis

Agreed on all the above guys - but good to know it can be done, at least in a roundabout way.

How it really should work is that you can see analytics just like any other email, for both A and B variants - whether that's overall clicks and opens, or link performance, or whatever. If I'm running an A/B test on content, often I'm looking for particular link performance - not total clicks. Current functionality makes it very difficult to do this, unless I set a custom conversion target at setup, which is limiting.
Phillip_Wild
Level 10

Re: A/B Testing in Email Analysis

Hi Sarah

That's exactly how you should be doing it. The only thing I can think of is that you've selected the wrong email.

Sorry, I know that's probably not super helpful!

Phil