SOLVED

A/B Tests and Reporting

Go to solution
Anonymous
Not applicable

A/B Tests and Reporting

Hello,

We are looking to deploy an A/B test with 8 variants which will only test the email's subject line. I have a few questions on what I can track with the A/B testing functionality as opposed to running these as separate email templates. With the A/B testing functionality, are we able to:

1) Control the sample size going to each variant and control?

2) Will we be able to identify which lead received which email?

Thanks in advance for any insights into this topic.

1 ACCEPTED SOLUTION

Accepted Solutions
Dory_Viscoglio
Level 10

Re: A/B Tests and Reporting

Hi David, you will be able to know who received which variant by constraints in the filter:

pastedImage_0.png

If you're testing 8 variants at one time, I don't believe you'll be able to decide the size of each audience -- but it depends on how you're setting this up. Are you doing champion/challenger, or an email program?

View solution in original post

5 REPLIES 5
Dory_Viscoglio
Level 10

Re: A/B Tests and Reporting

Hi David, you will be able to know who received which variant by constraints in the filter:

pastedImage_0.png

If you're testing 8 variants at one time, I don't believe you'll be able to decide the size of each audience -- but it depends on how you're setting this up. Are you doing champion/challenger, or an email program?

Anonymous
Not applicable

Re: A/B Tests and Reporting

To my knowledge we will be doing champion / challenger which is the primary functionality the A/B test uses right? Our other option was to just duplicate the control email, make the changes to each variation, manually assign contacts to each variation, and assign that way. Does this clarify?

Anonymous
Not applicable

Re: A/B Tests and Reporting

Thanks, Dory!

David_Gaible
Level 7

Re: A/B Tests and Reporting

If you're doing champion/challenger, you can control the audience size of who gets challenger and champion, but you can't make distinction between who is getting Test 1 vs Test 8 - the audience for each test will be the same. For this example, each test will get 1/16th of the total audience (1/8th of the 1/2 audience getting the test emails.

pastedImage_0.png

Anonymous
Not applicable

Re: A/B Tests and Reporting

Thank you, David. This is very helpful