A/B Tests and Reporting

[Anonymous]

Hello,

We are looking to deploy an A/B test with 8 variants which will only test the email's subject line. I have a few questions on what I can track with the A/B testing functionality as opposed to running these as separate email templates. With the A/B testing functionality, are we able to:

1) Control the sample size going to each variant and control?

2) Will we be able to identify which lead received which email?

Thanks in advance for any insights into this topic.

[Dory_Viscoglio · Accepted solution]

Hi David, you will be able to know who received which variant by constraints in the filter:

If you're testing 8 variants at one time, I don't believe you'll be able to decide the size of each audience -- but it depends on how you're setting this up. Are you doing champion/challenger, or an email program?

[Anonymous]

To my knowledge we will be doing champion / challenger which is the primary functionality the A/B test uses right? Our other option was to just duplicate the control email, make the changes to each variation, manually assign contacts to each variation, and assign that way. Does this clarify?

[Anonymous]

Thanks, Dory!

[David_Gaible]

If you're doing champion/challenger, you can control the audience size of who gets challenger and champion, but you can't make distinction between who is getting Test 1 vs Test 8 - the audience for each test will be the same. For this example, each test will get 1/16th of the total audience (1/8th of the 1/2 audience getting the test emails.

[Anonymous]

Thank you, David. This is very helpful