3 Replies Latest reply on Jul 26, 2016 3:46 PM by dac034e5ee8fc4ae3cc7702b463baa7272a38ad7

Pro and Cons of "Add A/B Test" Functionality

e273d327ca3a4d48583fc6b279307a7df508ee43 Jul 25, 2016 8:06 AM

Hello,

We are currently working to outline A/B testing that will test eight different variations of the subject line in our email. To my understanding there are two ways to do this:

a) We set this up utilizing the A/B test functionality within Marketo.

b) We clone and modify eight separate emails, and do analysis manually.

My question is whether or not there are cons to using the built in A/B testing split in Marketo. We want to be able to look at specifics of who the contacts responding to each variation are, and want to ensure we won't have any restrictions to the type of data we can see.

Thanks in advance for any responses.

Re: Pro and Cons of "Add A/B Test" Functionality

Jamie Lewis Jul 25, 2016 9:22 AM (in response to e273d327ca3a4d48583fc6b279307a7df508ee43)

You will be able to see who received the varients in the standard A/B testing. The only con is that the test will run for a specific amount of time before the winner is sent to the remaining members of the target audience. You could actually just set the test to 100% so that all people recieve the original test. My only concern with an 8 subject test is that you have a list large enough to provide you with meaningful results.
Like Show 0 Likes(0)

Actions
- Re: Pro and Cons of "Add A/B Test" Functionality
  
  dac034e5ee8fc4ae3cc7702b463baa7272a38ad7 Jul 26, 2016 3:46 PM (in response to Jamie Lewis)
  
  Hi Jamie,
  
  I have to test two emails with different bodies. If I use the a/b test function, and set the test sample to 100% would it be sent 50/50? I'm not looking for a clear winner just would like to distinguish how well the two compared to each other. If I do that, do you know if I would lose any information like click/ open rate?
  
  Like Show 0 Likes(0)
  
  Actions
Re: Pro and Cons of "Add A/B Test" Functionality

Josh Hill Jul 25, 2016 9:45 AM (in response to e273d327ca3a4d48583fc6b279307a7df508ee43)
If you are running a batch, just use the built in functionality. The only caveats are
record the data you get from the reports in another sheet
you can use Was Sent Email with Test Variant to see leads.
Like Show 0 Likes(0)

Actions

Go to original post