A/B test whole email

Anonymous
Not applicable

A/B test whole email

I'm setting up an A/B test for whole emails. The winner criteria will be custom conversion by clicking on the CTA button (button color test). When using the filter "clicked link in email," the emails don't pop up. Should I use "email is any" and link is (url)?

And second, I wanted to change program status in the program as well, but the emails don't show? Do I need to change program statuses post-send?

Thanks.

7 REPLIES 7

Re: A/B test whole email

Hi Sherin,

Can you please provide some screenshots?

-Greg

Anonymous
Not applicable

Re: A/B test whole email

This shot is in the Winner Criteria section. I want the criteria to be a person clicked on our button (specific link). I was going use “Email is” and list both testA and testB, but neither show in the drop down.

pastedImage_1.png

Therefore, I used “Email is any” and listed the link. Will this work?

The same happens for when trying to update program status – the emails don’t show up.

pastedImage_2.png

Re: A/B test whole email

It should work, and by default the A/B test is only going to consider the emails you definied in the Test Settings step when determining the winner.

Here's an obvious question but worth double-checking anyway...are the emails approved? 

Level 10 - Champion Alumni

Re: A/B test whole email

You need to first make sure everything is approved (not just the email) - including the actual send (the highlighted box below):

pastedImage_0.png

Once you've confirmed that, your email will appear in triggers/filters as "NAME OF PROGRAM.NAME OF TEST".  For example "2017 test.Subject Line Test"

Re: A/B test whole email

Sherin, in case you didn't find this yet. You need to select the email and then add a constraint for which version of the test you want to report on.

pastedImage_0.png

Level 10 - Champion Alumni

Re: A/B test whole email

Hi Dory - this is good to know as well (specifically if you want to narrow the trigger by test variant).  I think Sherin's issue was that she wasn't even able to select the email in the main trigger in the first place.  Whenever you include any A/B testing, the name of the email takes on a value that differs from traditional emails (a bit confusing, I know).  And as mentioned above, the entire program must be approved in order for this value to appear.

Re: A/B test whole email

Mine didn't work anyway I tried it.  We tried to approve everything, like you said, but that didn't work either.  It also seems counter intuitive to approve a campaign before you finish configuring the settings.

Here is my scenario...

https://nation.marketo.com/thread/52877-how-to-setup-whole-email-ab-test-with-custom-conversion