A/B Test with Nurture

[FeliciaTan]

Hi,

I have a question about A/B testing with nurture. If I want to split the test into 25% & 25% and send the winner's version to the remaining 50% of the audience, how to set it up under nurture? I know the email program provides declaring winner version and sends the remaining group winner. Wondering if we can declare the winner and send winner version automatically?

Thanks for everyone's help!

[Darshil_Shah1 · Accepted solution]

We might need more information on your setup here, e.g., are you doing nurture using Marketo's Engagement program or building manual nurture?

Well, the OOTB AB testing is only available in the Email send program, and you can't add the Email send program to the Engagement program's stream as content. However, you can set the Champion vs. Challenger (CvC) email testing at the email level, which is supported for the emails sent via the trigger campaigns and Engagement program. Similar to the AB test in the Email send programs, in the CvC test, you can do the whole email, subject line, and address testing (can't do the send date time test in the CvC test due to obvious reasons).

In the CvC test, you define the champion criteria and configure the reports that are sent to you from Marketo. You can also declare the date (a future date) when the Champion is selected based on the defined champion criteria. Until the Declare Date, Marketo will continue to send the old champion and challenger(s). Only the new champion will be sent once the date/time is reached.

In contrast, if you're building manual nurture with Email send programs, you can set the AB test at the individual program level, set the testing criteria, send the tests, and then set the winner email to the remaining audience. However, be advised that in this case people in the testing and the winning emails group will receive emails at different dates/times.

The latter group will be sent the winning email once the winner is declared. With the AB test, you can set the send winner email date time (send test and send winner must be at least 4 hours apart). Marketo will automatically declare the winner based on the winner criteria and send the winning email to the audience's remaining (i.e., the non-test group) portion.

I hope this is helpful. Please let us know if you have questions.

[uditmathur]

Hi,

As @Darshil_Shah1 mentioned to test your Email as part of your Engagement program you will have to use Champion/Challenger test.

Just to add a few points to keep in mind wrt CvsC test.

• You can not test your email for Date/Time test in CvsC i.e. you can only test your emails for Whole Email, Subject Line, From Address.
• If you are testing more than 1 version of your email (more the 1 challenger) than the %age of distribution selected for challenger will be shared equally with all the challenger mails.
• You will have to manually declare the winner after completion of the test.