A/B Test in Nurture

[Anonymous]

I have leads automatically added to a list based on an action taken on our website. I would like to A/B test within the nurture. Is it possible to do so with 1 list? Can Marketo split up the list or do I need to specify test lists? All documentation and help articles are saying to create multiple lists and then assign each email to one of those lists. Not sure how everyone else is doing this.... preferably I would like to have 1 list and the system would just randomly send either A version or B version.

Appreciate the help!

Christina

[Anonymous]

Christina, would Natalie's idea accomplish what you are trying to do?

Instead of sending the email using the Random Sample Flow, you would be assigning to lists based on the Random Sample Flow and then sending based off list membership. It is still a triggered campaign using the Random Sample Flow step though. It would be kooky if it worked...

[Anonymous]

Hi guys,

I've recently also tried to set up A/B testing of a triggered email and came across this thread for guidance.

I ended up following the instructions on this page and it worked great for me:

https://community.marketo.com/MarketoArticle?id=kA050000000L6EBCA0

The trick is to use the Random Sample flow action to randomly add triggered leads to two lists.  Then you use logic to send email A to leads in list A and email B to leads in list B.

I've been running a campaign like this for a few days now and it seems to work great.

Hope this helps others looking to do the same!

Cheers,
Natalie

[Anonymous]

Yep....it is what it is

[Anonymous]

Kooky is rght! So ABCD style - you mean you broke up the alphabet? That seems like a lot of work for 'automation'. LOL

[Anonymous]

Marketo is kooky like that sometimes. Even when the logic is supposed to work, it may not be supported. If Marketo suggests that as a best practice, I guess that's just how it needs to be done. It's not a true A/B test but close enough.

I've used the ABCD style before as well and it works.

[Anonymous]

So this is what Marketo said "since this campaign is a trigger campaign, you will not want to use the ‘Random Sample’ flow action with a trigger campaign. To keep it simple and to make it work, for now, please get rid of all the Choice steps, and ensure that you’re sending them only one e-mail. If you’d still like to send different copies of emails, I’d recommend using Flow Steps such as, “If First Name starts with a, b, c, d,e,f’ etc, then send Email 1, If First Name starts with g, h, Ii, j, k, l, m, then send Email 2, etc. That way you’re emulating the random sample functionality for a trigger campaign."

I setup the random sample but it's not working correctly 😞 Is it really not possible to build A/B test for nurture? A trigger is what is going to send someone to any nurture so I dont see how this is different. The trigger is just 'a field is completed'. Any thoughts? Just doesnt seem right to me...

[Anonymous]

Awesome! Thanks so much.

[Anonymous]

In the smart list area, just reference the list, i,e. member of list x.

On the flow, that can work as well or do 2 flows the "do nothing" as default. Either way should work

[Anonymous]

I was looking at using the Random Sample logic. I need to add it to Smart List area - just once? Do I even need this?

Then on Flow tab, I created 1 flow step that said If Random Sample is 50, send A, and gave B as the default. Are you saying I need to have 2 Flow steps for each group of 50% with no default set?

[Anonymous · Accepted solution]

You can use randomly send using only 1 list using the Random Sample choice in the flow action. The 50 represents 50%. It will randomly choose half, then choose the other half. 

Smartlist: Member of List X

On you flow, use the "add choice" for your flow actions.

Flow
If: Random Sample is 50
Email: Version A

If; Random Sample is 50
Email: Version B

[Anonymous]

In the next few months Marketo will be rolling out a new nurturing tool with customer engagement. Not sure if this would offer A/B testing or not but it looks like nurturing tracks will be WAY EASIER.