A/B Testing - Static Lists Only?

[Anonymous]

If I'm reading the documentation correctly

The only method to conduct a 50/50 AB test is by creating 2 static lists.

Is there a method to use a smart list (or some other means)?

Thank You.
Tim


 

[Anonymous]

I'll follow through with the results. 

Hoping that documenting success will help another community user.

[Anonymous]

Thank you Josh.

[Anonymous · Accepted solution]

Hi Timmy,

I  would add Email 2 as the default step and then I always leave the remainder of the random percentage as default.  Marketo sends to the remaining percentage that didn't receive email 1. In this case 50%.

The reason is mostly convenience.  One less step to add.

The other is numbers, if you have 101 contacts and you do two 50% steps, because there is still that default step I assume the system would send 50 people email 1 and 50 people email 2, and one person doesn't receive it.  A lot of thinking for 1 email, but again it's easier to just use the default so why not?

[Anonymous]

Josh and Kim

Thank you for your kind replies.

Josh,
RE: 50% + Default

Please clarify, you're suggesting 

Random Send for Email 1

Default Send for Email 2

QUESTION - How would Marketo know the "Default Send for Email 2?"


----------

Kim 
RE 50% + 50%

If I understand correctly, you're suggesting

Random 1 (50%) + Random 2 (50%)

Wouldn't this create a condition where

25% would have double send (held as it was previously sent)
+ 25% of the list that didn't receive an email?

QUESTION - If I understand, this random 50% + random 50% = 25% that do not receive the email.

---------

It doesn't make sense that we'd create 2 static lists (I do understand AB testing is in the next version)

Thank you,

Timmy





 

[Anonymous]

Are you A/B testing a landing page or an email?

If it's an email:

• Select the "send email" action
• In the flow, add a condition to the "send email" step
• Set the condition to "Random Sample" = 50
• Set the first step to email A
• Set the default action to email B

If you're wanting to update static lists to keep them 50/50 it would be basically the same flow steps except you would change "send email" to "add to list" and seperate your a & b lists.

[Anonymous]

Yay Champion Challenger release!!

Timmy,

Use flow step Send Email and then select "add choice" twice (if you are testing two things). Then you would put If RANDOM SAMPLE is 50 send EMAIL 1 and for the second choice if RANDOM SAMPLE is 50 send EMAIL 2.

[Anonymous]

What do we do when contacts are being added 24x7x365  ??

With a static list..... new people (potentially more important) may be ommitted... 

Especially if there are Next Steps in the campaign....   Would this mean we'd be creating a new static list for every execution of a campaign?

OR how would we automatically repopulate the static lists?
 

[Anonymous]

FYI - AB split test and Champion Challenger test features are due out in our December release!

Static list is the best way b/c it is a set number of leads versus with a smart list, the count can change at any time.