8 Replies Latest reply on Oct 24, 2013 8:07 AM by 42161

    A/B Testing - Static Lists Only?

      If I'm reading the documentation correctly

           The only method to conduct a 50/50 AB test is by creating 2 static lists.

           Is there a method to use a smart list (or some other means)?

           Thank You.


        • Re: A/B Testing - Static Lists Only?
          FYI - AB split test and Champion Challenger test features are due out in our December release!

          Static list is the best way b/c it is a set number of leads versus with a smart list, the count can change at any time.
          • Re: A/B Testing - Static Lists Only?

                 What do we do when contacts are being added 24x7x365  ??


                 With a static list..... new people (potentially more important) may be ommitted... 


                 Especially if there are Next Steps in the campaign....   Would this mean we'd be creating a new static list for every execution of a campaign?


                 OR how would we automatically repopulate the static lists?

            • Re: A/B Testing - Static Lists Only?
              Yay Champion Challenger release!!


              Use flow step Send Email and then select "add choice" twice (if you are testing two things). Then you would put If RANDOM SAMPLE is 50 send EMAIL 1 and for the second choice if RANDOM SAMPLE is 50 send EMAIL 2.
              • Re: A/B Testing - Static Lists Only?
                Are you A/B testing a landing page or an email?

                If it's an email:
                •           Select the "send email" action
                •           In the flow, add a condition to the "send email" step
                •           Set the condition to "Random Sample" = 50
                •           Set the first step to email A
                •           Set the default action to email B
                If you're wanting to update static lists to keep them 50/50 it would be basically the same flow steps except you would change "send email" to "add to list" and seperate your a & b lists.

                • Re: A/B Testing - Static Lists Only?

                       Josh and Kim

                  Thank you for your kind replies.

                       RE: 50% + Default

                       Please clarify, you're suggesting 

                       Random Send for Email 1

                       Default Send for Email 2

                  QUESTION - How would Marketo know the "Default Send for Email 2?"


                       RE 50% + 50%

                       If I understand correctly, you're suggesting

                       Random 1 (50%) + Random 2 (50%)

                  Wouldn't this create a condition where

                       25% would have double send (held as it was previously sent)
                       + 25% of the list that didn't receive an email?

                  QUESTION - If I understand, this random 50% + random 50% = 25% that do not receive the email.


                       It doesn't make sense that we'd create 2 static lists (I do understand AB testing is in the next version)

                       Thank you,




                  • Re: A/B Testing - Static Lists Only?
                    Hi Timmy,

                    I  would add Email 2 as the default step and then I always leave the remainder of the random percentage as default.  Marketo sends to the remaining percentage that didn't receive email 1. In this case 50%.

                    The reason is mostly convenience.  One less step to add.

                    The other is numbers, if you have 101 contacts and you do two 50% steps, because there is still that default step I assume the system would send 50 people email 1 and 50 people email 2, and one person doesn't receive it.  A lot of thinking for 1 email, but again it's easier to just use the default so why not?

                    • Re: A/B Testing - Static Lists Only?
                      Thank you Josh.

                      • Re: A/B Testing - Static Lists Only?

                             I'll follow through with the results. 


                             Hoping that documenting success will help another community user.