2 Replies Latest reply on Jul 7, 2014 11:49 AM by Jason Scott

    A/B Testing Data Questions

    Jason Scott

      It seems when an a/b/n test is excuted all the "A" versions are sent then all the "B" and so on.  Can anyone verify this?
        • Re: A/B Testing Data Questions
          Josh Hill
          Yes, that is how it should be.

          You should select the A version to send during the setup process. The B versions are also set.

          Are you experiencing something else?
          • Re: A/B Testing Data Questions
            Jason Scott
            Thanks for the reply.

            That is what I'm seeing however, this leads to some poor data upon execution.  Deliverability starts to decline as an email is sent and therefore the "C" or "D" and so on versions gets poorer and poorer.  Every test we’ve run “A” has the highest deliverability followed by “B” then “C” and so on.

            I believe Marketo is also assigning everyone within the campaign to a test group then deduping.  Once the email is executed “A” sends then “B” however “B” excludes any dup that was sent “A.” So, “B” test group is now a lower number of samples.  Then if there’s a “C” test group it’s even lower since any dup sent “A” or “B” is excluded.