A/B Testing Data Questions

Jason_Scott
Level 4

A/B Testing Data Questions


It seems when an a/b/n test is excuted all the "A" versions are sent then all the "B" and so on.  Can anyone verify this?
Tags (1)
2 REPLIES 2
Josh_Hill13
Level 10 - Champion Alumni

Re: A/B Testing Data Questions

Yes, that is how it should be.

You should select the A version to send during the setup process. The B versions are also set.

Are you experiencing something else?
Jason_Scott
Level 4

Re: A/B Testing Data Questions

Thanks for the reply.

That is what I'm seeing however, this leads to some poor data upon execution.  Deliverability starts to decline as an email is sent and therefore the "C" or "D" and so on versions gets poorer and poorer.  Every test we’ve run “A” has the highest deliverability followed by “B” then “C” and so on.

I believe Marketo is also assigning everyone within the campaign to a test group then deduping.  Once the email is executed “A” sends then “B” however “B” excludes any dup that was sent “A.” So, “B” test group is now a lower number of samples.  Then if there’s a “C” test group it’s even lower since any dup sent “A” or “B” is excluded.