A/B Testing Data Questions

[Jason_Scott]

It seems when an a/b/n test is excuted all the "A" versions are sent then all the "B" and so on.  Can anyone verify this?

[Josh_Hill13]

Yes, that is how it should be.

You should select the A version to send during the setup process. The B versions are also set.

Are you experiencing something else?

[Jason_Scott]

Thanks for the reply.

That is what I'm seeing however, this leads to some poor data upon execution.  Deliverability starts to decline as an email is sent and therefore the "C" or "D" and so on versions gets poorer and poorer.  Every test we’ve run “A” has the highest deliverability followed by “B” then “C” and so on.

I believe Marketo is also assigning everyone within the campaign to a test group then deduping.  Once the email is executed “A” sends then “B” however “B” excludes any dup that was sent “A.” So, “B” test group is now a lower number of samples.  Then if there’s a “C” test group it’s even lower since any dup sent “A” or “B” is excluded.