a/b test had uneven delivery

[Esena_Jackson]

I conducted an a/b test but I don't see how it could have been weighted to send 75% to one group and 25% to the other. Seems like that highly impacted by results toward the message that went to 75% of my group. Does anyone have insight into a/b setup and why my distribution is so uneven?

[Darshil_Shah1]

Are you talking about the uneven deliveries of the test variants? If so, then this is quite unlikely as Marketo tries sending different variations to equal portions of the chosen Test Sample Size, but yes, if you have a small test audience, then you may see such numbers as Marketo would not be able to distribute evenly for the smaller test audience size. What are your test audience size (in terms of #) and the number of test variants?

[Esena_Jackson]

I'm pasting a screenshot of the dashboard below. I don't see a setting that would have influenced the delivery of a particular variant of the message.

[Darshil_Shah1]

Yes, you're correct, there isn't any setting that could be used to modify the send numbers for the test variants, if there were any such option present, then the test results would be skewed. Are you able to confirm if the number of deliveries for the AB test is accurate based on the sample size that you selected to receive the test emails?

[EllenSchwier]

Hi Darshil,

You commented on another post about the date range of an A/B test impacting this overview. The screenshot here does not include the date range. Could that play a role?

[Darshil_Shah1]

Hi Ellen,

Unfortunately, I don't think so. In a single email send program, as you'd know, users are able to perform an A/B test only 1x and the test window pulls the metrics from the only 1 test that was performed. I don't think there's an option to pull in the custom date/time window, just the test window. In the other question, OP was comparing the program's dashboard data with the EP report, which has the ability to set a custom date range.

I wanted to check the possibility of any deletions from the database that might have skewed the numbers, but as I'd expected, like the Email Performance report, the dashboard in the Email Send program pulls engagement data from the email asset level (and not from the person's act log), so that nulls out the possibility of skewed data due to person deletions. Lastly, I wonder whether this could just be due to a lag in the data retreival on the dashboard? Please let me know if I'm missing anything here. Thank you!