AB Testing in Smart Campaigns

Level 2

AB Testing in Smart Campaigns



I recently deployed an AB test through a smart campaign. The original recipient list was around ~410k people. Here is how I set up the AB test in marketo: 


Post deploying the AB test, I pulled a report -- and the campaign did split the sample size 50/50 -- however, the campaign only deployed to about 320k people total. I double-checked the smart list and the recipients should have been over 400k, and I also made sure there were no duplicates.


Does anyone have any suggestions as to where the gap could be in these numbers? Was it something I did wrong or am missing in the flow? Side note: I traditionally set AB tests up through email programs (which I generally think are easier) -- however, the email I deployed has dynamic content, and it is easier to set up subject line AB tests through smart campaigns. 


Hope this insight and background help & appreciate any additional tips on this issue, thanks!

Level 10 - Community Moderator

Re: AB Testing in Smart Campaigns

Don't see how this would be related to the Send Email step — random sampling or not.


Are you sure people didn't go over communication limits? What are the Smart List criteria? Did people run through the same SC (an earlier version perhaps) and are set to only qualify once?