AnsweredAssumed Answered

Random sample flow step not very random

Question asked by Phillip Wild on May 9, 2018
Latest reply on May 16, 2018 by Sanford Whiteman

Hi everyone

 

A question about using Random Sample as a flow step. I had a flow step that looks like this:

 

Add to List

Choice 1: Random Sample is 50 - add to List A

Default: Add to List B.

 

We then used those lists to report on bookings, as an A/B test.

 

So here's the weird thing. We have a custom field in Salesforce called "Agency Secondary". I grouped the people in each list by this, and here's a sample of what I see:

 

Agency SecondaryList AList B
Agency 10650
Agency 210550
Agency 32460
Agency 40116

 

This does not look very random to me! My guess is that Marketo is taking large chunks of my total list and then allocating then to lists in, say,  1,000 person chunks. Not cool.

 

What should I be seeing here, and is there any way I can make this closer to random? I know that "random" is an interesting topic in computing so it's never going to be truly random, but the above seems a bit ridiculous to me. That's fine if you have consumers where different people in a given chunk of 1,000 shouldn't show similar behaviour, but at a particular travel agency (which is what I'm dealing with) there would be specific booking demographics of clients that could easily sway any A/B test I'm looking to run.

 

Any ideas?

Outcomes