How does "Random Sample" work?

[Phillip_Wild]

I've seen "Random Sample" in Flow steps when you use "Add Choice". But...is this self contained random samples, or is it reducing the amount of leads flowing through as it goes?

As an example:

Choice 1: If Random Sample is 33%, send email A

Choice 2: If Random Sample is 33%, send email B

Default: Send Email C

Would this work? Does the second random sample start afresh from the full amount of leads, or is it smart enough to remove anyone who qualified for Choice 1?

My theory is that it will remove as it goes, which means you would need to adjust the percentages. So if you started with 100 people:

* 33 people qualify for Email A and get that send. 67 remain.

* 33% * 67 = approx 23 people then qualify for Email B and get that send.

* The remaining 100 - (23 + 33) = 44 people will get Email C.

So if you wanted to get three splits of 33% each, the second choice should say 50%.

This is basically just a more difficult way to do an A/B test through that functionality, so I wouldn't necessarily use it like that - but I'd love to know 100% how the functionality works so I can think about how to use it.

Thanks, Phil

[Anonymous · Accepted solution]

Hi Phillip Wild​!

I struggled with this at first too. You have the right idea though.

With the random sample flow step, you'll need to be sure that all choices equal 100. In your example above, Choice A would get 33%, Choice B would get 33% and Choice C would have to get 34%. Also, you will need to add the various sample choices to a static list instead. You are unable to directly send the email in the random sample flow step. Once you add the members of each sample to a list, you create the email send flow step and add the constraint "member of list".

You may want to add a small wait step depending on the size of the send. I typically like to add a 2 minute wait step.

Hope that helps! Let me know if you have questions.

[Phillip_Wild]

Hi Christina

Interesting. So despite the fact you can add “Random Sample” to a choice step in a “Send Email” step in a flow, you still can’t actually send an email in that fashion?

I think I know what you mean - but just to clarify, you would need to do the following in a flow series:

• Add to List

-Choice 1 - If Random Sample is 33%, then add to Static List A

-Choice 2 - If Random Sample is 33%, then add to Static List B

-Choice 3 - If Random Sample is 34%, then add to Static List C

Wait 2 mins

Send Email

-Choice 1 - If Member of List Static List A, then send Email A

-Choice 2 - If Member of List Static List B, then send Email B

-Choice 1 - If Member of List Static List C, then send Email C

Is that right?

Thanks, Phil

[Anonymous]

Exactly right! That will work.

Side note: Under the "send email" flow step listed above the last choice should be choice 3 - probably just a typo.

[Anna_Bruning1]

Christina Reynolds​, are you talking about email (mailbox) programs? I know you can't use random samples in the smart lists for those programs, but I have used random samples in flow steps of regular programs to send emails so I can confirm that it works.

[Justin_Norris1]

Phillip Wild​ My experience is the same as Anna Bruning​ in that you can use random sample directly in the "Send Email" flow step. Christina Reynolds , where did you encounter that restriction?

The random sample divisions are based on the original total as Christina Reynolds described. Each percentage is a percentage of the original 100% and the "else" option sends to the balance of the sample after the previous percentages are removed.

[Anonymous]

Thanks Justin Norris​ & Anna Bruning​ for pointing that out! We first started using "random sample" last year for A/B testing in default programs, and we weren't able to add it as a choice in the email flow step for a default program. I had never gone back to look at it again (stuck in an old habit I guess). I saw your comments this morning, so I tested it out and IT WORKS! Hooray! Thank you for the comments.

Phillip Wild​ - Skip the add to list flow step and send straight from the send email flow step!

Send Email

Choice 1: If Random Sample is 33, Email A

Choice 2: If Random Sample is 33, Email B

Choice 3: If Random Sample is 34, Email C

[Phillip_Wild]

That makes sense! It seemed weird that the option was there but I couldn't use it.

Thanks all.

Phil

[Josh_Hill13]

Guys,

Your choice step should be

Random Sample=33% to List A

Random Sample=33% to List B

Default to List C <==this will send the remaining 34% to List C.

I would be very careful about doing this any other way.

Yes, you can use this tool in most flow steps. You do not need to do this in an Email Send AB Test.

[Anonymous]

Hi Josh,

Can we get individual analytics for each sample send?

For example, I've been using the A/B testing method but I was told that I cannot get the individual stats (opens, clicks, bounces, unsubscribes, CTRs...) for each email within the test. I need to find a way to test different whole emails and be able to pull these stats.

Thanks!