Random Sampling in Email Campaigns

[Anonymous]

Have been using the random sample functionality to send subject line/creative tests for a little while now. We were under the impression that when using random sampling in a single campaign that there would be no overlap between random samples. Found out this week that this is not the case.

For example if you are sending out a subject line test to two 50% random samples, there could be people that receive both subject line tests. What needs to be done to avoid this possibility is to add an additional choice in your request campaign Flow step, "Was Sent Email".

Marketo doesn't allow duplicate Emails to be sent for a particular campaign and has the qualification rule that specifies each lead can run through the flow once, so it was surprising that this scenario could occur. 

Has anyone run into issues w/ random sampling sending to the same person twice within a campaign?

[Anonymous]

That shouldn't happen to the same email address.  Someone might receive two emails if Marketo has two separate email addresses for the same person.  For instance, kelly@marketo.com and kabner@marketo.com are two different people to Marketo, so if both are in the same campaign each will get an email, but both come into my single inbox.

If it is two separate email addresses within the same campaign, and the campaign is set to Run Once, then please submit it to support.

[Anonymous]

Remembered there was additional item, this is all possible when there is a wait step in the flow and using random samples.

[Anonymous]

Yes, you're correct.  Marketo will send another email in a different step.  To solve this Add a Remove from Flow action to your flow:

1. Send Email - Random Sample 50% email 1, remainder get email 2
2. Wait 3 days
3. Remove from Flow if was sent email 1 or email 2 (from step 1)
4. Send email  (1or2 -- you'll need to add the winner of the test, but put in a placeholder step and email when you create the campaign.

[Anonymous]

I'm sure there is a reason, but wouldn't be more logical if a campaign that did include a wait step followed the same logic as a campaign without a wait step? Even our account contact thought that was how it worked.

[Anonymous]

Yes, just use two different campaigns. One to send the test, another to send the winning email to the rest of the list.  Use the wait step if you want to do it all in one campaign.