Batch Email Campaign with Random 50% Holdout Test Cell

[Douglas_Meierdi]

Hello Marketo Community!

I'm sending a campaign to a specific audience (registration source+timeframe+no further activity) and want to send it only to a random 50% of said audience so that I can compare if there's any significant increase in conversion between the two groups.

I initially set up an email program, but how do I hold-out a random 50%? Is it possible to do with the A/B Whole Email Test and just not select an email?

Thanks in advance

[Devraj_Grewal · Accepted solution]

Doug,

Are you testing content? If you are using the same content but testing the engagement between the two groups, I don't see the benefit if they are randomly selected.

If you are looking to perform a random sample, it can be accomplished with a flow step in a smart campaign like this:

1. Send Email: 8/10

Choice 1: If random sample is 50%

Default: Do nothing

2. Remove from flow

Choice 1: If was sent Email 8/10

Default: Do nothing

3. Wait 1 hour

4. Send Email

This randomly sends the email to 50% of the group and waits one hour before sending the email to the rest of the group.

[Douglas_Meierdi]

I'm testing offer. It's an audience from a registration source that is high volume and low conversion to customer. So if I set up the smart list to identify this audience and then use the Send Email flow step with Choice 1 Random Sample is 50% - it will only send the email to 50% of the audience I created in the Smart List - correct? I don't want to send it to the remaining 50%.

[Devraj_Grewal]

Yes, using the random sample will work as you've described and make sure the default choice is to "Do nothing". If you are only sending the email to the random 50%, then use an email performance report and in the report's smart list, just use the filter "was sent email" and select this email in question.

[Douglas_Meierdi]

Tested it in our Sandbox - works perfectly - thanks for the assist Devraj!