SOLVED

Batch Email Campaign with Random 50% Holdout Test Cell

Go to solution
Douglas_Meierdi
Level 2

Batch Email Campaign with Random 50% Holdout Test Cell

Hello Marketo Community!

I'm sending a campaign to a specific audience (registration source+timeframe+no further activity) and want to send it only to a random 50% of said audience so that I can compare if there's any significant increase in conversion between the two groups.

I initially set up an email program, but how do I hold-out a random 50%? Is it possible to do with the A/B Whole Email Test and just not select an email?

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions
Devraj_Grewal
Level 10 - Champion Alumni

Re: Batch Email Campaign with Random 50% Holdout Test Cell

Doug,

Are you testing content? If you are using the same content but testing the engagement between the two groups, I don't see the benefit if they are randomly selected.

If you are looking to perform a random sample, it can be accomplished with a flow step in a smart campaign like this:

1. Send Email: 8/10

Choice 1: If random sample is 50%

Default: Do nothing

2. Remove from flow

Choice 1: If was sent Email 8/10

Default: Do nothing

3. Wait 1 hour

4. Send Email

This randomly sends the email to 50% of the group and waits one hour before sending the email to the rest of the group.

View solution in original post

4 REPLIES 4
Devraj_Grewal
Level 10 - Champion Alumni

Re: Batch Email Campaign with Random 50% Holdout Test Cell

Doug,

Are you testing content? If you are using the same content but testing the engagement between the two groups, I don't see the benefit if they are randomly selected.

If you are looking to perform a random sample, it can be accomplished with a flow step in a smart campaign like this:

1. Send Email: 8/10

Choice 1: If random sample is 50%

Default: Do nothing

2. Remove from flow

Choice 1: If was sent Email 8/10

Default: Do nothing

3. Wait 1 hour

4. Send Email

This randomly sends the email to 50% of the group and waits one hour before sending the email to the rest of the group.

Douglas_Meierdi
Level 2

Re: Batch Email Campaign with Random 50% Holdout Test Cell

I'm testing offer. It's an audience from a registration source that is high volume and low conversion to customer. So if I set up the smart list to identify this audience and then use the Send Email flow step with Choice 1 Random Sample is 50% - it will only send the email to 50% of the audience I created in the Smart List - correct? I don't want to send it to the remaining 50%.

Devraj_Grewal
Level 10 - Champion Alumni

Re: Batch Email Campaign with Random 50% Holdout Test Cell

Yes, using the random sample will work as you've described and make sure the default choice is to "Do nothing". If you are only sending the email to the random 50%, then use an email performance report and in the report's smart list, just use the filter "was sent email" and select this email in question.

Douglas_Meierdi
Level 2

Re: Batch Email Campaign with Random 50% Holdout Test Cell

Tested it in our Sandbox - works perfectly - thanks for the assist Devraj!