What's a control group?
A control group is established in order to facilitate tracking the impact of a marketing campaign. It is executed by selecting a random sample from the target audience and excluding them from the marketing tactic. This allows you to filter out the effect of those individuals who might have purchased anyway as well as the effect of other channels that might be difficult to measure, such as TV or radio advertising.
What are the options for control groups in Marketo?
There are several ways of structuring control groups in Marketo. However, only one approach provides complete capabilities for program-level reporting, so if you want to report in Marketo (particularly in Revenue Explorer), having separate programs is your best choice. The chart below compares the four major approaches:
Lead Custom Field | Static List | Program Status | Separate Program | |
Can be in multiple control groups at a time | N | Y | Y | Y |
Works for both campaign and tactic-level reporting | N | Y | Y | Y |
Visible in RCA | Y | N | Y | Y |
Usable in all RCA program reporting | N | N | N | Y |
Example
To see how this would be executed in Marketo, we will use the example of Campaign A, which will have a control group, and which consists of two marketing tactics: Tactic 1 (an email) and Tactic 2 (a direct mail), each of which will also have a control group. For the purposes of this example, we will set the control group at 10%.
Program Structure
For Campaign A, we create two programs – one to contain the eligible audience and one to contain the control group. We do the same each of the tactics. The channel utilized for the campaign audiences is operational, while the channels utilized for the tactics are selected based on the type of tactic (Email and Direct Mail).
Selecting Control Groups
Selecting control groups is done using Marketo’s random sample functionality, which allows you select a percentage of a target audience defined in a smart list and then take some action.
To select the campaign control group, we define our audience in the smart list and then run a Change Program Status flow action to select a random sample of 10% and add that group to the Campaign Control Group program. The balance of the audience is added to the primary program.
To select the tactic control group, we define our audience in the smart list based on membership in the primary campaign program.
In the flow, we then select another 10% of this remaining audience to pull out into a control group specific to this tactic. The balance is placed in the program that will complete the outbound activity.
The end result with 111 people who are initially eligible for the campaign would look roughly like this:
Reporting
In reporting, the outbound program and the control group program can be viewed side-by-side for comparison purposes. To facilitate this by ensuring the programs show up next to each other in the reporting, I recommend naming them virtually identically, with the addition of CG at the end to indicate which program is for the control group.
Each program should also be tagged as being a member of that specific campaign:
This will allow you to group the control groups and the active programs to compare success rates and pipeline/revenue generated:
Note: There may be one small flaw here with my operational control group programs not showing up in the Program Opportunity Analysis report. I'm still testing various things to figure out exactly what's making them not show up. I'll update this once I figure out the best approach.
If desired, these programs can be synced to Salesforce using the Salesforce campaign sync.
.png)