Help: Question about A/B testing in a multiple email drip with the ability to remove a lead during the test

[Anonymous]

Hey all,

This campaign has a lot of moving parts so I want to make sure everything will go as planned.

We're doing a drip sales campaign containing 7 emails to a list of contacts and would like to A/B test the titles.

I'm having the campaign separate people into two lists (rather than using "Random Sample" in the Send Email condition) because some of these emails are in response to the one that came before it (RE:blahblahblah), so the title would need to match the previous one that the person received, and if I do the random sample each time on the send, a person who received email 1-A may receive email 2-B and have a confusing email title.

To get this done, I have created a smart campaign which does the following for everyone that is a member of a specific salesforce campaign:

1. Add to list A & B (Random sample of 50% to each)
2. Send Email (If member of list A, send email 1-A) (If member of list B, send email 1-B)
3. Wait until 8AM two days later
4. Send Email (If member of list A, send email 2-A) (If member of list B, send email 2-B)
5. Wait until 8AM tuesday of the following week
6. Send Email (If member of list A, send email 3-A) (If member of list B, send email 3-B)
7. Wait until 8AM two days later
8. And so on until email 7.

First, does this seem like the correct way to do this?

Second, I want to be able to pull people out of the drip when a salesperson selects a field in salesforce, to do this I have a smart campaign using the trigger of if the value of that field is changed to true, remove from flow of the smart campaign i explained above.

Will this actually remove the person from the next email? Or will marketo still pull them in because they're a member of list A or B that I created?

Thanks!

[Devraj_Grewal · Accepted solution]

Bobby,

The process you outlined above will work. And yes, if you use that trigger of the SFDC field with the flow step of "remove from flow" for the smart campaign sending the email, they will be removed. However, make sure the smart campaign sending the emails only allows leads to run through the flow once. That way, once a lead is removed from the flow, they cannot qualify for the smart campaign sending the emails again.

In addition to having that SFDC field change remove leads from the smart campaign flow, I would also think of any other criteria that should remove a lead (ex. Lead Status changes, lead unsubscribes, lead's email address bounces, etc.)

[Anonymous]

Thanks Devraj,

My main worry was that when it would get to the 'send email' step, it would pick everyone it sees in List A or B, even if they were pulled out of the flow with a remove from flow step, since they still sit in the list. What you're saying means that only people who are still in the flow can qualify as part of a list that is referenced by that flow?

Thanks again!

[Devraj_Grewal]

That is correct, so even if those leads are in List A or B, if they are not in the flow, they will not execute those steps.

[Grégoire_Miche2]

Hi Bobby,

This will work but I do not like remove from flow too much as it will also remove the lead as a member of the smart campaign, thus modifying potentially some stats.

I would rather create a smart list R, filter: sales field is true

Then I would change the flow as below:

1. Add to list A & B (Random sample of 50% to each)
2. Send Email (If member of list A, send email 1-A) (If member of list B, send email 1-B)
3. Wait until 8AM two days later
4. Send Email (If member of smart list R, do nothing) (If member of list A, send email 2-A) (If member of list B, send email 2-B)
5. Wait until 8AM tuesday of the following week
6. Send Email (If member of smart list R, do nothing) (If member of list A, send email 3-A) (If member of list B, send email 3-B)
7. Wait until 8AM two days later
8. And so on until email 7.

-Greg

[Anonymous]

Hi Grégoire,

With this implementation, I'd need to also make it remove them from List A or B when that salesforce field is set as true, right? Or else I imagine it would go like this:

It would see them in smart list R and do nothing. Then it would also see them in list A or B in the next part of the Send Email step, and send them the email, right?

[Grégoire_Miche2]

No, since you will always test smart list R, so the "do nothing" will always catch the lead. If you are afraid the sales field does not remains to true, yes, you can remove the lead from A and B, or create a list C and use it to test the "do nothing". This would give

1. Add to list A & B (Random sample of 50% to each)
2. Send Email (If member of list A, send email 1-A) (If member of list B, send email 1-B)
3. Wait until 8AM two days later
4. add to list C (if member of smart list C, add to list C) (else do nothing)
5. Send Email (If member of list C, do nothing) (If member of list A, send email 2-A) (If member of list B, send email 2-B)
6. Wait until 8AM tuesday of the following week
7. add to list C (if member of smart list C, add to list C) (else do nothing)
8. Send Email (If member of smart list C, do nothing) (If member of list A, send email 3-A) (If member of list B, send email 3-B)
9. Wait until 8AM two days later
10. add to list C (if member of smart list C, add to list C) (else do nothing)
11. And so on until email 7.

The principle remains the same: use a list C to mark the lead, but avoid the remove from flow.

-Greg