Nesting programs in nurtures: best way to exclude prior email sends in smart list?

[Jasmine_Foruzan]

Hi,

Kelley Sandoval and I are considering converting all the emails in our nurture streams to nested programs housing smart campaigns. One of the reasons we're considering this is to improve our A/B test reporting, as we found that using the traditional Champion/Challenger test for our nurture emails resulted in the following issue: When we ran separate one-off email blasts that referenced these nurture emails, our A/B test results for those nurture emails were inflated on one side (since the email was referenced in the blast, but only one version of the email actually ran since Champion/Challenger does not work for batch campaigns, making it appear as though that version got more opens and clicks). We found a workaround by creating 2 separate reports that look at each test version, excluding the email blast campaign, but this is not scalable.

Keeping that in mind, what is the best way to set up programs within nurture streams to ensure that our A/B testing reporting is efficient and we are also excluding anyone who has already received the email via one of our blasts (which are separate batch campaigns). At first, we considered putting a "was sent email" exclusion in the program smart list, but Marketo documentation states the following: "If the engagement program calls a smart campaign within a program, people who don't meet the smart list criteria won't receive the email. They also won't move on to the next program in the current cast or in subsequent casts. Essentially they'll  be stuck in an infinite loop, where they never qualify. You may want to use choices on flow steps to filter people out instead of adding these criteria to smart lists when you nest programs inside of an engagement program."

We want to ensure that individuals who later become members of our nurture programs do not receive the same email they received in a prior blast outside the nurture. At the same time, we also want to make sure they are able to progress to subsequent programs within the nurture stream. Any ideas on how to best do this?

[Anonymous · Accepted solution]

Hey Jasmine Foruzani​!

We use default programs to a/b test our nurture emails! This is another reminder for me to write up a blog post on this

First, I recommend creating default programs, but *not* nesting them in the engagement program. Create a folder, and put them outside! This way, you can use the same emails in various engagement programs. Meaning, you can create a generic "ecommerce ebook" program, and it can be used in your lead nurture, and your e-commerce customer nurture, and you don't have to clone a bunch of programs.

not like this:

Smart List: the Smart Campaign's smart list should be "Member of Any Engagement Program".

Flow: use the "random sample" option, (I usually use 50/50) and you can A/B test that way!

For exclusion, I recommend creating a program status called "Exclusion", and if someone receives an email, and isn't a member of that program, then you exclude them from the program. Edward Unthank has a good video that describes how/why this works: Intelligent Lead Nurturing (MUS 14) Edward Unthank

Let me know if you have questions!

[Jasmine_Foruzan]

Thank you Allison Sparrow​ that was helpful! Have you heard of anyone using a "remove from flow" step in the program's send email smart campaign to remove anyone who has received the email in the past from the smart campaign flow? I tested this with a nurture stream and it seemed to work where I did not receive the 1st email (which I had already received in a prior blast), but I received the emails in the subsequent programs within the stream. Any thoughts on if this is actually effective? Thanks!

[Anonymous]

Jasmine Foruzani​ that could work, just not sure it's the most elegant option. I think ideally you want to create a campaign that doesn't qualify people that will automatically get removed. But yes, it would make sure leads wouldn't receive it!

[Anonymous]

Any thoughts on mixing traditional nurture with nesting? For example, We have a simple engagement engine with 11 emails and we're a/b testing 2 emails. I want to nest the 2 that I'm a/b testing but not the other 9. Would the stream cadence just pick up the non-nested emails after the nested programs run?

[Robb_Barrett]

I'm working on a blog post about how to create a kick-butt nurture program.  Stay tuned.

[Danielle_Wong]

Ever publish it? Would love to read. Thanks!

[Robb_Barrett]

I don't know if you'll be able to read this since I can only publish blog posts to the Chicago user group for now:

Length Matters: Engagement Programs that Don't Finish Too Quickly...

[Danielle_Wong]

Thanks Robb. I was able to view it! I bookmarked it for later