A/B Testing with dynamic content in emails

[Scott_McKeighe2]

Hey community,

I'm trying to build out an Engagement Program containing A/B tests running on emails that also contain dynamic content. So far, I haven't been able to get the dynamic content to show up, though the A/B tests themselves show fine (it just treats the email, dynamic content or no, as static).

The real use case is that we'd like to test different criteria, but have the emails shift contents based on a segmentation scheme like Language Preferences. English users would receive the English language set of tests, French would receive French, German would receive German, and so on.

Is there a way to do this without a separate engagement program? I could probably use Languages as streams, but we'retrying to use streams for timezones. Hence the question.. 😉

Any help would be most appreciated.

Thanks!

[Josh_Hill13 · Accepted solution]

I believe you would need separate emails for each language if you wanted to AB test. What are you trying to AB test in the email?

[Scott_McKeighe2]

Hey Josh,

We're trying to test a number of different things - subject line, preheader, layout, CTAs, etc., depending on the send in the program. I was just hoping there was a tidier way to construct this using localization/language via dynamic content through segments.

[Anonymous]

Hey Scott,

How did you operationally set up the engagement stream so that you could test the subject line, preheader, CTA, etc?

I am unable to add in an email program into the engagement stream so it is impossible to test different emails- trying to get more data driven with our nurture campaign..

Any help is appreciated.

Best,

Amber

[Scott_McKeighe2]

Hey Amber,

The easiest route I've found is to put a Program into nurture stream , and use a Smart Campaign to conditionally flow an email to my target audience segment. I've used Random Sample to do strict A/B here a lot, and I've also targeted different data values (like Language, Metro Region, Is Customer, etc.) to send a different version as well.

Depending on how many streams your Nurture Program has, you could also create these kinds of tests within groups, e.g. an "English-speaking" stream vs. "French," with different A/B tests or variations nested underneath.

An important note would be to be mindful of how Marketo interprets what content to cast when someone's in a nurture stream. If it's strictly an email (not a program), Marketo checks to see whether that target was sent the email prior, and if so, skips. If it's a program, it checks whether that target is a program member, not necessarily if they've seen the email.  To that end, if you're using streams and programs a lot, it's a crucial step to create an Exclusionary membership status for people who should be skipped from a potential piece of content, and add people to programs accordingly. Depends, of course, on your business needs.

[Dan_Stevens_]

Depending on how you plan on excluding members from those programs, you need to be aware of some issues that can occur (leads being stuck in an endless loop without moving on to the next program) when using smart campaigns to qualify whether or not someone should receive a specific email.  See here for more information:

Re: Excluding leads from nurture case based on asset already downloaded

Nested program in nurture - smart campaign email sends - leads getting stuck in loop?

Re: If a lead doesn't qualify for an Engagement Program cast, does it proceed to the next email?