A/B Testing: 3 Options in Marketo

[Joe_Reitz]

Hi everyone,

To open up the Nation Talks series, I wanted to share a quick overview of one of the most useful tools Marketo offers: A/B Testing.

There are three main avenues you can take, and a big part of which type of A/B test you run is tied to what program type you're operating in. This video highlights how and when to use an email program A/B test, how champion/tests work in default & engagement programs, and ends on a "random sample" use case.

This was a blast to create, and I look forward to seeing how we as a community can continue to sharpen one another!

To learn more about The Nation Talks!

[Ouida_Linkous]

With the champion/challenger method, how do you schedule?  I can find documentation no how to choose and clone the challenger (which is weird, why do I have to clone the challenger email.  Why can't I just choose it?).  What I cannot figure out how to do is ever run the thing?  I am sure I am missing something super obvious, but I cannot get to the bottom of it. 

[Anonymous]

Hi Joe,

thanks a lot for your comments and recommendations. It's very helpful and I appreciate that.

All Best,

Jan

[Joe_Reitz]

If the webinar is a recorded event, you could just set up an email using either the champ/challenger tool or the random sample trick in the video.. but you can't embed an event or email program type into an engagement program.

The reason is because usually these types of programs will be a one-time deal, so it wouldn't make sense to put them in a neverending nurturing stream. The whole engagement engine hinges on being a machine you don't have to drive... just throw people in and let them get fed.

We leave all our webinar & event programs as standalone event programs, and if we want to temporarily promote one in any of our streams, we just build a basic email, and put it at the top of our stream (like below) so that it's the next email ANYONE in the stream receives. Then we edit the availability so it automatically deactivates itself before the next cast.

[Anonymous]

Hi Joe,

we are no new in marketo. We have been doing it that way you suggested. Now, we want to leverage the opportunity to embed batch campaigns (webinars, events and so on) into engagement program. The only question we have is how the smart campaign for nurturing should be set up? Could you pls help if you have any experience with it?

Thanks, Jan

[Joe_Reitz]

If you're just getting started, I'd recommend just keeping it simple. Create a basic engagement program that contains all your emails, set the cadence, and sit back.

[Anonymous]

Hi Joe,

thank you for your response. I think we are not gonna use design studio emails while doing A/B test.

We actually had a problem with Engagement program composed by default programs triggered by smart campaigns. Is there any guide how to do engagement programs by using default programs, please? How should smart campaign look like?

I would appreciate any recommendation.

Thanks a lot,

Jan

[Joe_Reitz]

Hey Jan Vacek​,

You can do Champion/Challenger tests in the Design Studio, as well... But it's not something we tend to do. The only emails we keep in the Design Studio are either templates or auto-responders, like "Thank You's" and alerts.

Anyway, just right-click the email and select "New Test." If you wanted to do random sample for some reason, you'd have to do it at the program level.

[Anonymous]

Hi,

amazing post that claryfied me 3 different methods of A/B testing in Marketo. Thanks for that.

I just have one more question: will Challenger A/B testing work with emails in design studio as well or does it work only with local emails?

Thanks,

Jan

[Joe_Reitz]

mischief managed

Glad you found it useful! There's going to be some truly mind-blowing stuff coming out later this month from some other Marketo Champs, so stay tuned!

[Anonymous]

This is extremely useful! As someone new to Marketo who recently had to revamp our Nurture program, we wanted to do some A/B testing.

I started out doing it the normal way until I ran in to challenges because you can't use the first two methods with that type of program - they simply don't function as they are meant to.

While the Community helped me out with an answer (Random Sample), this video has added some learning for me, and I will definitely be using some listening campaigns to move people into other areas/lists as a result of the actions they take with the emails we are testing.

Thanks for a great resource!

Michelle

[Joe_Reitz]

Yep, that's the idea here.

Nation Talks is a pretty new idea, but stand by for more awesome videos from other Marketo Champs & power users this month!

[Lauren_Beth]

As Marketo is by nature a very visual and process oriented tool, any training on video will be very valuable.  I also like that it is available here rather than in the University which makes it easier to find and use.