Champion/Challenger vs A/B Test in Engagement Program

[S_S]

Hi everyone! We are working on building our first Engagement Program, and we want to test the subject lines on the emails. I've seen conflicting info about champion/challenger and a/b and whether you can do them with batch campaigns or only triggers.

Our ideal scenario: create a stream with 5 emails in it, test subject lines within the engagement program (so send out subject A to half, send subject B to half, all still within the engagement program, and see which does better). We don't want the "winner" to be sent to the people who received the non-winner because they already received the email in the stream. Half would get email 1A, half would get email 1B, and then they would just continue in the stream. Then upon next cast, half would get email 2A and half would get 2B, all the way through the 5 emails.

Once we get a decent amount of people through the stream, we would then stop the testing and just use the winner for each email in the engagement program moving forward.

Any ideas on best way to set this up? Is it as simple as below? I just created an A/B test and then dragged Attract More Customers Webinar into the stream. We'd use batches to put people into the stream.

 

Hope that makes sense - thanks in advance!

[Tracey_Bartz1 · Accepted solution]

What you have in your screenshot is called a champion/challenger test. It is how we do testing within the stream of an engagement program. We don't call it A/B testing because that term is reserved for email programs. 

Your ideal scenario is exactly how champion/challenger works. 

Champion will be your version A and challenger will be your version B. In the setup screen you'll want to set your slider to 50% receives champion and 50% receives challenger. Then in every cast the test will be sent out 50/50 to whoever qualifies for that email in that cast. 

The important distinction between champion/challenger test and A/B test is that champion/challenger is an ongoing test (it runs in every cast until you declare your winner). An A/B test on the other is a one-time test because it's in an email program which is a one-time send. 

Once you declare your winner in champion/challenger that's the version that will be used in the stream going forward for 100% of cast recipients. And because of stream functionality people who received one of the versions during the test period will not receive the winner again once it's declared. They just keep moving through the stream. 

[S_S]

Thank you! This is exactly the info I needed. Appreciate the quick response.