AB testing in engagement program

[shaniala]

Hello, may I ask what is the best way to do ab testing in an engagement program? Champion challenger email or the random sample? Please share any related article or videos, thank you

[Neha8]

Hi @shaniala,

The champion/Challenger would be better for the testing of the email. Here is the link: https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/email-marketing/general/functi...

Champion-Challenger is Better When:

• You Have a Proven Benchmark: You want to test a new variation against an established best performer.
• Incremental Improvements Matter: You’re looking to make incremental improvements based on an existing high-performing version.
• Simpler Setup: You prefer a simpler setup with fewer variations to test directly against a known quantity.

Conclusion

The best approach for A/B testing in your engagement program depends on your specific needs:

• For targeted comparisons and incremental improvements with a focus on a known benchmark, the Champion-Challenger approach might be more suitable.
• For comprehensive testing of multiple variations and obtaining statistically valid results, a Random Sample approach is generally better.

Thanks,

Neha 

[uditmathur]

Hi @shaniala ,

You can use Champion/ Challenger test Emails only in your Engagement program streams if you are using emails nested in a Programs within streams C/C test is not supported.

If you are having nested programs in your stream only option you have will be the Random Sample. 

Also C/C only works with Trigger campaigns. 

You can read how to setup C/C test here

Hope this helps 😊

[Ruchi_Lapran1]

Go with Champion/Challenger approach if you are seeking a quick feedback. If you want to conduct a more thorough analysis with reduced bias and statistical conclusions, random sampling A/B testing is recommended. To further deepen the criteria, and you have a global audience to target, you can also consider including the timezone send feature during testing.

Do consider that the need to perform A/B test should be goal oriented and you have an audience or seedlist qualifying for the A/B test.