Standard Champion Criteria - Champion/Challenger Test

[Mariana]

Hello,

I´m thinking of doing a champion/challenger test and wanted to know if there is champion criteria recommendation - between click% and engagement score.

Also, do you happen to know if the engagement score in this test takes into account engaged behavior (Open, Click, Program Success) or  based on program membership and success status.

Thank you!

[Disha_Goyal6]

Hi @Mariana, please see my comments for your query

Query - I´m thinking of doing a champion/challenger test and wanted to know if there is champion criteria recommendation - between click% and engagement score.

Ans - Click to Open % and Engagement score both have their pros and cons.

Click to Open % - Percentage of emails that were opened and had a link clicked in the email. Ii directly measures the immediate interest generated by your content. It indicates which variant drives more immediate action from viewers. But it does not measure the quality of interaction after the click. High CTR does not always correlate with meaningful conversions.

Engagement Score
Definition: A composite metric that evaluates how users interact with your content beyond just clicks. It reflects the quality of engagement and user interest. It helps understand how users interact with content after the initial click. It take more time to calculate, To give people a chance to engage with your content, the score is calculated 72 hours after the email is sent.

Recommendations :

> If your primary goal is to drive immediate actions such as clicks to a landing page or sign-ups, CTR might be more relevant.
> If your goal is to foster deeper engagement with your content, an engagement score will be more meaningful.

For advertising campaigns, CTR might be more critical, while for content marketing, engagement score can provide better insights.

Query - Plus, do you happen to know if the engagement score in this test takes into account engaged behavior (Open, Click, Program Success) or based on program membership and success status.

Ans - Engagement score is calculated based on user engaged behavior (Open, Click, Program Success) and disengaged behavior (Unsubscribe).
But When programs are used as content in streams, the engagement score is based on program membership and success status, not email interaction (clicks, opens, unsubscribes).

Thanks,

Disha

[uditmathur · Accepted solution]

Hi @Mariana ,

To add to the above comment, Click to open ratio is mostly used when you are doing an Subject Line test.

Also, i will advise to avoided CTO ratio be used as a success criteria since many of the email scanners do a dummy click on the email sends & also with increased threats because of BOT activities these metrics could be highly elevated & if used as success will not be correct.

Hope this helps