A/B testing Questions

[Ketki_Sharma]

1. I am planning to setup an A/B test to test click rates by day of the week. So in that case Date/Time-based A/B test would be appropriate? I have a question from this page which is one of the final step of setting up the test - Schedule the A/B Test - Marketo Docs - Product Documentation .

It calls the two variations of experiment as test and winner. What is winner in this case (since we are still setting up the test, we don't have a winner). For example, we are currently sending our campaigns on Wednesday, but we have a hypothesis that Friday might be better. So we are planning to test wednesday vs friday. Which day would test and winner in this case?

2. It says on this page - Define the A/B Test Winner Criteria - Marketo Docs - Product Documentation that "Once the A/B test is over, Marketo can automatically send the winning email at the scheduled time, or you can review the results and decide which email goes out when." How do I stop marketo from sending the winner variation automatically? I do not want to send the winner variation to the rest of my recipient list (which did not take part in A/B test)?

[Josh_Pickles · Accepted solution]

Hi Ketki,

If you're using the AB testing feature, you can drag the Test Sample Size slider to 100%, meaning that 50% run through day 1 and 50% through day 2. 

If you set up the slider this way, you technically never have a winner as defined by the AB testing feature. 

If you're looking for more insight on AB testing, I really recommend watching the Making the Most out of your A/B Testing recording of Jessica Kao at this year's Adobe Summit -> Adobe Summit 2019—The Digital Experience Conference | March 24–28, 2019 

[Ketki_Sharma]

Thank you! We set up the test as you recommended.