We want to run an A/B test on an outgoing email to our prospects database to promote a whitepaper download.
However due time sensitive nature of the whitepaper, ideally we'd like to dispatch the email as soon as it's published.
So plan is to run an A/B test, using an email programme, but increasing the test sample size to 100% - i.e. reaching everyone on the mailing list, without waiting do declare a winner. (Idea is to use test results for future campaigns).
Curious to know if anyone has used the email programme this way? I'm struggling to think of any negative impact. Any thoughts/insight are welcome.
Hi Halid, I'm not sure if it was an isolated incident that I had because Marketo never was really able to clarify, but last time I tried to send my sample to 100% of my test group, it only sent to a percentage (I want to say it was about 75%), and then it sent the winner to the remainder of the list at the end of the test.
This could have been an isolated incident, and maybe sending to 100% of your DB at a time works, but since this issue I haven't tried it using email programs. I set up two separate emails, and use a flow step that says "If random sample equals 50% send email A, otherwise send email B". This is also nice for me because I have my standard reporting which I prefer to look at.