AB testing for batch and blast emails in a smart campaign

[ashah123]

Hi Team ,

I am willing to create an A/B testing setup for our batch and blast email sends through a smart campaign. We use a smart campaign because we send 4-5 emails each month as part of a monthly campaign setup.

I researched several blogs and posts, which suggested that A/B testing for batch and blast can only be done through the options in the email program setup tab, as shown below.

I also discovered a champion/challenger approach, which is only applicable in engagement programs or for triggers.

I would like to know if we can implement A/B testing for our batch and blast emails using the individual smart campaigns we utilize for our monthly sends.

Looking forward to some option if they are available to leverage within the smart campaign for us to leverage and some documents if you could share which will guide us in the correct direction

Regards,

Akshat

[uditmathur]

Hi @ashah123 ,

My bad for the earlier wrong reply.

You can not directly create an A/B test in the smart campaign, but you can use Random sample in the Add Choice set the %age you want to send your different email to & select respective emails. Than based on the engagement matrix you can use that as winner.

Although, i will recommend to use the email program & use your different emails as variants for A/B test.

Hope this helps. 😊

[ashah123]

Hi @uditmathur ,

We are leveraging email program but since we have multiple emails to be sent in a month we are leveraging smart campaigns to send those emails 

Eg : If we have 4 emails in a month we create 4 different smart campaigns to send those emails

and we want leverage AB testing for the above use case

The one you are referring is withing the email program but it will work well when we just have 1 email to be sent for that campaign

Hope this makes more sense to what i added

Do let us know if there is a way to handle AB testing withing a smart campaign 

Parameters : subject line and whole email test is what we are looking for currently

[ashah123]

Hi @uditmathur ,

Yes we are already using random sampling for throttling our emails at a certain rate per hour .

But dont you think we still need remove from the flow step to exclude those who already received..as i saw on one of the posts earlier

Below is what we are doing currently for throttling

But coming back to my original ask , how do we determine the winner , is it something we will have manually decide and create another smart campaign to send the winner email ?

Like you mentioned before where do we set the engagement matrix in the smart campaign ?

Reagrds,

Akshat

[uditmathur]

Hi @ashah123 ,

Yes you will have to manually decide the winner based on the engagement matrix (opens, clicks, OTC ratio etc.) & will have to create a separate SC to than send the most engaged email to the rest of your audience.

You can get email matrix from the email performance report. 

[ashah123]

Thankyou for the response , that is what i wanted to confirm 🙂