How to overcome communication limit while using AB testing

[Yuvraj_Chavan]

If anyone has an idea about this issue? Please suggest the solution.

Thanks in advance....!

[Veronica_Holme4]

Hi Yuvraj - that's not quite enough info to offer you any advice. Can you please outline a scenario in detail you're trying to solve?

[Yuvraj_Chavan]

Hi Veronica Holmes

I am a new user, so please help me to resolve this type of issue.

It is a general question if I am working on AB testing and now I am going to send the email to 10% of my total audience, but as per my understanding, due to communication limit, an email will not be delivered to most of the lead. this means its direct effect on AB testing result. right? 

so, my question is how we can overcome this issue while doing AB testing?

Thanks,

Yuvraj Chavan

[Dan_Stevens_]

Communication limits affect all email communications - not just A/B testing.  Those limits are defined by you (or your admin).  Are you concerned because you often include your entire marketing database in all emails and send several each week?  I'm still not quite sure why you perceive this as an issue with A/B testing vs. overall email sends.

[Yuvraj_Chavan]

Hi Dan,

Thanks for your response,

Let me correct,

We have one campaign request, In that, we need to perform AB testing. (we have Email 1 and Email 2.)

1) we have 1000 leads as a target audience.

2) Need to perform this AB testing on 10% of actual TA. (means 100 leads)

3) If we run this AB testing for 100 leads. that means 50 leads will get Email1 and remaining 50 leads will get Email B.

Right?

Question: if most of the leads (from 100 leads) will not getting an email because of communication limit. it will affect our AB testing result.

then how we can overcome this issue.

[Dan_Stevens_]

Yuvraj, what makes you believe that the leads will not be able to be sent the email as a result of communication limits.  Let's say you have your limits set to 1/day and 3/week.  As long as no one in your targeted list has received an email for that day (or week), the A/B test will run as expected.

[Keith_Nyberg2]

Hi Yuvraj,

A/B testing allows you to defined success based on the metric you choose and compare results of A to B. So my recommendation is to run the test with your desired target. Once you see the results you can determine if the sample size is large enough to be statistically significant for your purposes. If it isn't, work to get your audience size to a number that you are comfortable with. At the end of the day, regardless of the sample size, the test will show you performance of A compared to B, so whether it's 100 or 1000 people, you should still get some insight into what is performing better. And remembering that behavior of a sample typically mirrors the behavior of the larger population.

[Veronica_Holme4]

Yeah... I agree with everything that everyone is saying - Communication Limits are there to protect the customer. You override them at your peril.

You can't disable comms limits just for the A/B test. You can disable comms limits (if you have permission to do so) for the entire email program which would include the A/B test. Enable/Disable Communication Limits in an Email Program - Marketo Docs - Product Documentation

My advice? Don't go down that path. Follow Keith Nyberg's suggestion and experiment with your sample size instead to get something you are happy with.