Trouble with A/B Testing Results

Anonymous
Not applicable

Trouble with A/B Testing Results

Hi all. Has anyone experienced issues with A/B testing results not being sent to an entire database? I am working to submit a support ticket, and I was wondering if anyone had any tips or similar experiences with A/B test results not working properly.

Tags (1)
11 REPLIES 11
Sara_Greaves
Level 5

Re: Trouble with A/B Testing Results

Hi Jack,

Can you clarify your question a little further? Do you mean that a "winner" was not sent to the rest of your target list?

Anonymous
Not applicable

Re: Trouble with A/B Testing Results

Hi Sara,

Correct, I don't think the winner is being sent to the rest of the list. When I look at the mailbox icon, it still has a clock around the icon.

Sara_Greaves
Level 5

Re: Trouble with A/B Testing Results

Okay. How long ago did you set up the email? I notice that I get that symbol sometimes if I don't refresh after launching a campaign. I know that's not a technical response, but it's worth mentioning. We can also investigate your set up. I'm assuming that you were doing a test and set the test percentages. What percentage of your target list did you send to each group? If you did a 50-50 split, there's no leads left in your target list to send the winner to. The more detail you can give, the better

Jessica_Kao3
Level 10 - Champion Alumni

Re: Trouble with A/B Testing Results

How long did you set the test up for?  The minimum is four hours.  So if you launch the test at 6am, the winner at the earliest won't be sent until 10am. 

Did you have the winner declared manually or automatically?

Anonymous
Not applicable

Re: Trouble with A/B Testing Results

Hi Jessica,

We set up the test so that after 24 hours, it would go to our audience. Attached are two screenshots of a quarterly newsletter that didn't go out to the remaining of our database.
Screen Shot 2016-11-11 at 2.52.11 PM.png

Screen Shot 2016-11-11 at 2.52.37 PM.png

Anonymous
Not applicable

Re: Trouble with A/B Testing Results

We also set up on our end that the winner would be automatically sent. Is there an option in Marketo to review that?

Jessica_Kao3
Level 10 - Champion Alumni

Re: Trouble with A/B Testing Results

It does look like the test is still running.  I think the test is  stuck which would warrant a support ticket.   You can see that the test is still running and a winner hasn't been declared.   I'm assuming clicks to open with a 3 subject line test.  Time to call support on this one I think. 

Dayna_Wellman3
Level 3 - Champion Alumni

Re: Trouble with A/B Testing Results

I agree with Jessica, I think the program is stuck. Email programs follow the Campaign Limit that is set up in your Admin. You can read more about it here. If your email campaign is trying to send to more people than your Campaign Limit allows, it will fail. They usually will fail on their own, but it might take a while.

Dayna Wellman
Campaign Operations Consultant @ Etumos
Anonymous
Not applicable

Re: Trouble with A/B Testing Results

Thanks, everyone, for providing insight. We opened a Marketo case, and according to support, it looks like the email wasn't in the approved state. I sent them the attached screen shot, so hopefully they will provide some insight. Screen Shot 2016-11-30 at 4.09.18 PM.png