Does Campaign Queue for Batch Campaigns Affect Email Processing & Delay Sending/Delivery?

[Guitarrista82]

Hello,

We have periodically noticed long delivery delays when sending emails from batch smart campaigns. Yesterday, for example, we had pre-scheduled an email to send to 1.7 million people at 7am MST and we noticed it took until around 7:30pm MST (12 hours later) for them to fully deliver. 

Taking into consideration a couple of factors - large email base, Cyber Week - our leadership wants us to find a solution to avoid such long delays - especially because our sales team is largely dependent upon generating revenue from email but is only available for a certain amount of time during the day.

With this said, I am open to any suggestions the Community can provide (please share!) but also have a specific question.

All the campaigns we use to send emails are batch campaigns scheduled the day before. We typically schedule them for sending at 7 and 8 am.

I've always been under the impression that when a campaign "runs", whether it's pre-scheduled or we use the "Run now" feature, that the campaign immediately starts processing:

However, according to a Marketo support rep I'm communicating with, Marketo can only process two batch campaigns at a time and the rest are queued until the previous ones complete. And in this article https://nation.marketo.com/t5/knowledgebase/how-campaign-processing-works/ta-p/248264 it says that there is a priority to how the campaigns are run with "high priority" campaigns running first in the order they were added to the queue. And it isn't until those campaigns are "completed", that the next highest priority campaigns are executed.

A couple of questions:

1. The table above says "Send email (triggered)" is considered High Priority - by "triggered" does this mean only emails that are sent via Trigger campaign are considered High Priority or do emails sent via Batch campaign get High Priority?
2. The article says "High priority campaigns run first in the order they were added to the queue. Once those are finished, the next highest priority campaigns are executed in time order and so on down until all have completed." Question 1: When it says "in the order they were added to the queue", what is it referring to, when the campaign itself was scheduled or the time that the campaign is scheduled to send the email? So if I have two campaigns both scheduled to send an email at 7 am, which one will take precedence? Question 2: By "finished" does the article mean that the campaign has to completely process everyone first before the next campaign in the queue can begin processing?

As I stated earlier, I've been of the mind that campaigns immediately started "running" when they hit the scheduled time - so if Email A is scheduled to send at 7 am tomorrow, when the clock hits 7 am, the campaign starts processing. But now it seems that if there is another campaign (or several campaigns) scheduled to send an email at the same exact time, one of them is going to take precedence over the others....Is that precedence based on the time the campaign was scheduled? So if I scheduled Campaign A at 4:30 pm today and then scheduled Campaign B at 5 pm, and BOTH campaigns will send their respective emails at 7 am tomorrow, Campaign A will be added to the queue first and get precedence tomorrow?

Lots of questions, but we need to know how we can resolve these delivery delays and if the batch processing is playing a part in the delays.

Thank you,

LK

[Darshil_Shah1 · Accepted solution]

@Guitarrista82, batch and trigger campaigns are processed in separate queues (the article you have linked also states the same). Additionally, the campaign priority override feature that you mentioned is applicable to the trigger campaign only. You cannot override the batch campaign's priority. When several batch campaigns are in a queue, the normal campaign priority is based on the flow steps in each campaign, followed by Marketo (unlike trigger campaigns, in which you can override the priority).

---

A couple of questions:

1. The table above says "Send email (triggered)" is considered High Priority - by "triggered" does this mean only emails that are sent via Trigger campaign are considered High Priority or do emails sent via Batch campaign get High Priority?

Yes- trigger campaigns with a Send Email flow step take the highest priority in the trigger queue. So, if you have more than one trigger campaign in the queue, the one with the Send Email flow step in it would be processed before the one w/o). I would think batch campaigns would also follow a similar priority order, but since the scheduling of batch campaigns is managed by users, I'd recommend you be a little cognizant and not schedule large batch campaigns at the same time/very near to each other such that it'd overwhelm the instance and cause campaign execution delays.

---

1. The article says "High priority campaigns run first in the order they were added to the queue. Once those are finished, the next highest priority campaigns are executed in time order and so on down until all have completed." Question 1: When it says "in the order they were added to the queue", what is it referring to, when the campaign itself was scheduled or the time that the campaign is scheduled to send the email? So if I have two campaigns both scheduled to send an email at 7 am, which one will take precedence? Question 2: By "finished" does the article mean that the campaign has to completely process everyone first before the next campaign in the queue can begin processing?

This is referring to the trigger campaign queue IMO- when multiple trigger campaigns are fired simultaneously, Marketo processes them as per the priority order (default/overridden). If you have multiple batch campaigns scheduled at the same time, I'd think (and supported with some testing), that both the campaigns would run at the same time (i.e., at the scheduled time), however, if there are too many people in each (and additional things that'd put a strain on the instance - complex smart list, number of flow steps, choice steps and their complexity), you might see apparent delays in the campaign processing. By finished, the article means that once all the higher priority trigger campaigns in the queue are done processing (i.e., all flow steps in all high priority campaigns are done executing for all the qualified people), Marketo takes up the next-in-line priority campaigns (i.e., the ones with the Medium priority and then the ones with the low priority).

As a solution to your problem, you should use the Email Send program with the Headstart feature turned on. It gives you that option by processing the program in advance. With Head Start selected, the program will begin processing approximately 12 hours prior to the scheduled time. Once processing starts, the program is locked. Since people are processed 12 hours prior to the scheduled time, this is more likely to deliver emails to people on time than regular batch campaigns that process people in the smart list right at the time of the scheduled send. As said earlier, you should work on scheduling campaigns such that at least the ones with a lot of people in them don't clash/are scheduled right next to each other so you have minimal send and execution delays. Maintaining a campaign calendar is effective- Marketo also has one available OOTB. 

[Guitarrista82]

Thank you Darshil, apologies for the delay in responding.

This is very helpful!

[Darshil_Shah1]

You’re welcome, @Guitarrista82! I am glad to be of help. 🙂

[Michael_Florin]

---

However, according to a Marketo support rep I'm communicating with, Marketo can only process two batch campaigns at a time and the rest are queued until the previous ones complete.

---

Just wanted to add that this number "two" your support rep is talking about depends on the number of batch campaign "workers" that are available to your Marketo instance. That number can be increased, if you ask for it. I worked in environments that had six concurrent batch workers.