SOLVED

Campaign Queue Backlog

Go to solution
Anonymous
Not applicable

Campaign Queue Backlog

We are experiencing a huge backlog of triggered campaigns in the Campaign Queue, which is causing delays to our data updates and programs that sync to Salesforce.

I initially engaged Marketo Support when I noticed the delays, and they told me that it was the sheer number of trigger campaigns we had active that was causing the backup. That combined with inefficient smart list operators (using "contains" or "is not", instead of "is") was severely slowing down Marketo's ability to process new trigger campaigns.

This article on optimizing smart lists was helpful in explaining the impact of those filters: https://community.marketo.com/MarketoArticle?id=kA050000000KzcsCAC

I was wondering if anyone has faced these challenges, and what were some steps that you took to alleviate the problem?
Tags (1)
1 ACCEPTED SOLUTION

Accepted Solutions
Justin_Cooperm2
Level 10

Re: Campaign Queue Backlog

Yes, it absolutely will negatively impact performance of other campaigns in your subscription if you have tons of unused active trigger campaigns. You should do routine cleanups and de-activate anything that is not necessary. I also encourage people who use trigger campaigns to see if they can execute the same operations with a recurring batch campaign. Trigger campaigns really should be reserved for operations that must occur in real time. In general, trigger campaigns are slower if you are attempting to execute an operation for 5,000+ leads.

We have two queues that are part of trigger campaign processing. The first evaluates whether an activity has caused an active campaign to be triggered. It then pushes that over to a second queue that is a queue of campaigns that have already been triggered that need to be processed. If you were to import 1M leads in a list import and also had an active campaign with “New lead was created” trigger, it would take some time (it will vary, hard to estimate and also involves whether the trigger campaign has additional filters as part of the smart list qualification) to evaluate and push these over into the second queue to await processing. The next queue contains the campaigns that have been triggered and need to be executed. This queue is a bit faster than the first as we have parallel processing in place (6 processors) to execute these campaigns. There is also prioritization in place for the second queue. There are three levels of priority and the highest level will be processed faster. The type of trigger doesn’t matter (assuming there aren’t additional filters specified in addition to the trigger, which can also slow things down). What matters is the flow step. Trigger campaign processing is prioritized as follows: Campaigns with “Send Email” or “Send Alert” flow step are highest priority, campaigns with “Request Campaign” flow step are second-highest, and campaigns with any other flow steps (including “change data value") are lowest priority. The amount of time a trigger campaign takes to execute depends on what is presently in the trigger campaign processing queue. If there are many active trigger campaigns and several campaigns are triggered at once, there can be some wait time, especially for low priority trigger campaigns. It is always best to avoid trigger campaigns if the campaign will trigger extremely often. Otherwise, if there are a bunch of active trigger campaigns then this second queue could grow to be quite large. Finally, the more active trigger campaigns you have, the longer it will take to determine if the campaign was trigger by an event that took place. Best practice is to clean up (de-activate) any old active trigger campaigns that aren't needed any longer.

View solution in original post

16 REPLIES 16
Josh_Hill13
Level 10 - Champion Alumni

Re: Campaign Queue Backlog

What is the count of triggers? Many SMB accounts seem to slow down around 100-150 triggered campaigns.

I would go through the list via Campaign Inspector and see if you can convert some to Batch campaigns, especially data management/appending/normalization campaigns. scoring demographics are sometimes better with batch, depending on your needs.
Anonymous
Not applicable

Re: Campaign Queue Backlog

Thanks Josh,

We have almost 1,800 active trigger campaigns in the Campaign Inspector! I have not paid much attention to this in the past, reasoning that it would not cost anything to keep campaigns active if nothing was actively triggering them. Do you know if it affects the sytem performance to have these campaigns active even if they are not running on a regular basis? 

I did convert many smart campaigns to batch campaigns and deactivated a bunch that were redundant - the campaign queue cleared up relatively quickly. Still, I would like to know if there is a limit to how many smart campaigns Marketo can process at once? I suppose it depends ont he conditions/filters involved, but is there a ballpark number I should be aware of?
Caryl_Mostacho
Level 7 - Champion Alumni

Re: Campaign Queue Backlog

Hi Robberto McDowell​ this is interesting.

I've also been told that having many "Campaign is Requested" triggers can slow down Marketo as well.

Justin_Cooperm2
Level 10

Re: Campaign Queue Backlog

Yes, it absolutely will negatively impact performance of other campaigns in your subscription if you have tons of unused active trigger campaigns. You should do routine cleanups and de-activate anything that is not necessary. I also encourage people who use trigger campaigns to see if they can execute the same operations with a recurring batch campaign. Trigger campaigns really should be reserved for operations that must occur in real time. In general, trigger campaigns are slower if you are attempting to execute an operation for 5,000+ leads.

We have two queues that are part of trigger campaign processing. The first evaluates whether an activity has caused an active campaign to be triggered. It then pushes that over to a second queue that is a queue of campaigns that have already been triggered that need to be processed. If you were to import 1M leads in a list import and also had an active campaign with “New lead was created” trigger, it would take some time (it will vary, hard to estimate and also involves whether the trigger campaign has additional filters as part of the smart list qualification) to evaluate and push these over into the second queue to await processing. The next queue contains the campaigns that have been triggered and need to be executed. This queue is a bit faster than the first as we have parallel processing in place (6 processors) to execute these campaigns. There is also prioritization in place for the second queue. There are three levels of priority and the highest level will be processed faster. The type of trigger doesn’t matter (assuming there aren’t additional filters specified in addition to the trigger, which can also slow things down). What matters is the flow step. Trigger campaign processing is prioritized as follows: Campaigns with “Send Email” or “Send Alert” flow step are highest priority, campaigns with “Request Campaign” flow step are second-highest, and campaigns with any other flow steps (including “change data value") are lowest priority. The amount of time a trigger campaign takes to execute depends on what is presently in the trigger campaign processing queue. If there are many active trigger campaigns and several campaigns are triggered at once, there can be some wait time, especially for low priority trigger campaigns. It is always best to avoid trigger campaigns if the campaign will trigger extremely often. Otherwise, if there are a bunch of active trigger campaigns then this second queue could grow to be quite large. Finally, the more active trigger campaigns you have, the longer it will take to determine if the campaign was trigger by an event that took place. Best practice is to clean up (de-activate) any old active trigger campaigns that aren't needed any longer.

Anonymous
Not applicable

Re: Campaign Queue Backlog

Thanks Justin Cooperman​!

That's by far the most information I've received on this topic. Very helpful.

Anonymous
Not applicable

Re: Campaign Queue Backlog

Justin Cooperman

Follow up question: are triggers in engagement program streams and segmentation filters processed the same way?

Justin_Cooperm2
Level 10

Re: Campaign Queue Backlog

Yes. Say I had a segmentation on "Job Title" and I just changed mine in the Lead DB. It does not mean that instantaneously, that would be reflected in the segmentation. The trigger would go through same evaluation and execution to update my segment. So, there would still be a period where I was in my old segment.

Kenny_Elkington
Marketo Employee

Re: Campaign Queue Backlog

In addition to these, Revenue Cycle Model transitions, and Auto-Assignment rules follow this model.

Christina_Phill
Level 1

Re: Campaign Queue Backlog

Hi all-

This exchange is super helpful! We have been seeing an increased slowdown as well and we run a ton of triggered campaigns. Does anyone know if those on the Enterprise level subscription experience the same problems? We are relatively good about cleaning up our triggered campaigns and removing old or obsolete ones. We are growing rapidly and will only see an increase in triggered campaigns going forward, so we are trying to determine if we need to jump to the Enterprise licensing level or would we experience the same thing? Would appreciate any insight you have.

Thanks,

Christina