We have built a system of lead entry and scoring campaigns, which heavily relies on "Request Campaign" flows. Under heavy traffic our campaign queue gets pretty crowded and processing tends to slow down which causes various headaches.
Now I have heard that "Request Campaign" is actually not an ideal way to connect two Smart Campaigns. Records would flow faster, if for instance "Add to list" and "Was added to list" was used as the connector.
Can anybody confirm that is true?
In high traffic cases, yes this is typically true: Marketo basically has an internal prioritisation system that decides which types of processes are run at a higher vs lower priority - and request campaign flow steps are known for falling in the lower priority bucket. In low traffic cases it's not noticeable, but in high traffic cases you'll definitely see that "add to list" alternative processes will run faster.
From Scott Nash Under the Hood II session:
for reference and guidance
Per this comparison, add to list is quicker.
Thank you, Grace & Grant!
Yes, I know that trigger campaigns run with different priorities. But I'm wondering if that is actually relevant for my case. If we talk about trigger priorities, we're talking about what runs earlier and what runs later, right? But I need to process stuff faster - not relative to other campaigns, but absolute.
So if I have 10 campaigns that request each other in a chain, and 50.000 new records hit that chain at the same moment - which unfortunately is our reality - and the processing of the whole chain takes 4 hours, can I somehow cut that time down by not using "Request Campaign" and "Campaign is Requested" anymore?
Well, Grant, if Add to List is four times as fast as Request Campaign, this might actually do something. I'll test it.
So this is a challenge. I have been dealing with this for years and you really have to look at your architecture and ask yourself some questions:
Also, you can trick Marketo by placing an Empty Send Email flow step at the TOP of any Trigger campaign to instantly push it to the TOP of the queue.
Any Wait Step more than 4:59 mins will instantly DROP your trigger or batch down the queue.
I read about daisy chains on etumos.com and in fact our own lead entry flow looks very much like the Lifecycle Processing programs here. Yes, "Person is created" is only used as a trigger in the first campaign. And yes, all our Engagement Program transitions are batches.
You're saying all scoring flows should be batches. Some of our scoring flows are batches, but most of them are triggers. With the idea that the MQL - or lead that passed the scoring threshold - should be transported to sales as quick as possible. That needs to be trigger-based concept, doesn't it?
Speed to MQL is overrated. The only MQL Sales wants is the Contact Me/Call Me one. Or a direct dial Your flow should auto MQL that one.
Yes, I get that if someone is live on the site and triggers an MQL (Visits Page or something), it'd be good to let a rep know. There are a couple of thoughts I have about that:
When the system is backed up, what do you see when you go to the Campaign Queue?
You should be able to see the campaign priorities as High, Medium, Low. This might help determine where the blockage is occurring:
To weight something as 'high priority' that isn't normally, you can add a blank 'Send Alert' step as the first item in the campaign OR try the campaign priority override within Marketo Sky (I'm not using Sky at the moment so can't comment on how well this works)
Campaign Queue Priorities
Overriding Priority in Marketo Sky
Do your 10 campaigns have any steps that are calling Webhooks?
In my experience, these are rather slow and bulk data updates are best done via the API
It's hard to troubleshoot without knowing your database size and what the campaigns are doing, but hopefully this gets you started.
Yes, out of our 18 - 10 was just a number - lead entry Smart Campaign, 5 contain webhooks. De-duping, creating a hash value, and calling DemandBase and two more lead augmentation services. That's probably a little too much, and I'm thinking about how to peel complexity off of it. Something like: If the DemandBase or DiscoverOrg were able to fill all blanks in the data, don't call anything else anymore. But honestly I don't have a good idea as to how to formulate this scenario in a flow step choice.