Request Campaign vs. other triggers

[Michael_Florin]

Hello!

We have built a system of lead entry and scoring campaigns, which heavily relies on "Request Campaign" flows. Under heavy traffic our campaign queue gets pretty crowded and processing tends to slow down which causes various headaches.

Now I have heard that "Request Campaign" is actually not an ideal way to connect two Smart Campaigns. Records would flow faster, if for instance "Add to list" and "Was added to list" was used as the connector.

Can anybody confirm that is true?

Thank you!

Michael

[Grace_Brebner3]

Hey Michael,

In high traffic cases, yes this is typically true: Marketo basically has an internal prioritisation system that decides which types of processes are run at a higher vs lower priority - and request campaign flow steps are known for falling in the lower priority bucket. In low traffic cases it's not noticeable, but in high traffic cases you'll definitely see that "add to list" alternative processes will run faster.

[J_Grant_Gray]

From Scott Nash‌ Under the Hood II session:

https://nation.marketo.com/thread/45832-under-the-hood-ii-batch-campaigns-recording

for reference and guidance

Per this comparison, add to list is quicker.

[Michael_Florin]

Thank you, Grace & Grant!

Yes, I know that trigger campaigns run with different priorities. But I'm wondering if that is actually relevant for my case. If we talk about trigger priorities, we're talking about what runs earlier and what runs later, right? But I need to process stuff faster - not relative to other campaigns, but absolute.

So if I have 10 campaigns that request each other in a chain, and 50.000 new records hit that chain at the same moment - which unfortunately is our reality - and the processing of the whole chain takes 4 hours, can I somehow cut that time down by not using "Request Campaign" and "Campaign is Requested" anymore?

Well, Grant, if Add to List is four times as fast as Request Campaign, this might actually do something. I'll test it.

Thanks again!

Michael

[Josh_Hill13]

So this is a challenge. I have been dealing with this for years and you really have to look at your architecture and ask yourself some questions:

1. Does this really need to be a trigger?
   1. For Scoring flows, I'd say 100% of campaigns should be batches. Seriously. 
   2. Contact Me and Freemium Trial flows should be Triggers/Fast Track that are part of your Lead Lifecycle system.
2. When you say Lead Entry, I take that as some sort of Lead Lifecycle management initial processor.
   1. Ed Unthank has thoughts about daisy chaining these with Request Campaigns. Check his site at etumos.com or my articles on marketingrockstarguides.com.
   2. Think hard about what really needs to happen RIGHT NOW vs. LATER. I'd say you can have a fairly efficient LLC with Request flows that do hand offs for various reason. In fact, I run a fair amount of Request flows but they branch off from 3 centralized Flows that are the only CORE entry points for most of the records.
   3. Do not use Person Is Created except in your FIRST entry trigger. Otherwise you will overwhelm your trigger eval queue. 50000 records x Number of Person Is Created Triggers = X (50000 X 10 = 500,000) which will take forever and BLOCK everything. Same is true for ANY other trigger like List is Imported or Data Value Changes. Be real careful how and when you use these.
   4. A LOT of things can simply be batched, especially Engagement Transitions (read my slides on this). Really zero reason to ever use a trigger for Engagements.
3. Put this all on paper/whiteboard/lucidchart and consider when things actually need to occur to drive business,

[Josh_Hill13]

Also, you can trick Marketo by placing an Empty Send Email flow step at the TOP of any Trigger campaign to instantly push it to the TOP of the queue.

Any Wait Step more than 4:59 mins will instantly DROP your trigger or batch down the queue. 

[Michael_Florin]

Thanks Josh!

I read about daisy chains on etumos.com and in fact our own lead entry flow looks very much like the Lifecycle Processing programs here. Yes, "Person is created" is only used as a trigger in the first campaign. And yes, all our Engagement Program transitions are batches.

You're saying all scoring flows should be batches. Some of our scoring flows are batches, but most of them are triggers. With the idea that the MQL - or lead that passed the scoring threshold - should be transported to sales as quick as possible. That needs to be trigger-based concept, doesn't it?

[Josh_Hill13]

Speed to MQL is overrated. The only MQL Sales wants is the Contact Me/Call Me one. Or a direct dial  Your flow should auto MQL that one.

Yes, I get that if someone is live on the site and triggers an MQL (Visits Page or something), it'd be good to let a rep know. There are a couple of thoughts I have about that:

1. if they didn't ask for a call, why call right this moment?
2. if you do call, have a script better than "I saw you were on our site..." And even then it's creepy to call in the middle of browsing. People have done that to me and it's weird. Don't be that guy.
3. If the MQL is triggered overnight, the salesperson has a list to call the next day and may be better received by the lead.
4. If your sales funnel is more like ecommerce, high speed, then a rep is probably not involved until after the CC swipe...but that is a good reason to use triggers for scoring.

[Mark_Price]

When the system is backed up, what do you see when you go to the Campaign Queue? 

You should be able to see the campaign priorities as High, Medium, Low.  This might help determine where the blockage is occurring: 

To weight something as 'high priority' that isn't normally, you can add a blank 'Send Alert' step as the first item in the campaign OR try the campaign priority override within Marketo Sky (I'm not using Sky at the moment so can't comment on how well this works)   

Campaign Queue Priorities
https://nation.marketo.com/docs/DOC-1137-how-campaign-processing-works

Overriding Priority in Marketo Sky
https://help.marketo.com/hc/en-us/articles/360010183814-Priority-Override-for-Trigger-Campaigns

Do your 10 campaigns have any steps that are calling Webhooks? 

In my experience, these are rather slow and bulk data updates are best done via the API

It's hard to troubleshoot without knowing your database size and what the campaigns are doing, but hopefully this gets you started. 

[Michael_Florin]

Thanks Mark!

Yes, out of our 18 - 10 was just a number - lead entry Smart Campaign, 5 contain webhooks. De-duping, creating a hash value, and calling DemandBase and two more lead augmentation services. That's probably a little too much, and I'm thinking about how to peel complexity off of it. Something like: If the DemandBase or DiscoverOrg were able to fill all blanks in the data, don't call anything else anymore. But honestly I don't have a good idea as to how to formulate this scenario in a flow step choice.