As your use of Marketo matures, so do their customers become more skilled in the use of the platforms. And that means some customers push the limits of what the system is designed to do. Whether it is running dozens of data correction flows, scoring, and lead lifecycle or accumulating workflow detritus, an older system is often a slower system. Remember the Windows entropy problem? It's similar.
To keep your leads flowing fast, there are some tricks and best practices for running a system efficiently and effectively. These tips are helpful for new system builds (do it right the first time) as well as rebuilds or cleanups. These tips are considerations and each situation may require different designs.
Why do systems like Marketo slow down?
Like any computer program, Marketo is limited by hardware as well as software. Marketo is designed to handle marketing focused activities like scores, syncs, and email sends. Marketing tech people like me have taken advantage of other functions like data normalization, personalization, and lead lifecycles to perform tasks that the system can do, but maybe wasn't expecting to be used heavily.
There are three key components that can slow down an instance:
- Number of trigger campaigns
- Volume of Leads
- Complexity of smart lists
There is not a hard and fast rule for the number of campaigns or leads that Marketo can handle well. Based on my experience, slow downs can occur with as few as 150 triggers and 100,000 leads. I've seen systems with 1,400 triggers and heard of systems with 3,500! With regular cleaning and good systems design, it is fairly easy to keep your system running fast.
Increasing speed, reducing load
Trigger campaigns are always on, always listening. That means for every lead that changes, the system must check if it meets those conditions. If you import a lot of leads, or change a lot at once, the system will have to check all of those at once. According to Marketo, each lead is set serially into the flow, further slowing down the system.
Reduce the number of Triggers!
- Convert triggers to batches.
Does every score have to happen immediately? Probably not. Behavior based triggers are prone to frequent hits and complexity, so move these to a nightly batch. Data normalization can also help. Batches also run all the flow steps for every lead at once, instead of serially, which reduces total processing time.
The real reason triggers cause problems is there are two queues. Whenever a change to a lead happens, it hits every potential Trigger in the backend - the Trigger Evaluation Queue. You cannot see this and Marketo Support won't show most of you what is going on there.
The key thing to know is the formula: Change x # of Leads.
100,000 Leads Imported x 20 Lead is Created Triggers = 2,000,000 evaluations
Yes. It gets big, fast. The fewer triggers you hit, the faster Marketo can send the lead to the correct Trigger that's visible in the Campaign Queue. Some big offenders could be your SalesOps team or Product team making batch changes on their end, only to let 1,000,000 records sync to Marketo to hit that Evaluation Queue. This Queue will slow down your processing and routing inside Marketo because it takes longer for the lead to hit the correct set of Triggers and then be processed as you expect.
- Reduce the Qualifications Rules from Every Time to something else.
- Reduce Triggers like Change Data Value and Lead is Created to single points of entry or batches whenever possible.
- Reduce the number of leads that can flow through with filters
- Leverage the auto-deactivation after 6 months.
- Clean up triggers quarterly with the Campaign Inspector.
- Compress flows by adding flow steps to one campaign based on the same trigger. Eg: Interesting Moments and Scoring can often be on the same flow.
Smart Lists Should be Less Complex on Triggers
Depending on the situation, you may want to reduce the complexity on batches and triggers. Instead of Marketo looking for the list and running it, just put the filters in the trigger itself. Or reduce the number of nested smart lists called in a smart list. Whenever you ask Marketo to call another Smart List, it has to wait until all of the other smart lists finish, before putting together the final counts.
The more complex a smart list, the harder it is for the system to figure it out, which increases backend processing (which you do not see on the Campaign Queue) and even creates campaign failures from timeouts.
Change the Time of Day of Batches
While this may vary with your business and location, batch data flows should occur at night, or the time of day where leads aren't on your site frequently. This helps give priority to email sends that may start around dawn or through the day, as well as triggers that route leads to sales on weekdays.
Use Wait Steps in Processing
There is a much longer discussion on this regarding Lead Lifecycles and Ed Unthank's lead processing articles. Many times, some campaigns are subject to a race condition between themselves or the CRM. To mitigate this, where the lead should have been updated, but wasn't in time for the next campaign to process properly, you can do three things:
- Add Wait Steps at the start of the flow to let the other flows catch up. Sometimes 5 to 30 minutes are needed.
- Add Wait Step after a Sync to CRM step to let the CRM assign a Lead Owner. May require 10 - 30 minutes if the system is complex.
- Request Campaigns to control the order of operations.
Now remember that Wait Steps will also slow down the overall time it takes for a lead to finish processing. Sometimes this is good if you think Marketo isn't fully registering the Data Change before the next step. Other times, it may cause the overall time to take much longer.
With Project Orion on the horizon, we may all see our processing times reduce, especially with behavioral data. Just because it goes faster, doesn't mean you should abandon good workflow design.
.png)