In Marketo, there are often multiple ways of designing the same program using one or more smart campaigns, particularly when it comes to nurture. Not all ways of doing things are equal from a performance/efficiency standpoint. I recently had the opportunity to do some testing to confirm which of the variations would be the most efficient and which tradeoffs are worth making.
Note that throughout all of these tests, smart list optimization techniques were applied as described in this earlier blog post.
These tests were run with batches of 20,000 leads to compare overall and average processing time. They were done with a specific campaign objective in mind and, as such, may not be applicable in all situations.
The basic premise of this campaign was that an Interest field was being populated by various actions in the Marketo database. Based on this Interest field, we wanted to check against a list of qualification criteria to update another field, indicating which nurture program and which stream the person should be in. We then wanted to move them into the appropriate stream of the appropriate nurture.
Question 1: Are two triggers triggering off the same action better than one trigger with choices in the flow step?
Even this morning I saw someone suggest on the discussion forums that someone consolidate two identical triggers into one single campaign. I think this is a pretty common viewpoint among Marketo users, even myself. But is it always the best choice? In this particular example, I had several filters on the smart list in addition to the triggers. I ran a test of this variation against a variation in which I moved the qualification criteria into the flow step and combined the two campaigns into one campaign, but using choices and referencing smart lists containing the filters.
Variation A (criteria in SL)
- Average run time of campaign A: .1557 sec per lead
- Average run time of campaign B: .2115 sec per lead
- Summed total processing time for 20,000 records: 7,500 sec
Variation B (criteria in flow with choices)
- Average run time of combined campaign: .3539 sec per lead
- Summed total processing time for 20,000 records: 7,077 sec
So, as you can see, the difference is relatively minimal. However, since in Variation A, I can be running two campaigns simultaneously, the total time to complete could be slightly lower. As you add additional flow steps with more choices, however, the performance of Variation A pulls away from the performance of Variation B.
Question 2: If I have shared criteria across multiple campaigns and want to move them into a smart list to make it easier to manage, how much efficiency gain do I lose?
Often times when we're constantly repeating the same criteria over and over again, it can be tempting to pull that criteria into a central smart list to make it easier if we have to update it. But as most Marketo users know, Member of Smart List references do add a little more performance overhead. Is it still worth it? How much performance do we lose? I took the criteria from variation A and moved all of the shared components into a central smart list to test this out. (Note by this point I was doing tests on batches, not triggers, so my baseline comparison will be different from Question 1).
Variation A (criteria in smart campaign)
- Average run time of campaign A: .2928 sec per lead
- Average run time of campaign B: .1230 sec per lead
- Summed total processing time for 20,000 records: 2,394 sec
Variation B (criteria in separate smart list)
- Average run time of campaign A: .1122 sec per lead
- Average run time of campaign B: .1385 sec per lead
- Summed total processing time for 20,000 records: 2,471 sec
As you can see, the additional cost for moving the smart list criteria out into a central smart list was relatively small, so this may be worth doing in cases where you have complex criteria being used over and over and you want to maintain central management.
I'll share some more tests in a few days when I have a little more time to write them up.
.png)