01 More Dupes.jpg

How to Control Order of Operations in Marketo

Justin_Norris1
Level 10 - Champion Alumni
Level 10 - Champion Alumni

If you spend long enough building in Marketo, you will very likely encounter scenarios where things don’t happen they way you’d expect.

You may have two different smart campaigns - which are in themselves perfectly functional - produce a bad result because they didn’t execute in the right order.

I would say 90% of the issues I’ve had in Marketo have been some variant on this simple theme of order of operations.

Order of operations is often the difference between a stable, predictable, and effective Marketo instance and a disorderly chaotic mess. If you aren’t constantly thinking about and controlling the order that things happen, your Marketo systems will eventually break, no matter how much cool stuff you build.

Do you have a race condition?

(You may want to ask your doctor...)

A race condition is when a successful outcome of a process (e..g, a smart campaign) is dependent upon some other process being completed first, but those processes occur asynchronously and in an uncontrolled way.

When the processes don’t occur in the right order, then the dependant process fails.

Symptoms of a race condition include:

  • Leads being misrouted in your CRM because they were synced before key data values were written to the lead record.
  • Alerts being sent to no one because they fired before a sales owner was assigned.
  • Lead Lifecycle Stages getting overwritten because a lead qualified for multiple lifecycle campaigns at the same time.
  • People in your office talk about how “Marketo is broken”.
  • Your life as a Marketo Admin becomes like:

01 More Dupes.jpg

(This actually happened to me.)

Fortunately, race conditions are both preventable and treatable. Here’s how.


Tactics for Controlling Order of Operations

The list of tactics below isn’t definitive, but it covers the most obvious ways to control order of operations and the ones I’ve found to be most useful.

All these methods need to be used within a well-designed system in order to work well, but that’s a subject for another post.

1) Keep Your Actions in a Single Smart Campaign

The simplest way to control the order of a series of flow steps is to keep them all in a single smart campaign.

Pros

  • Simple and effective.
  • Order of operations for flow steps in a single smart campaign is basically guaranteed. A lead moves from one flow step to another in sequence.

Cons

  • This approach is suited only for very simple scenarios, with a few actions that always go together in a linear sequence. Once you introduce multiple conditions and more advanced logic, this approach breaks down.
  • Long lists of flow steps that do lots of different things are hard to understand.

When to Use

If you have a few simple activities you are trying to chain together in a straightforward linear flow, this method works great.

Example: Form Auto-Responder

When a lead fills out a form, you want to add the lead to a Salesforce campaign, send the lead a thank-you email, and then send an alert to distribution list. All these steps happen in quick sequence and have no other dependencies outside the campaign. So based on these requirements, a single smart campaign works very well.

Eg 1 - Single Campaign - Smart List.jpg

Eg 1 - Single Campaign - Flow.jpg

2) Wait Step

A wait step allows you to pause your lead in a flow for a designated period of time before doing something else. While your flow is waiting, it gives a chance for other stuff to happen elsewhere in your system.

Wait steps are the most commonly used (and misused) method for controlling order of operations that I’ve seen.

Unfortunately, wait steps are still basically just guesses. A wait step waits however long you tell it to, and has no awareness of what you might be waiting for. Systems are unpredictable - depending on conditions, your wait step might be too long or too short.

Pros

  • Wait steps are simple and easy to understand.
  • Wait steps are precise when all you want is to pause a process for a specific period of time, irrespective of what else might be happening in the system.

Cons

  • If your other process is not specifically time-bound, there is no guarantee that the wait step will be long enough. It’s better than nothing, but definitely not bulletproof.
  • If a wait step proves too short, people tend to make them longer and longer to provide more security. Unfortunately, this means that your process gets slowed down to the lowest common denominator and you lose velocity the rest of the time.
  • Campaigns with wait steps are deprioritized in the campaign queue.

When to Use

Wait steps are generally not a robust method for coordinating asynchronous processes with no clearly defined time interval.

This doesn’t mean never use them in these scenarios (sometimes the complexity of doing otherwise is impractical). However I would make them my last resort.

To my mind, there is really just one scenario where a wait step is always a suitable option: when you want your process to wait for a defined period of time (no matter what else is happening).

Example: Free Trial Emails

A lead initiates a free trial of a product. Your campaign sends a welcome email, then waits 30 days while the lead tries out the product. At the end of the 30 day period, the flow checks if the lead has upgraded their trial, and if not, sends an upgrade offer.

In this case, a wait step works just fine because we know the trial is exactly 30 days and we are checking their current state after 30 days before sending the next message.

Eg 2 - Wait Step - Smart List.jpg

Eg 2 - Wait Step - Flow.jpg

3) Request Campaign

The Request Campaign flow step (and its companion, the Campaign Is Requested trigger) are oddly polarizing in our community. I find this feature inspires either passionate enthusiasm or antagonism among Marketo users, depending on who you ask.

I’m a fan of Request Campaign. I find it the most direct and logical way of having one campaign trigger another. It provides enormous flexibility in how you structure your campaigns and makes it very easy to route people through operational programs multiple times.

On the flip side, you have many people state that this trigger can slow down your Marketo instance.

Now, I haven’t seen any hard and fast evidence or official statement that the Campaign Is Requested trigger requires more processing time than any other trigger on a one-for-one basis. To be honest, I suspect that part of the issue is that Request Campaign can lead to more “trigger-happy” design patterns, causing an overall increase in the volume of trigger campaigns. And that definitely can decrease overall performance, regardless of the trigger you use.

However, even assuming it is a bit more resource intensive, it doesn’t mean we can’t use this trigger; in some scenarios, it is still the most efficient tool for the job considering all the factors involved.

The main thing is to use it judiciously, which is a rule to apply to all trigger campaigns in general.

Pros

  • Request campaign is a very logical and bulletproof way to ensure one campaign triggers another.
  • Flexible and extensible for a wide variety of situations.
  • Quick and easy to deploy for ad-hoc use.
  • Allows for centralization of repeatedly used functions, which is easier to maintain.
  • No limit on how often a campaign can be requested for any one lead.

Cons

  • Potential performance impact at higher scale.
  • Some people find it harder to work with and troubleshoot.

When to Use

I use Request Campaign in scenarios where I need one campaign to immediately trigger another and other methods are too clunky. I think it’s especially suitable where you have a central operational campaign that you need to repeatedly trigger from a wide variety of places -- i.e. a high ratio of flow steps requesting a single trigger.

Example: Centralized MQL Processing

You have a series of multiple flow actions that need to happen when someone fills out a fast track form, all of which terminate in a sync to Salesforce. You need to ensure that the Salesforce sync happens only once all the previous flow actions are complete.

Furthermore, your demand gen team is super form-happy, and you have over 100 unique form placements to deal with.

You could recreate those flow steps 100 times in each form program followed by 100 “Sync to SFDC” flow steps, but this is brutal to maintain if anything changes and makes it impossible to coordinate your sync with other processes.

Instead, create a single MQL processing campaign in your lifecycle program that includes a sync flow step. Now you can now request it from anywhere you want, maintain one version of the process, and control when and how someone enters that flow.

This uses only one Campaign Is Requested trigger and ensures that your process works even if a lead fills out the same form multiple times.

Eg 3 - Request Campaign - Smart List.jpg

Eg 3 - Request Campaign - Flow.jpg

4) Static Lists

Static lists can be used in almost the exact same way as Request Campaign. Instead of a Request Campaign flow step, use an Add to List flow step. Instead of a Campaign Is Requested Trigger, use the Added to List trigger.

There are some benefits to doing it this way. Static lists can be deployed quickly and in an ad-hoc way and disposed of just as easily. Lists also provide some additional insight that can be used in your operational reporting, since the list keeps a running count of people who have passed through the process.

The main drawback is you can only be added to a static list once. If you use the fast-track form example above, what happens the second or third time someone fills out the same fast track form?

To make sure your process doesn’t break, you would need to ensure you have additional automation to remove the lead from the static list when the process is complete. This is extra overhead and it creates the possibility for error (plus sets you up for more potential race conditions).

Pros

  • Allows one campaign to trigger another another.
  • Flexible for a wide variety of situations and very extensible (no functional limit to the number of static lists you can create).
  • Quick and easy to deploy for ad-hoc use.
  • Allows for centralization of repeatedly used functions, which is easier to maintain.

Cons

  • Managing 5 bazillion static lists.
  • For repeated processes, additional automation is required to remove leads from the static list after a process is complete. This creates the potential for failure if a lead isn’t removed correctly or not removed quickly enough.

When to Use

Static lists are a very sensible choice in most scenarios you might encounter, as long as there is no concern around being able to successfully remove the lead from the list in time.

I find static lists especially valuable when you need to identify a subset of leads that are waiting for a particular process to end, usually triggered by a data value change. The static list membership becomes a filter on a campaign triggered by that data value change, allowing you to control when those leads move through the flow.

Example: Send Alert After Owner Is Assigned

When someone fills out a fast-track form, you want to send an alert to the owner if the record has an owner in your CRM (we’ll use Salesforce for this example). However, at the moment the lead fills out your form, they may OR may not already have an owner assigned.

If they do have an owner assigned, your campaign should send the alert to that person. If the owner is not yet assigned, you need to wait until the owner is assigned (an outcome controlled by another process in another system) and then send the alert.

Campaign #1

The first campaign checks if the lead has an owner; if it does, the alert is sent, and if not the person is added to a static list to identify them as someone waiting for an owner.

Eg 4a - Static List - Smart List.jpg

Eg 4a - Static List - Flow.jpg

Campaign #2

The second campaign listens for lead owners changing BUT uses the static list membership as a filter. The list allows you to precisely identify the leads you want to take action on once their owner changes.

Eg 4b - Static List - Smart List.jpg

Eg 4b - Static List - Flow.jpg

5) Program Statuses

Program statuses are another very flexible method for controlling order of operations. The flow step is Change Program Status and the corresponding trigger is Program Status Is Changed. You can also use the “Member of Program” filter with specified program statuses.

Program statuses can be especially elegant when you are trying to coordinate a multi-stage operational process, since the various stages of your process can map against the statuses of a program channel. (My colleague Kristen did a Summit session all about this method, with some useful examples.)

For operational channels, make sure to exclude them from reporting and keep all your program statuses at the same number. That way, when a process is complete, you can easily move someone back to the first status if you want them to move through the process multiple times.

Pros

  • Allows one campaign to trigger another another.
  • Flexible for a wide variety of situations.
  • Allows for centralization of repeatedly used functions, which is easier to maintain.
  • Can be used repeatedly for the same lead if set up correctly.
  • Provides a quick “at-a-glance” view of how many leads sit where in your process
  • Program status changes are logged permanently in the activity log and do not get archived.

Cons

  • Less quick and easy to deploy - requires admin access to create new channels.
  • Potentially less extensible than other options.
  • Difficult to modify program statuses once existing programs are using them - plan your processes carefully.

When to Use

Program statuses would be my preferred option for operational programs where you have a clear multi-stage process that forms the foundation of your system. It is also an excellent choice for marketing activities that occur consistently in the same way.

For one-off or ad-hoc usage, a static list is a better choice as you don’t need to commit to creating a new channel in the admin.

Example: Lead Scoring Threshold

You have a single operational program set up to score your leads. Your scoring program has a defined score threshold. Leads reaching the threshold should be transitioned to MQL.

Additionally, your program needs to allow leads to be recycled, at which point there is a holding period where they are not eligible to reach MQL status again. After the holding period, their behavior score is reset to zero and they can reach MQL again.

Program Channel

Eg 5 - Program Statuses - Channel.jpg

Program

The scoring flows would go in their appropriate folders, while the two campaigns below live in a “Management” folder.

Eg 5 - Program Statuses - Program.jpg

Campaign #1

When the lead reaches your scoring threshold, change their program status to “Reached Threshold”. This should serve to activate a trigger in a central lead lifecycle processing campaign (like the MQL flow above), which will handle everything associated with routing the lead to MQL.

Additionally the other program statuses serve as a filter to ensure that leads who have already reached a threshold or are in purgatory after being recycled do not get re-processed.

Eg 5a - Program Statuses - Smart List.jpg

Eg 5a - Program Statuses - Flow.jpg

Campaign #2

The second campaign triggers once the lifecycle stage changes to recycled. The lead is moved into a waiting program status for the appropriate period (30 days in this example). After the wait, the scores are reset and the program status changes again.

Eg 5b - Program Statuses - Smart List.jpg

Eg 5b - Program Statuses - Flow.jpg

In this case, the program statuses serve to trigger a process in another program and control the flow of leads within this program. Another benefit of this structure is that you can see at a glance from the program summary view how many leads you have in each stage.

Eg 5c - Program Statuses - Program Summary.jpg

6) Data Value Changes

If you are orchestrating processes that involve updates to lead data, data value changes are a very natural way to connect them.

There are many possible use cases here; the send-alert-after-owner-change example above is also a good example of a process triggered by a data value change, since the Salesforce process of assigning an owner triggers the alert in Marketo when the owner value updates.

There are a few restrictions here - most obviously, a field must exist in your schema in order to trigger off a change to it. For ad hoc use where a field isn’t already involved, it would be better to use another method, such as adding someone to a static list.

Pros

  • Allows one campaign to trigger another another.
  • Allows you to leverage existing data updates to control order of operations without any additional build or campaign overhead.

Cons

  • Requires a data field to exist in your schema.
  • For repeated processes, additional automation is required to reset a data value after a process is complete. This creates the potential for failure if the field isn’t updated correctly or quickly enough.
  • Data value changes are archived after 90 days, making this more difficult to audit over time.

When to Use

If a process is already updating a data value, and this update indicates that other processes can start, then using this as a trigger is a smart idea. Lifecycle stage updates, score changes, a lead source being set, and so on.

Example: Data Enrichment for Lead Routing

You and your SFDC admin want to team up to create some automated lead assignment rules. To route leads properly, you need some key data values to be populated - and you don’t want to kill your form conversion by adding a bunch of new fields.

Enter a new data enrichment vendor, who provides you with a webhook that you call in Marketo when the lead is created to enrich them with data like company size, headquarters location, and industry.

Your SFDC Admin sets up assignment rules so that when the leads sync to Salesforce they will get sent automatically to the right reps based on the enriched data. Huzzah!

However, at first you run into some problems. Both your data enrichment webhook and your lifecycle flow for new leads trigger off of “lead is created.” You need to find a way to make sure that the leads are not sent to Salesforce until the data enrichment webhook has finished pushing in data. This can happen almost instantly or take 1-2 minutes depending on how the enrichment vendor’s API is behaving.

The answer is to chain these actions together using a data value change.

Campaign #1

Your data enrichment process actually becomes the first step in your lifecycle processing, because it needs to be complete before you take any other action. So when the lead is created, you call your webhook.

Eg 6a - Data Value - Smart List.jpg

Eg 6a - Data Value - Flow.jpg

Campaign #2

In the webhook response containing the enriched data, your vendor will also pass back a status code. You can map this value into a field and use it to trigger the next step of your process.

Eg 6b - Data Value - Smart List.jpg

Eg 6b - Data Value - Flow.jpg

Of course this is an oversimplified example and in real life there are other things to consider. For example, if your enrichment webhook fails for any reason, you need a backup to ensure the lead is still processed. But you can see how this method is a lot more bulletproof than a wait step. You are essentially guaranteeing that your sync is not going to happen until the enrichment is complete.

Wrapping Up

Obviously there are about 5 different ways of executing any of the examples I mentioned above. Those ways aren’t all created equal, but there’s room for reasonable people to prefer different methods for various reasons. That level of flexibility and freedom is part of what makes Marketo great.

I’m really interested in what other people are doing in the trenches day to day, so please share your agreements, your disagreements, and your favorite methods I didn’t mention below.

15768
26
26 Comments
Jessica_Cross1
Level 6

The "alerts being sent to no one because they fired before a sales owner was assigned" is such a good example. Learned early on that after a new lead is created, gotta add in at least a 5 minute wait step for the round trip to the CRM to get an owner and come back again before triggering the send sales alert email.

SanfordWhiteman
Level 10 - Community Moderator

This is a superb piece, Justin.  I especially like how you were able to define a race condition in terms of widely understood Marketo features.  I literally have two draft posts titled "Your code has a race condition" where I'm trying to address the problem for Forms API JavaScript users (who are likely not professional programmers) and it's much harder to explain that way.  Kudos!

Anonymous
Not applicable

This is a beast of a blog with an incredible amount of knowledge shared. I've bookmarked to come back to this as a refresher/pulse check since we're new to Marketo. But this definitely has a few ahas for me already. THANK YOU for posting, very much appreciated!

Justin_Norris1
Level 10 - Champion Alumni

Jessica Cross​ Yes this seems like a really common issue. It is scary to think of how much lost revenue might be attributed to alerts being sent into outer space.

SydneyMulligan
Level 10 - Champion Alumni

"Leads being misrouted in your CRM because they were synced before key data values were written to the lead record."

I had this problem at my last company and it was A NIGHTMARE. This caused lots of people in our organization to say "Marketo is broken" which is always what your Marketo admins want to hear

This is an excellent post!

Justin_Norris1
Level 10 - Champion Alumni

Thank you, Sanford Whiteman​. I guess it's the one time I had an advantage from not being a developer

That being said, I have found that trying to cultivate a "developer mindset" (something helped by the advice you share regularly) has made a big difference in understanding things like race conditions and how systems should work together 

Justin_Norris1
Level 10 - Champion Alumni

Thanks for the kind words Christina Wells​! Hopefully the 5 years I've spent banging my head against a wall on these types of issues can now save you some headaches along the way .

Justin_Norris1
Level 10 - Champion Alumni

Sydney Mulligan​ Thanks! Yes I think the lead routing issue is a really common one.

It does sting when people say "Marketo is broken". It's quite unfair to Marketo, since it's just a machine that follows the instructions we give it.

I think the root problem is that many admins - especially with their first Marketo instance as they are learning the ropes - tend to build their system organically. (I definitely did.) So different pieces of functionality pile up, and they don't necessarily cohere into a well-structured whole. At a certain point I think people are better off to blow it up and start again, which is what I did about a year in to my first instance. This was a great learning experience. 

Overall, it speaks to the need to plan out your Marketo implementation as carefully as you would plan the development of a piece of software or any other important business system. 

Guy_Goldstein6
Level 5

That's got to be about 5 years of awesome problem solving wrapped up in a really well presented post.

Hats off to you mate

Justin_Norris1
Level 10 - Champion Alumni

Thanks Guy Goldstein​!