Marketo / Salesforce Sync Limits & Priority Info

[Anonymous]

Hi Everyone,

    Since the beginning of the year, I’ve been working with Marketo support on questions around the limitations and priorities between Marketo and Salesforce.com syncing. I have received some answers from Marketo Support that I have not been able to find anywhere else so I figured I’d share them here to spread the knowledge.

At the beginning of the year, we went through a rather large data migration. We knew from past experience that making updates to large amounts of data would cause a rather large sync backlog (one was as large as a several-day backlog). These backlogs would also result a major impact to our daily operations.

Even though we knew these backlogs would occur, we were never really sure how much data would actually cause the problem to happen. This time, we were hoping to get ahead of it and try to calculate the impact on our sync cycle based on the amount of data we needed to migrate.

Working with Marketo support, below is what we found to help us calculate how much data we could migrate at a time, without causing an impact to our operations:

• The sync cycle between Marketo and Salesforce.com will run approximately 10,000 record updates per hour, per object. Please note that some object types have different batch sizes. Only one object can sync at a time and Marketo handle them in the order of the schema / object list shown below.
  • For example: if you have 20,000 lead updates to pull and 20,000 contact updates to pull, Marketo will handle 10,000 lead updates, then 10,000 contact updates, then return to another 10,000 lead updates, finishing with the final 10,000 contact updates.
    • NOTE: This example does not consider any other objects being updated.
    • One caveat with Contacts is that updates will only occur as long as there is no backlog on the Account object.
  • The native sync between Marketo and Salesforce identifies records that need to be evaluated for an update based on the Last Modified Timestamp (also often called the system mod stamp). This timestamp is updated any time there is a change written to any field on the record (Note that formula and lookup fields do not update this timestamp). Whether one field is updated, five fields or a field that is not visible to the sync user, this update will queue for one evaluation for this record.
  • The creation of a new lead record (or other object) is an insert action. The initial sync from Marketo to Salesforce, must be through a Sync to SFDC flow step. The insertion process is a direct API call per record to Salesforce and can take 2-5 seconds to complete per record.
    • NOTE: the 2-5 seconds is just the timing for the sync process to create the lead and update the fields from the initial list. Our system also has several operational programs that append data and assign leads to sales territories. This information also needs to be considered as there can be additional updates queued after the initial insertion.
    • So, if you upload the list in Marketo, you will need to consider: 1) the insert action, 2) the number of fields in the list, 3) any Marketo operational smart campaigns that append data, and 4) any Salesforce logic that appends data and assigns leads to Sales. This Salesforce data still needs to sync back to Marketo after it is appended to the record. So this data should also be considered.
    • If the size of your data is large enough to trigger a sync backlog, the sync will continue to operate in a First in First out order.
      
      
• Forcing a sync using Marketo’s “Sync Lead to SFDC” flow step will always take priority over the standard sync cycle, and will push the update through even if there is a current sync backlog. However, make sure to use caution as the Salesforce flow steps can use anywhere between one and five API calls. Large pushes through these flow steps will use your API quickly.
  
  
• Marketo syncs schemas and objects in a very specific order. So, if you are making updates to multiple types of objects at the same time, the sync will update one object type at a time.  Below is the order that Marketo prioritizes these objects
  • Lead: synch schema
  • Account: synch schema
  • Contact: synch schema
  • User: synch schema
  • Task: synch schema
  • Opportunity: synch schema
  • Opportunity Contact Role: synch schema
  • Campaign: synch schema
  • Campaign Member: synch schema
  • Campaign Member Status: synch schema
  • Lead: pull conversions
  • Account: pull deletes
  • Account: pull updates
  • User: pull updates
  • Lead Queue: pull updates
  • Lead Status: pull updates
  • Account Type: pull updates
  • Lead: pull deletes
  • Contact: pull deletes
  • Lead: pull updates
  • Contact: pull updates
  • Lead: push updates
  • Contact: push updates
  • Campaign: pull deletes
  • Campaign Member: pull deletes
  • Campaign Member Status: pull deletes
  • Campaign: pull updates
  • Campaign Member: pull updates
  • Campaign Member Status: pull updates
  • Opportunity: pull deletes
  • Opportunity Contact Role: pull deletes
  • Opportunity: pull updates
  • Opportunity Contact Role: pull updates
  • Event: pull updates
  • Task: pull updates
  • Email Template: push new
  • Email Template: push updates
  • Task: push new
  • Task: push updates
    
    
• The above are all standard objects. Any custom objects you may have created will be prioritized after all of the standard objects have synced. Reminder: Contact updates will only occur as long as there is no backlog on the Account object.
  
  

Getting around the sync backlog

• Using the above info, you should be able to identify roughly how long the sync will take based on the amount of records and fields you will be updating.
  • There isn’t really a way around this that I have found except for spreading out any large data updates over time. I asked Marketo if there was a way to get a temporary increase in bandwidth for the sync cycle while we performed our annual maintenance, the answer was no. The speed in which the sync operates is dependent on Salesforce’s API. Increasing the batch sizes will only increase the duration of the sync cycle for the object that has been increased, it will not increase the throughput for the overall sync process.
  • As is a recommended best practice, if your update is going to be very large, it is recommended to break down the update into smaller chunks and spread them out over time to avoid this backlog. Also, schedule the updates to occur outside of business hours, to avoid any impact to your daily operations.
    
    
• Take note of any sales alerts that this update could impact. For instance, if you have a sales alert sent out of Marketo, and you are currently experiencing a backlog, Marketo might alert Sales of information that doesn’t exist in Salesforce yet. This can definitely create frustration among Sales!
  • If volume here is relatively low, you can always consider adding a “Sync Lead to SFDC” flow step to trigger before your alert is sent. Just be careful with how many of these are used as each time this flow step occurs, it counts as an API call. Depending on the system you are syncing to, some have limits on how many API calls they can receive in a day.
    
    
• Document any operational rules that automatically append data in different scenarios (e.g. when a new lead is created) and how many fields the operational rules update. This should be done for both Marketo and your CRM so you know how many total fields will be updated when performing large list uploads or data migrations.
  
  

That’s about all I have for this right now. Hope this information can help anyone else here in the community.

Thanks!

Jason

NOTE: Post updated on 04/03/17 to clarify a few technical details provided by Marketo support.

[Kristina_Schro1]

Thank you for sharing. Does anyone know of a way to see the sync backlog either via Salesforce or Marketo? I wish I could get more insight into whats going on. Thanks

[Chris_Glanzman]

Every time I have investigated the sync backlog, I have had to work with Marketo support, and they have given no indication that I could access the information in a different way.

[Maritza_Henriq2]

Does anyone know of a way to identify records that were in the process of going through a flow when the API limit is reached in SFDC? We are trying to find a way to isolate them so we can set up a new triggering action for the lead to go through the flow again. If anyone can share a scalable way to manage this, it would be much appreciated. Thank you!

[Mike_Reynolds2]

Hi all. Just wanted to drop a note in to mention that we had some incorrect information given out in one of our Support cases, which is why Jason edited the post above. Felt it was was worth mentioning that this is an excellent discussion with some really good detailed info. The clarification edits made were due to errors on our side in the case. We got one of our integration experts to confirm that the info in the original post above is 100% accurate. Again, excellent post with lots of great information!

[Denise_Greenb12]

Mike Reynolds​ - One of my clients was having a big problem in December of 2017 due to the sync objects priority. At that time, we were told that in the standard order of priority lead conversions were processed only after accounts were updated. The list up above in Jason's post seems to indicate the opposite.

The problem we were having a year ago - and seem to be having again now - is that there are times when Marketo sees a lead-to-contact conversion as a "New Person." Then later (sometimes over an hour later), you see a correction - Marketo executes a "Merge" on the 2 records followed by a "Convert Person" event. This causes all kinds of problems because the "New Person" event causes campaigns with "Person is Created" or Data Value changes->Previous Value is NULL" triggers to fire.

I'd like to confirm the "standard" order of priority so as to better understand what I'm seeing now. A year ago, Marketo made a change to the order of priority for us that seemed to make the problem we were seeing then better. I'm wondering if this got reversed because now we're getting double demographic scoring caused by New Person events caused by conversions.

(I have logged a support case on this. Just want to confirm how things are supposed to work).

Thanks in advance!

[Nina_Valtcheva5]

Denise Greenberg​ - we're dealing with the same issue where Marketo counts a lead to contact conversion as a new person creation. I'll raise a ticket too but would be interested to see if Marketo Support was able to help offer a solution for you. This is impacting our lead scoring program since the person's score values are essentially doubled every time this happens which is so misleading.

[Denise_Greenb12]

Hi Nina,

I have an open case with support now that has been escalated and will be glad to report back here when we have a resolution.

This is an issue I have observed on and off for a few years in different Marketo instances and I think Marketo should find a way to fix it. In particular, once Marketo "realizes" that the new person it created is not new but a conversion, it executes a "normal" merge - which means the Contact record wins over the Lead record, data in fields that were populated on the Lead record but not on the "new" Contact record gets lost - and the new person ID persists. It seems to me that since Marketo is able to recognize that what it thought at first was a new record was actually a conversion, that the engineering team should be able to find a way to treat it as a conversion so that the data integrity is maintained.

As I said above though, I will report back!

Denise

[Denise_Greenb12]

Hi All -

Here is the promised information as a result of our escalated support case. It turns out that our Marketo instance has Multi-batch queuing enabled. With MBQ, the objects sync simultaneously so the sync order shown above (and also documented here - by the way: https://nation.marketo.com/docs/DOC-2006) doesn't apply. To quote Marketo Support: "When an instance is using single batch queue, pulling lead conversions always executes before contact/lead updates. When multi-batch queue is enabled, the order is undetermined and we cannot guarantee that pulling lead conversions will always execute before pulling lead/contact updates.

Our engineering team is currently looking into possible solutions to prohibit that behavior and it is also slated for our Q1 release."

Regarding the 2 instances I found where the conversion synced over as a New Person and was not subsequently merged with the pre-existing record, Marketo said "we see on the back-end that the reason why we did not merge the lead and contact record is due to an exception that was returned when Marketo tried converting the lead and contact record. This is a known issue with our engineering team and the solution for this is slated for our Q1 release."

Denise

[Dan_Stevens_]

We've recently come across this issue on the MS Dynamics side (especially after upgrading to v9).  Although, in our case, the dupes remain and require manual intervention to dedupe.  But the biggest issue is the "new" record is now the active record and contains no prior activity.  Is this what you're seeing on the SFDC side (if only temporarily)?:

Duplicate records created in Marketo after upgrading to Dynamics 365 Online v9

[Denise_Greenb12]

Hi Dan,

I have seen two instances of this - where the conversion syncs over as a new (Contact) record and Marketo does not catch up later and merge it, resulting in duplicates on the Marketo side. I don't know how many other instances of this there may be lurking that I haven't caught. This is clearly another result of the same underlying problem and the Marketo Support escalation team is aware of it. I'll report back specifically on this, too.

Denise

[Nina_Valtcheva5]

Wow that's even worse! We don't get any dupes in SFDC or Marketo after this happens.

[Mike_Reynolds2]

Hi Denise Greenberg,

Best bet would be to check with Support, and it sounds like you already have a case open. I'm not doing direct support any more, so I'm not as well versed as I was previously. But, I know our Support team will have the most current and the most accurate info. Mention this discussion thread in the case so we can get an update here if any details need to be corrected.

Thanks!

Mike

[Denise_Greenb12]

Hi Mike,

Thanks very much. I have just done so. In the meantime, I wonder if you remember enough to just clarify something for me. If the above is correct - which bullet point includes pulling new Contacts that were created. In my (unfortunately sketchy) notes from a year ago, what I wrote down was that Marketo was re-ordering things for us so as to pull "what Contacts had been created" after "what Leads had been converted." I'm having a bit of difficulty reconciling that terminology with the terminology above.

Denise

[Dan_Stevens_]

Is this relevant for both SFDC and MSD?

[Mike_Reynolds2]

Yes and no. The batch size of 10k per sync operation is the same, and the recommendations for getting around the backlog would work with MSD also, but there's a difference in the sync cycle operations because the objects in MSD are different than those in SFDC. Core concept is essentially the same, but the objects are different.

[Anonymous]

Great post here, I understand the sync objects priority, but how do field changes impact backlog?  Does it matter if 10,000 records have one field update or 30 field updates?  Would the backlog be the same or would more field updates cause the backlog to slow down even more?

[Grégoire_Miche2]

Hi Jason,

Thanks for this detailed and very informative post.

A few comments :

• I have obtained from support a temporary increase in sync bandwidth for a large insertion of records (>3 000 000 records)
• The limit of 10000 field updates per hours seem very low to me compared to my experience. for instance, I create 4600 leads in Marketo from SFDC in 10 minutes with 6 fields in each record.

-Greg

[Anonymous]

Hi Greg,

    I found out more information on this:

• The 10,000 updates are per object, not per field, as I originally thought. Marketo does say that this number is a rough estimate and will vary based on the object type.
• When I asked for clarity around the bandwidth increases, they mentioned that they cannot raise the total bandwidth as this is a limitation of the Salesforce API. What can be changed (on a circumstantial basis) is the amount of records that are synced per object type before moving onto to the next object type.
  • From what they explained, the sync process handles objects in a "waterfall-style" where it will sync ~10,000 of one object type, then continue to the next object type. If the amount of updates on an object is greater than the ~10,000 cap, the sync will not just sit on that single object until it is done with all updates. It will sync the ~10,000 updates on the object type, then move to the next object type and will go through the list I have above in that order.
  • So, what the increase can do, is lift the cap from ~10,000 to, say, 20,000 so it can handle more of one object at a time. But this would only be done in a situation where there needed to be a one-time effort to do a large update on a single object type, like the example you provided.

Jason

[Grégoire_Miche2]

Hi Jason,

Thx, this is much more in line with my own experience

-Greg

[Anonymous]

Hi Greg,

     Thanks for the input! That's unfortunate to hear about the temporary increase in sync bandwidth, I put in a support case asking specifically for this and they told me that it was not possible and that I need to break up my data and perform my annual updates over a longer period of time.

As for the 10,000 field updates, I agree, it sounds very low from what I've experienced with past systems. I received this number from two different support contacts at Marketo, so I am wondering if it is a "soft" limit that they try to encourage customers to stick to.

Jason