Salesforce Sync Backlog

[Anonymous]

Get ready for a mouthful.

We update changed records every 2 hours and all records every night, and we are experiencing HUGE backlog issues from our overnight sync. The sync rate is healthy enough for our 2-hour updates, but it gets so bogged down on the overnight that it takes most of the day to catch up. And since account updates outrank contact updates, there's a big delay in when new contacts are getting their welcome info.

I need help on figuring out how to change either the way we update records or what fields are causing updates. But we collect such specific product usage data that it seems we're victims of our own success.

Has anyone experienced anything similar to this? How do we fix it?

A big internet hug awaits whoever can help.

[Anonymous]

Around 4 pm on March 13, we observed that it was taking over 5 hours for new Contact to synch to Marketo and we had no backlog.  We reported this to Marketo tech support that afternoon and around 7 pm the following day they responded "On 3/13, our engineers detected a slow down with the SFDC Sync. We are investigating this issue and have an escalation in place." 

We haven't received an update yet.  We've noticed a rash of performance degradations with synch and emails over the last week or so.  It may be a load balancing / capacity issue and take some time to fix.

[Matt_Rushing1]

Zachary - I realize this is an old post, but what did you do to resolve the backlog issue.  We have pretty much the exact same issue you describe, and Support hasn't been able to help.  They claim it's an issue on our end syncing too much data, but we don't feel that it's out of the norm to update usage data several times per day.

[Anonymous]

Zachary Hanz​! Matt Rushing! I feel your pain! We had some *major* issues with this, and while we've experienced issues since November 2014, we resubmitted a ticket on 5/19/15, just around the time you guys experienced issues.

After speaking with our premier support rep over a couple weeks, they found a bug: "We've heard back from engineering and it looks like we have a diagnosis. What they found is this: For the fields on the Account object that are not visible to the synch user, whenever one is updated, Marketo is checking and synching the record when it's more efficient to skip it . This is in the case where the only update to that record is made on an invisible field. This is an "expensive" synch operation and engineering has said that they are going to change the behavior. I'm working on getting an ETA from them, and all of our possible workarounds in the short-term. I'll get you an update as soon as I have something more concrete from them. Thank you both."

Basically, they found a bug that still updates all Account fields, regardless of whether those fields are visible or not to the lead.

Looks like they are trying to push out a patch in the next couple weeks.

A couple things you can do in the meantime (I really need to write a blog post about this):

• Enable SFDC Campaign sync, and set up your system so that leads sync through Programs instead of a "Sync to SFDC" campaign. Enabling SFDC Campaign/Program sync allows for 300 leads per API call, as opposed to 1 lead per API call in the "Sync to SFDC" campaign flow.
• Hide fields from your Marketo API user in SFDC and Marketo (you have to hide them from both!)
• Are there any custom objects synced to Marketo instance for no reason? Disable that sync!!
• Are you using SFDC integrations that look for things like "Last email sent", "status updated on", "date submitted"? If these fields/integrations are synced to Marketo, it makes it super noisy.
• Ask support to up your API limit temporarily to get you through the backlog, once you've enabled SFDC sync. This will only work however if you take advantage of the SFDC program sync!

Let me know if you have any questions, I have daily charts and everything, as it was holding back leads from entering our system!

[Anonymous]

Zachary Hanz​ did this answer your question?

[Anonymous]

Matt Rushing​ what Allison Sparrow​ mentioned is basically what we wound up doing a while back. I believe they upped our API limits, and we also blocked/hid a TON of fields. It was partially our fault with how often and what fields we update from our database, so it was mostly a matter of determining what fields we really needed and getting smarter in our logic (using static fields like Trial Start Date = X Days Ago vs. constantly synced fields like Days Active = X).