Looooooooong delays with sync from SFDC

[Anonymous]

Does anyone have tips to troubelshoot when the bi-directional sync between Marketo and SFDC takes hours? Especially when a lead/contact is created in SFDC and takes forever to sync to Marketo? Has anyone else experienced delays in the SFDC sync?

Some context:
We've been exerpiencing lengthy delays for the last two weeks, been working with Marketo support, but still it takes HOURS to sync to SFDC. Here's the last 5 syncs to give you an idea, it's unpredictable and ridiculous. 

Sync in progress: Started {Aug 14, 5:04 AM}
Synced Aug 13, 10:19 PM
Synced Aug 13, 11:40 AM
Synced Aug 13, 12:35 AM
Synced Aug 12, 3:20 PM

A support rep said: "Part of the issue is that when you use the Marketo Program to Salesforce Campaign sync the action of adding/updating the Lead to the Salesforce Campaign is queued up and performed by the bi-directional sync. If the Lead is converted, deleted, or removed from visibility to the Sync ID it will not be recognized and as a result Marketo will be bringing an old SFDC Lead ID that will not be valid for one of those reasons."

I'm not sure if that's the main reason for the delay, but we are now adding/updating members to SFDC campaigns in the Flow rather then syncing to SFDC campaigns at the program level. 

We have smart campaigns that update/normalize data when a lead or contact is created in SFDC - but those have been running for months now. There hasn't been a massive upload/update of leads/contacts into SFDC recently. So, I'm at a loss and need some help...

Thanks!

[Anonymous]

Out of curiosity, as we are having similar problems, have you seen any improvement in sync times by switching to flowing campaign data to SFDC vs. program-level sync?

[Anonymous]

Hi Courtney,

We haven't seen any improvements just yet from switching to the Flow step. But we're optimistic that this will help in the future since some of our reps convert leads to contacts very quickly. 

[Conor_Fitzpatri]

Grant,

I'd suggest you perform an audit on the fields syncing over from SFDC to Marketo. Support can help identify the fields that are syncing the most data. We had tons of SFDC fields that changed frequently (and therefore were hogging up space in the sync queue) and were vital to our enigneers, but had no impact on our marketing programs. If you are selective on which fields are being synced to Marketo, you can greatly improve performance and clear some space in the sync queue.

[Anonymous]

Conor has a great point. Only bring into Marketo what you need for your activities. Removing fields that are constantly updating can help with the sync time, especially if you have a large database. How many records do you have in your database?

[Conor_Fitzpatri]

Also, I think that turning off some of the Activities such as Email Sent in the Sync Options menu (under Admin - Integration - Salesforce) will help performance. I'm not certain of this, but it seemed to help. Besides, it saves on API calls and storage space in SFDC.

[Anonymous]

So I actually have this problem on a regular basis......

If it's not a specific field, or a mass update hasn't been done, then here is the alternative that you may not know about.

We have a lot of code that looks at transactional data and because our code looks at every single account to check if new transactions have been uploaded.

Even though there is no "update" to any Account records therefore no changes to contacts, because the SQL process grabs the account and adds it to a table but actually does nothing it still counts as an edit, "a system edit".

So it's not necessarily something where you can block a field update

It's really hard to report on in SFDC and Marketo so all I have is a indication kind of report and I take this to our SFDC developer to check if they've had processes run that touch every account......normally it's a yes. Generally they work with me to try and rebuild their processes to try and only look at the data that actually needs updating

My contacts can actually take up to 24 hours to fully run through when I have one of these updates. So imagine my pain. I know this is a much bigger delay than what you've got, but it is handy to know.

You can create a smart list and use the filter "Updated At" and put in a date range and when you see nothing has been updated within your date range then there has been a system edit.

You can log a case to Marketo Support and they can help you to speed up the process if your backlog is huge.

[Edward_Unthank_]

Hey, this is one of my specialties! Send me a message and I'll show you a behind-the-scenes troubleshooting tool.

What you're going to want to do is narrow down the problem to see where you can hone in your effort in trimming the sync down. Here are the steps I would recommend taking:

1. Call Marketo Support and get the numbers for each object type that is syncing in a normal sync cycle. You'll probably find that a certain object or two is the real problem child, and then you can know to focus in on expediting that individual sync. If you call up Marketo Support, they have a tool on the back-end that shows how many individual objects get synced at a certain time. Have them tell you these numbers, and you can see if the problem is regarding specific object syncs such as the Account object, Lead object, Contact object, Campaign object, or otherwise. Each one has a certain amount of records to sync in a normal sync cycle, and a whole sync goes in a certain order where it has to go one-at-a-time—aka, if your Account sync is taking 3 hours every sync cycle and your Lead sync is taking 2 minutes, Lead sync will have to wait for Account sync every time regardless.
2. Call Marketo Support and have them get a list of the most-updated fields (I believe they give you the top fifty fields). This will prioritize your fields in determining which fields are problematic, and then you can use that to make business decisions on whether they should be hidden or if you should push Marketo to come up with better options.
3. Figure out what your noisy fields are and see if you can retain the data without requiring constant field updates. If there are fields that are changing values daily or more frequently than daily, see if there's a way to switch them to not update as frequently. A quintessential is a field for "days until renewal" that updates daily for every lead/contact. That kind of field will update for every lead in the database, so you can instead choose to create a "renewal date" so that it's updated once a year instead of daily. Or if you have a SaaS "total spending amount" in dollars that's updated from your product, you can bucket that into spending categories via formula fields or workflows that are updated less frequently than the exact dollar amount.
4. Hide fields that are updated consistently but don't provide actual business value to be held within Marketo. I mean that pretty precisely—the easy go-to answer Marketo Support will provide is that you need to hide more fields, but if that value provides active business value to reside in Marketo, it should stay there. Also, if you have a bunch of fields that exist but are only updated once in a lead's lifecycle, you're going to go crazy with long-tail efforts to clean them up when in reality you could hide 90% of them without addressing the real problem of a sync backlog.
5. Program/campaign sync is actually more efficient for API calls than "Sync lead to SFDC" flow steps, if you're running up against your daily API limit for SFDC API calls. You can get a max of 300 leads synced/API call for a program to campaign sync than through a "Sync lead to SFDC" flow step (which gets a maximum of 1 lead synced/API call). I can't speak to the issue you identified here about old leads still being synced to SFDC, just needed to put this in as an efficiency gain overall.
6. Based on how staggered and inconsistently timed your sync cycles are, a problem might be that SFDC data is being batch-updated all at once through a SFDC integration with something else. If you see this problem at 5am a lot, I would talk to your SFDC admins to see if there's a big nightly batch happening that updates or enriches lots of fields.

If you get through those, there are some technical areas in which Marketo might become more efficient. For example, in the last situation in which we had a huge SFDC/Marketo sync backlog, we found that any update to an Account, regardless of whether or not the Account field was visible to Marketo, triggered every attached Contact to go into a "to-be-synced" backlog to Marketo. We identified it as a problem and then Marketo Support was able to prioritize as an issue and find a different way to queue the syncing. I don't know if that update applied to everyone who uses Marketo or just that specific instance, though.

Basically, you'll want to optimize as much as you can for field syncing and getting streamlined and lean updates. Don't settle for syncing fewer leads between Marketo and SFDC! That's the easy way out that will just handicap Marketo's actual real value.

Cheers,

Edward Unthank | Founder, Etumos

[Justin_Eichelb1]

I agree, we had a similar issue, and we constantly hounded Marketo until they started to make changes on their end.  They have resources to affect your performance.  From prioritizing your instance in your pod, to what Unthank just stated.