One-Time De-Dupe Tool or Service Recommendations

[DeniseGreenberg]

Hi All! I'm looking for advice on a tool or service for a client for help with a one-time dedupe project. I posted about this a few years ago but this is a different situation (and a different company) and there may be new tools or services available now. Here's the situation:

1. Client has Marketo and SFDC
2. Client has duplicate rules enabled in SFDC
3. 60K+ records in SFDC are prevented from syncing to Marketo by means of a checkbox field and a Marketo-created custom filter on the back end
4. As a result of #2, they have just under 7K records in Marketo that can't sync to SFDC because SFDC sees them as dupes. They came in again via a form fill or similar.
5. Client wants to uncheck the "Do Not Sync" box and let all the 60K+ records from SFDC into Marketo
6. This will likely create even more dupes in Marketo but once these are cleaned up, the source for dupes will go away along with the resulting sync problems (yay!)

I'm looking for recommendations on how to do the one-time dupe clean-up. Note that the duplicates will be in Marketo, not in SFDC (because of the SFDC dupe rules) but to do the merging in Marketo we would need a tool that could merge Lead-Contact pairs. I've read that someone who knows how to do API coding (i.e.., someone with @SanfordWhiteman type skills) could use Marketo's REST API to do bulk merges but I don't know if this could handle Lead-Contact merges. So...

1. Tools or human API coder recommendations to do a one-time mass de-dupe in Marketo?

2. Or would we have better options available if we turned off the SFDC duplicate rules, let the Marketo dupes sync over, and then did the cleanup in SFDC?

I'd hate to lumber my client with a subscription service when they really only need a one-time cleanup.

Any suggestions welcome!!

[SanfordWhiteman]

For a one-time deal, you can run the potential duplicates created that day through a webhook which in turn calls the Marketo REST API merge endpoint. Since I know you already use FlowBoost, that can be used as the webhook platform.

Marketo can merge Marketo-only Persons and SFDC Contacts.

[DeniseGreenberg]

Hi @SanfordWhiteman - That would be great but:

1) How would we handle the existing duplicates?

2) For the potential duplicates created that day, do you mean trigger the webhook with "New Person Created" and a filter: member of smart list in Potential Duplicates? (Once we uncheck the Do Not Sync box I anticipate it taking a few days for all the leads to sync over)

3) When you say Marketo-side merges do handle Lead-Contact merges, do you mean when they are done programmatically via the API or a Webhook? Because you you can't do Lead-Contact merges via the UI Merge function.

[SanfordWhiteman]

---

1) How would we handle the existing duplicates?

---

Those could be run through too, you could just start with the ones you know you’ll be creating by opening the floodgates.

---

2) For the potential duplicates created that day, do you mean trigger the webhook with "New Person Created" and a filter: member of smart list in Potential Duplicates? (Once we uncheck the Do Not Sync box I anticipate it taking a few days for all the leads to sync over)

---

Yes, could be over a few days, again it’s just a way to isolate the latest-created person from each set of dupes so you don’t waste resources sending both loser and winner through the ‘hook.

---

3) When you say Marketo-side merges do handle Lead-Contact merges, do you mean when they are done programmatically via the API or a Webhook? Because you you can't do Lead-Contact merges via the UI Merge function.

---

Isn’t one of the people going to be a Marketo Person (only) and the other a Marketo Person & SFDC Contact?