Catching and Removing Duplicates

Level 1

Catching and Removing Duplicates



I'm noticing that in my Marketo instance there are a number of duplicates that are created. There are three sources a lead can enter our Marketo instance. An API integration to our backend data from our signup flow, a Marketo form fill, and a lead that is uploaded to Salesforce and then syncs to Marketo. There are cases where three leads are created with the same email as they entered from all three of those sources.


Does anyone have suggestions on campaigns or way to catch a duplicate and marry it to an existing lead. Additionally, does anyone have suggestions on how to properly remove duplicates that are taking up database storage currently?


Thanks in advance for any help!

Level 10 - Community Moderator

Re: Catching and Removing Duplicates

You can't "catch" a duplicate while it's in the process of being created, you can only merge it.


Limit the creation of duplicates by making sure you use the Email Address field as a lookup field wherever possible: some code using the API doesn't pay enough attention to the lookup field. And a large number of leads — the definition of "large" is up to you, of course — being created from SFDC where there already is an unsynced, Marketo-only person with the same email address signals that your MQL process needs some tuning.


Your Marketo instance will already have a System Smart List called Potential Duplicates in the Database area, and you can create your own SL of the same type.



Level 6 - Community Advisor

Re: Catching and Removing Duplicates

Hey there! Be sure to check out some of these related posts:


Your API and form sources hopefully shouldn't be creating duplicates, so SFDC is likely the culprit if you don't have any measures in place on the SFDC side to prevent duplicates. We run into duplicates most often when a person exists only in Marketo, and then the same person is created later via SFDC as a contact.


You can manually merge records together, but you should take a look at your existing processes to understand where the duplicates are coming from, and then look into solutions. We opted to go with Marketo's AutoMerge solution to scale the deduplication after we mitigated duplicates to the best of our ability internally.


To answer your question about removing duplicates, you'll have to decide what you're willing to part ways with in terms of data, but you could look for person records who are duplicates with no activity, and remove those.