All about Lead Deduplication and Merge

Level 1

All about Lead Deduplication and Merge

All of us know Marketo automatically Merges (De-duplicates) records when created with the same email ID. But then, there are scenarios where the leads will be created as duplicates and are not merged even though they have the same Email- address.

Let’s check out the scenarios and how to merge them.


Scenario 1: The leads coming through Form fill out.

Now for example: when a lead is filling out the form and has some issues with the submit button or network issues or maybe an issue with the form, when the lead tries to click on submit button multiple times continuously in the same time, no matter how many times the lead clicked on submit button, the same number of leads are created in the instance and Marketo won't be able to identify the original lead because of this “Race Condition”. Hence the duplicates are created and should be merged Manually.

If the duplicates are high in number then, unfortunately, there is nothing can be done from our end, you can recommend them to professional services.


Scenario 2: The leads coming through SFDC.

We saw many scenarios where SFDC creates duplicates. But why? Because Salesforce creates a new person based on SFDC ID, and when Marketo sees the new lead, it looks at the ID, not the email address. So, if a Lead exists in Marketo and then a Lead with the same email address is created in Salesforce and syncs down, Marketo will create a new duplicate Lead.

For example, You have a lead in Marketo from Form fill out and the same lead already exists in SFDC and synced to Marketo but then created two records when sync happened because Marketo lead doesn’t have SFDC ID because they didn’t sync to SFDC to have an ID. The best way to do so is to create a lead via a form fill and then sync it to Salesforce. If that cannot be avoided, then the records must be merged Manually or through Easy Merge (through Marketo Services).

(As a best practice, always merge the fields in SFDC and sync down to Marketo)


Scenario 3: The leads coming through Webservice API or Rest API.

We all know that Marketo does de-duplicate leads (based on email addresses) from web forms and list imports, it does not necessarily de-dupe leads coming in via the REST API. It depends on which REST operation is used. When performing lead upsert requests concurrently or in quick succession, duplicate records may result when making multiple requests with the same key value if a subsequent call with the same value is made before the first returns. This can be avoided either by using the createOnly, or updateOnly as appropriate or by queuing calls and waiting for your call to return before making subsequent upsert calls with the same key.

To Merge the leads, there is a paid service called Marketo Automerge, which you can purchase from your customer success manager.

You can find more information on the following link:!/Leads...


Note: The best possible way to avoid duplicates from different sources is to have a secondary deduplication field. You can ask the customer to add the custom dedupe fields on the below functions of Marketo.

1)Web Form Fillout
2)Munchkin API
3)Webservice API
4)List Import
5)Web Service API - List Import

Once the rule is applied Marketo will not allow the duplicates to create in Marketo from the above possible ways.


I hope this helps.

Level 1

Re: All about Lead Deduplication and Merge

How does the custom dedupe fiedl work? You create it, what kind of field? How is it supposed to function to prevent duplicates.

Level 10 - Community Moderator

Re: All about Lead Deduplication and Merge

How does the custom dedupe fiedl work? You create it, what kind of field? How is it supposed to function to prevent duplicates.

You're right to wonder!


A custom dedupe key isn't a single field. It's a compound key consisting of the Email Address and another (typically string) field.


It helps tune your instance to your business's definition of a duplicate. For example, in your instance, 2 people with the same Email Address but different Phone Numbers may not be considered duplicates, as they are marketed to separately.


It wouldn't say this "prevents" duplicates, but rather it more accurately defines duplicates.


Most Marketo experts are not in favor of compound/custom dedupe keys because they cause more problems than they solve.