If duplicate leads are an issue, be aware of how Marketo determines which lead is active

[Dan_Stevens_]

Many of us deal with duplicate leads on a regular basis.  Today, I just learned of something that's been causing issues for us, as we were always under the impression that Marketo considered the lead that was "updated last/most recent" as the active lead - and the lead that would be affected for things like list uploads, email sends, etc.  That's not the case.  This could have significant implications if you have a lot of duplicates in your instance. I just posted an idea that hopefully will get enough attention and change the way Marketo deals with duplicate leads:

NOTE: if you head over to the Idea page ( ), you'll note that a serious flaw has been identified - specifically around the way Marketo re-uses old LeadIDs (when the original lead is purged from Marketo) and allows new leads to take on these old IDs.  This could have serious implications, for example, for those of us that leverage the API in our environments.

[Grégoire_Miche2]

Hi Dan,

Thx for posting this and the idea. Learnt something.

-Greg

[Dan_Stevens_]

For your reference, here is a detailed breakdown of Marketo's duplicate rules processing:

Sending Batch Emails

If there are duplicates within your email list, only one email will go out. It will be the lowest Marketo ID (created first). This record is also used to populate tokens.  Example:

Created	Updated	Marketo ID	First name	Email
3/13/2010	4/13/2010	2749	Elizabeth	beth@company.com
12/15/2012	3/15/2016	10936	Beth	beth@company.com

In this case, the email will be sent to the first record with Marketo ID 2749. It will say “Dear Elizabeth” if you use a token for first name. Also, the activity entries will be attached to that record (sent, delivered, opened, clicked). When the email click goes to a page with Munchkin code in it, any web visits will also be associated with this record.

[EDIT 1/30/2018]: apparently this is not the case.  As you can see in the latest comments, here in the Idea related to this: , Aaron Dear​ uncovered a serious issue/bug - especially if you have a lot duplicates in your instance (in some cases, these duplicates are intentional).  Basically, there is no defined logic to determine which lead record is used when evaluating duplicates as part of a batch email campaign.

List Uploads

When a lead is included in a list import and the lead already exists in Marketo, it will affect the most recently updated record in Marketo (updated last) - in this case, the record with the first name "Beth".  If you're intention was to ensure Beth's record had the most up-to-date information so that some of that info could be used in an email send, you're out of luck.  Marketo will use "Elizabeth's" record for that send.

Sending Trigger Emails

While a batch email de-duplicates, trigger emails do not de-dupe because triggers “batch each record individually”. However, it is uncommon that trigger emails are used for batch emails, so this is not usually a problem. Also, it is rare that duplicate records will qualify for a trigger campaign at the same time.

Form Fill-outs

If someone fills out a form and their email address is in Marketo multiple times, the form submissions will be applied to the most recently updated record (by last modified date in the “updated” field). The data value changes in Marketo will only affect one record if there are duplicates; they will not affect the duplicate records. For example, if a record changes their title, the change will not occur for the duplicate records.

Assuming that the Marketo cookie information is included in the form (standard for Marketo landing pages or embedded Marketo forms), the web page visits will now be tracked on the most recently updated record. Luckily, the most recently updated record is often the record that is most up to date. (which is why the the way Marketo treats batch email sends is an issue).

Note: If there are duplicate records in Marketo and one of the records is cookied, the cookied record will be associated to the form fillout regardless if a duplicate record was updated last. As noted above, the form fill-out activity may not be associated to a record which was sent an email directing them to the form fill-out.

Web Activity

Web activity will only be associated with one record if there are duplicates. If a record has a duplicate record and he/she goes on the website, the munchkin code will only track activity for the record that most recently was cookied by submitting a form. If the visitor’s cookie is reset and visits the site again and fills out a form, the record may be associated to a different record in Marketo. Therefore, all web activity that is recorded may be spread across the duplicate records.

Merging Leads

When merging two leads, Marketo will always use the most recent one to define the "Created On" date.  This is undesirable since most would prefer to keep the initial/first date of the older lead as the "Created On" date.

Thanks to Kristen Carmean​ for surfacing this comprehensive list with us!

[Dan_Stevens_]

Here's a situation where you can mitigate the amount of risk (if duplicates are an issue).  Let's say you have an event program with 25 registrants.  10 of those registrants exist as duplicates.  But the most recent lead records are included in this program.  When sending out a reminder email, for example, as long as you define your smart list properly (e.g., member of program; program status is registered; etc.), only those QUALIFIED leads will be sent the email.  This is a perfect example where Marketo will, in fact, use the most recent lead record when sending out a batch email since that's the record that qualified in the smart list to begin with.

Hope this helps!

[Mark_Wallace1]

Hi Dan,

Im now having a number of issues with dupes and seems your in the same boat.  Quick question re the above.  In our instance where dupes are allowed any form fill creates a new duplicate.  So for the above how do you assign a form fill out to an existing version of the lead?

I have an approval process where sales fill a form that updates the lead record and triggers some emails to go to the customer.  But now when they fill the form it just creates a dupe which needs a manual merge to put the lead through the flow.

With duplicates allows how can you assign a form fill out to an existing lead?

thanks

Mark

[Dan_Stevens_]

Hi Mark - Marketo dedupes (on email address) by default on a form submit - and the "form fill-outs" scenario above would apply.  Do you have a custom setup where a purposeful duplicate is created on form submit?

[Mark_Wallace1]

Hello

Unfortunately we do, its a checking if the content of the product interest field changes and deciding if a dupe needs to be created.  Its a function of the marketo standard connector and MS dynamics.

Its starting to cause issues in more ways than one.

thanks

Mark

[Jenn_DiMaria2]

Wow. Thanks for posting this. I always thought it defaulted to whatever was in SFDC (because that's what it always seemed like when I'd go to merge them). Learn something new every day

[Dan_Stevens_]

Jenn - do you automatically sync all leads from Marketo to SFDC?  If so, then your assumption may still be true.  The issue arises when you have duplicate leads/contacts in SFDC (which, of course, also exist in Marketo).

[Jenn_DiMaria2]

SFDC is our master and we don't sync all Marketo leads to it, but we still have lots of issues with duplicates. Namely because our main contact form on the website is an SFDC web-to-lead form and many of our customers think they can reach Support this way. Of course, that then creates duplicates, which makes my life super fun

In many cases with those duplicates, we then have a Contact on an Account and then a Lead. If they've merely submitted the W2L form, those are easy to merge, but I remember when we first migrated our database, there was some best practice from SFDC about holding a Lead separately from a Contact, despite being the same information. I wish I remembered the details, but it's such a data pain.