When dealing with duplicate leads, Marketo should stay consistent on which lead record is considered the active/current lead

When dealing with duplicate leads, Marketo should stay consistent on which lead record is considered the active/current lead

If you have duplicate lead records in Marketo, it's important to know which record Marketo considers the "active" record.  Consider the following example:

Sending Batch Emails

If there are duplicates within your email list, only one email will go out. It will be the lowest Marketo ID (created first). This record is also used to populate tokens.  Example:

Created

Updated

Marketo ID

First name

Email

3/13/2010

4/13/2010

2749

Elizabeth

beth@company.com

12/15/2012

3/15/2016

10936

Beth

beth@company.com

In this case, the email will be sent to the first record with Marketo ID 2749. It will say “Dear Elizabeth” if you use a token for first name. Also, the activity entries will be attached to that record (sent, delivered, opened, clicked). When the email click goes to a page with Munchkin code in it, any web visits will also be associated with this record.

[EDIT 1/30/2018]: apparently this is not the case.  As you can see below in the latest comments, Aaron Dear uncovered a serious issue/bug - especially if you have a lot duplicates in your instance (in some cases, these duplicates are intentional).  Basically, there is no defined logic to determine which lead record is used when evaluating duplicates as part of a batch email campaign.

List Uploads

When a lead is included in a list import and the lead already exists in Marketo, it will affect the most recently updated record in Marketo (updated last) - in this case, the record with the first name "Beth".  If you're intention was to ensure Beth's record had the most up-to-date information so that some of that info could be used in an email send, you're out of luck.  Marketo will use "Elizabeth's" record for that send.

We’ve been running into issues with duplicate leads that exist in the same lead partition – specifically when sending batch emails that include tokens for Sales Execs (BDE) First, Last, and Email.  To ensure this info is updated before an email goes out, the marketer will make sure the list import contains the most recent data.  The emails still went out with outdated content/data.  Uploading a list with updated data won’t matter (when there's a duplicate) since Marketo doesn’t use the most recently updated record in Marketo when sending out the email.

Obviously, the main objective should be to deal with duplicates appropriately and regularly.  But this often is easier said than done.

When duplicates exist, I would like to see Marketo treat the active record consistently - in this case, the same for when sending batch emails AND list uploads - so that we can be certain our emails contain accurate personalization data.

To see Marketo's full list of duplicate rules, head over to the discussion: If duplicate leads are an issue, be aware of how Marketo determines which lead is active

29 Comments
Dan_Stevens_
Level 10 - Champion Alumni

Aaron - while you're having this active dialogue with Support, can you have them comment on this then?  Specifically, which record does Marketo use to send a batch email: the one with the lowest ID or the one with the earliest creation date?  Or did you already conclude from your testing that the lowest ID would receive it?

Sending Batch Emails

If there are duplicates within your email list, only one email will go out. It will be the lowest Marketo ID (created first). This record is also used to populate tokens.  Example:

Created

Updated

Marketo ID

First name

Email

3/13/2010

4/13/2010

2749

Elizabeth

beth@company.com

12/15/2012

3/15/2016

10936

Beth

beth@company.com

Aaron_Dear
Level 3

Support person's quotation:

I would once again like to refer you to another article I found about this subject. It is actually written by one of our TSEs.

The ruleset for which record is considered to be the winner, actually depends on the activity being performed, and this article goes over a lot of the questions you have.

https://marketoemployee.jiveon.com/docs/DOC-10512

(Aaron's Note: this is a Marketo Employee internal link I think - can't access)

The article does say that for email batches, it uses the lowest Marketo ID (smallest number).

My support case is closed - but my testing shows that the smallest ID gets the send.

Marketo - Dupe Flow Test.png

Here, there are 3 leads in a flow with 2 steps:

  1. Step 1 - Change Program Status
  2. Step 2 - Send Email

All 3 receive a change in status. Only the youngest ID'd receives the email (and corresponding activity).


This test just got me thinking -  I did not have a way to control for youngest Marketo record versus lowest ID. For example, if the records with higher ID were younger, would they get the email instead? So I ran this one more time, just with an email send step. Sure enough - smallest ID wins (and others receive nothing).

Screen Shot 2018-01-30 at 10.58.49 AM.png

Hope this helps!

EDIT: The plot thickens. My AE also noted that someone told him...

In the case of duplicate leads, the question of *which* lead gets selected for a  campaign (or gets an activity assigned or updated by a form/list) is best viewed from a user perspective as non deterministic. While often the case, the product does not guarantee lowest id or Last updated etc is the one that is used.

Which is a little concerning and a little interesting. Haha

SanfordWhiteman
Level 10 - Community Moderator

This is clearly a horrible P1 bug which should (if people cared as much as they should) require that the API be disabled for all users until it's fixed.

The only way you can (sort of) sidestep this is by carefully offboarding everyone from your instance, tombstoning their records in your own database.

Dan_Stevens_
Level 10 - Champion Alumni

OMG - you're not kidding about the plot thickening.  I agree with Sanford, this is a serious bug that I bet 99% of Marketo's customers aren't even aware of.  I'm not even sure where to escalate this.  If I open a support ticket, I'm just going to be told that this is expected behavior and to create an idea to have this "new feature" implemented. 

Copying some folks on the product/support team to see if they can point us in the right direction:

Mike Reynolds

John Clark

Kenneth Law

Roxann McGlumphy

Dietrich Struble

Roxann_McGlump1
Community Manager

Hi Dan,

Mike, Ken, and I are on the documentation team, not the product team. John and Dietrich are support, but this is really an issue for the product team. Justin Cooperman​, is there someone in Product who could comment on this?

Dan_Stevens_
Level 10 - Champion Alumni

Thanks for clarifying, Roxann.

Dan_Stevens_
Level 10 - Champion Alumni

Justin Cooperman - can you help us identify a member on the product team who can address this? 

Niranjan_Kumbi_
Marketo Employee

Responding on behalf of Product team.

Recognize the behavior is a bit confusing around the way master record is picked for email send. We'll investigate it internally to see if there needs an enhancement to bode well with SFDC sync updates. Is there a support ticket already?

Dan_Stevens_
Level 10 - Champion Alumni

Niranjan - it's not only an issue around how Marketo selects the smallest leadID as the active record (and therefore, we're to never know which lead record is used in our campaigns), but the fact that Marketo is assigning new/different leads to the same leadID held by a prior lead (that may have been deleted).

I'm not following the connection between CRM sync and where this may be an issue.  The lead record in Marketo should always contain the unique GUID to the CRM lead/contact record.  Now if leadID is used on the CRM side - which I suspect it is - then yes, this could be another issue we're facing.

There is no support ticket.

SanfordWhiteman
Level 10 - Community Moderator

As Dan says, this isn't about picking the master record when deduping on Email Address (or a custom compound dedupe key). Even if it the current master were elected at random that wouldn't be the worst thing in the world.

It's about this quote from Support:

a lead ID can be reused if a lead is deleted and then purged from the instance.

This is far and away the most fatal and mind-boggling technical detail -- if indeed it's true -- I've heard about w/Marketo, ever.

If this is true and is considered a WONTFIX, the entire API is broken. And, except bespoke in-house apps with a special tombstoning mechanism, every app that consumes the API is implicitly broken as well.

We need clarity on this, and clarity on future direction.