Velocity Loading with Duplicate Records

[Wyatt_Bales4]

As we continue to try and roll out Related Contact functionality, we still have an issue with duplicates in Marketo--mostly leads and contacts with matching email addresses. Normally for basic batch campaigns this isn't a big concern (besides some attribution and reporting issues); however, for email scripting it's causing a bit of a headache.

Problem: Email scripts inconsistently load (in preview, sample, nor actual send) when there is more than one record in the database with a matching email address.

Possible Culprit: When Marketo tries to understand which record to qualify in a duplicate scenario, batch campaigns qualify the first created record while trigger campaigns qualify the last updated record. I believe Velocity pulls in the most recently updated record which if doesn't match with what the smart campaign brought in, the script won't load.

I understand there are multiple ways of ridding duplicates from Marketo but focusing on this issue, I'm trying to understand if there's anything which can be done inside the email script itself.

Curious how Sanford Whiteman​ or other savvy email scriptors compensate for duplicates while using email scripting.

[Grégoire_Miche2]

Hi Wyatt,

Can you please provide some more information about the data schema you are using (what custom objet) and also the velocity code you have developed ?

-Greg

[Wyatt_Bales4]

Thanks for the response, Greg. We use various custom objects ranging from transactional nature (e.g. order history) to a reference table (e.g. onboarding status). The dedupe fields are usually a custom row ID with the link field either being email address or SFDC ID.

As for the scripts, some are #foreach loops while the majority are just a simple $object.attribute reference. All mimic the same behavior regardless of the schema or scripting logic.

[SanfordWhiteman]

Velocity runs in the context of the same lead that receives the email (you can verify this by outputting ${mktmail.id}, which is the Marketo ID).  So I'm not sure where you're seeing crossover -- there shouldn't be any way for lead X to have access to lead Y's attached objects, even if you wanted it to! Can you describe what you mean by inconsistently? Is it that you want to see data across leads?

[Wyatt_Bales4]

Thanks for the response, Sanford. It's not a matter of crossover but a matter of not populating when there's a duplicate.

For example, if I have a simple $object.attribute in an email pulling from a Custom Object, it will not display any value in the email if there is a duplicate record in the database (I'll explain more on how this isn't 100% of the time as it's dependent on which record was created first).

Because the smart campaign could be pulling in the Lead duplicate when in fact it's the Contact which has the Custom Object (which is confusing considering you're using a custom object smart list filter), the email script will render null because it doesn't see the Custom Object relationship. This has to due with how Marketo smart campaigns process duplicate records. Even though you provide very specific Contact-related smart list filters, the duplicate Lead can still qualify because it has a matching Email Address and was created first.

A simple test would be to create a lead in Marketo, sync over a duplicate Contact, and within an email script, reference a Custom Object field which is tied to the Contact. Running a batch campaign will qualify the first created record (i.e. the Lead) and thus the email script won't see the Custom Object.

It's the only common theme across the records which all pass through the same token. Support has confirmed duplicate records are the root cause but I'm still trying to research how to mitigate Velocity from showing null. I understand the root cause is qualification (even before it gets to Velocity) so I guess this question is more focused on smart campaign qualification/attribution than Velocity.

[SanfordWhiteman]

Right, as you say in your last paragraph, this isn't a Velocity issue.  It can be frustrating to try to build SL qualifications by CO (another example is when you add multiple CO filters and they don't apply to the same exact CO record).  Reducing duplicates is the key, or at least making sure only Lead or Contact, but not both, can ever have COs.

[Wyatt_Bales4]

Thanks, Sanford Whiteman​. Considering how prevalent duplicates are, I'm surprised more customers are not complaining about email scripts not loading values. Even if you reference normal fields on the Lead record, duplicates will cause the script to not load. We send over login details by concatenating different fields together and if there is already a record with a matching email address, there's no guarantee it will load the right details (e.g. we use an Account email address field for Restaurants to login but if the Lead is pulled in, the script won't load).

[SanfordWhiteman]

"Scripts won't load" doesn't seem like the right way to put it, they'll load whatever they can for the lead that Marketo chooses to qualify.

[Wyatt_Bales4]

"Incorrect smart campaign qualification due to duplicate records causing Email Script to load incorrect values."

Understood. I'll try to work on phrasing it's just unfortunately Velocity is the only "visible" indicator the wrong record has qualified.

[Wyatt_Bales4]

Justin Cooperman​ are you aware of any tricks to make smart campaigns ONLY pull in the record with the matching criteria and not its duplicate? (e.g. If there's a Contact and Lead with the same email address but only the Contact has a Custom Object relationship, make sure the smart campaign pulls in the Contact for the email script to reference)

Appreciate any Product insights you may have outside of enabling Purposeful Duplicates for the instance.