How do custom dupe rules work
If all dupe fields match, the existing record is updated, otherwise a new record is created. However, if email address is blank, a new record will always be created.
Custom Dupe Field | Result | |
Smith | New record | |
Jones | New record | |
Jones | Update record | |
New record | ||
Jones | New record | |
Smith | New record | |
Smith | New record |
So the matching is pretty strict, and only when both fields are exactly the same, the record is updated. A blank value for the Custom Dupe rule is also a value, and is therefore often the cause of duplicates.
If the email address is blank, Marketo will always create a new record. This is true with or without a custom dupe rule. The only way to update a record without email address is via the API, while specifying the Marketo ID or Foreign System Person ID (see below).
Keep in mind that Custom Dupe Rules usually apply to all input methods: API, list import and forms. So all forms need to provide all keys also. As an example, your unsubscribe page needs to include the Last Name (pre-filled, possible hidden), otherwise it will create new Leads when people unsubscribe (Last Name is empty and does therefore not match the Last Name on file).
If the dupe rule is by Lead Partition, there is a specific caveat: Lead Partition is not an actual field. For forms and List uploads, it uses the primary Lead Partition of the Workspace for matching purposes. That is usually the desired behavior. However, when using the API, you’ll need to specify the Workspace, which is only possible in some API calls, but not all of them.
Advanced configuration
There are a couple of advanced configuration options available for custom dupe rules. First of all, it is possible to enable them separately for API, list import and form submission. For example, if form submission does not have the custom dupe rule, it will simply match by email address, but list import and API would use the custom dupe rules. This is not a common setup, but it may be useful for some situations.
It is also possible to add more than 1 Custom Dupe Rule, using 3 or more fields. This is also uncommon, and you run the risk that a lot of unintentional duplicates are being created.
The SOAP API also offers some options to create duplicates. The most commonly used option is the Foreign System Person ID, or FSPID in short. This is an external ID that can be used instead of the email address. If you upsert a record with FSPID as the unique key, it will match on FSPID. On match it will update, otherwise it will create a new record. It is important to note that the FSPID can only be set via the SOAP API, and it is recommended to push forms through the SOAP API, as well as List imports, otherwise those Leads won’t have an FSPID.
Process to set up Custom Dupe Rules
Custom dupe fields can be set up by filing a ticket with Marketo Support. It is generally recommended to enlist the help of a Marketo consultant for more complex setups, otherwise you run the risk of unintentionally creating a lot of duplicates.
Conclusion
Although Custom Dupe Rules can be very useful in specific situations when the behavior is well understood, they often lead to the creation of large numbers of unintentional duplicates. It is recommended to carefully evaluate alternatives before implementing custom dupe rules and – if possible – enlist the help of a Marketo consultant to review the options.
.png)