How to add the duplicate records to blacklist.

[Anonymous]

Hi All,

Scenario:

Original id - 1. srividhya.mm@test.com - Marketo id: 1234567 is set as blacklist true

Duplicate id - 2. srividhya.mm@test.com - Marketo id: 7654321 is not set as blacklist true

When the target list is blasted with "srividhya.mm@test.com - Marketo id: 7654321",the email is actually sent, since marketo recognizes both email id as different due to different marketo id. But logically, its the duplicate email id of the blacklisted lead and marketo should not send the email.

Question:

Is there a way to set duplicate email ids to set blacklist as "true", when  marketo finds one among the email id is blacklist "true".

Please help to overcome this issue and steps involved (default if any available in marketo to set black list true for duplicate id / to achieve this through any smart list / smart campaign )

Thanks in advance!

struerividhya.mm@test.com - Marketo id: 7654321

[Rachel_Noble]

There may be a better way than this, but you could create a smart campaign with the following:

Smart list - Member of Smart List, List = Possible Duplicates

                   Blacklist = False (this will help prevent your campaign queue from getting backed up)

Flow - Change Data Value - Blacklist = True

Then schedule it to run nightly or prior to each email send you have.

[Josh_Hill13]

I would be very careful with that because it may not work as you think it does.

It would be better to use another data tool to manage that issue. or Pull the list and then re-upload the proper dupes with BL=T

[Rachel_Noble]

Also, look into mass deduplication tools (Marketo has one and there's lots of others) and go through this list regularly and clean it out. Clean data is the best.

[Grégoire_Miche2]

Hi Rachel,

What you propose here is a little dangerous, it will black list all your duplicates, no matter what.

If you do not want to use a third party tool, do the following:

Create a smart list "black listed = True". Export it. Open the CSV file, Keep only the email addresses. column. Run a deduplicate in Excel to keep each value only once. Copy (CTRL-C) that list of email

Create a smart campaign:

• Smart list: email address is [paste here the list of email addresses]
• Flow: change data value, attribute: black listed, new value: True

A little cumbersome but will work safely.

As suggested by Josh, you also should look into deduplication tool. Marketo's is a one of. But you may want to look at this one: New Deduplication App for Marketo​

Another solution would be to develop a small web service that could be triggered by a webhook. This web service would receive the email address of the newly blacklisted lead and  automatically black list all other leads in the DB with the same email address.

-Greg

[Grégoire_Miche2]

And also, vote for the following ideas:

-Greg 

[Josh_Hill13]

YES. That's what was in my head, but I didn't have time to write it out.

[Anonymous]

Hi Greg,

Thank you for your time! Yep, for one time blacklisting of records of duplicates, we propose / executed the same "manual export of blacklists and smart campaign" idea you had suggested.

This execution works for static / manual export of blacklisted contacts. Is there anything we can do / fix for the dynamic ones? Meaning, any new duplicate lead that enters marketo, needs to be checked if the email address already exist and has black list value  "true", if yes the new duplicate email id should also be marked as blacklist true automatically, may be using trigger or some functionality in marketo".

Thanks for your suggestions on de-duplication tool and developing web services etc., Just checking if we could achieve / fix this for dynamic list of leads flowing in to Marketo as well using the triggers / campaigns available in Marketo itself.

Do let me know / share if you have any thread / doc available on creating web services that would be triggered by webhook.

Because we are looking for a solution (of blacklisting duplicates) in a global level for an instance and hence doing it manually could not meet the actual request in global level. (though i understand that's the reason we are looking for Durable blacklist).

Please suggest on any further alternatives, if available.

Thanks,

Vidhya.

[Grégoire_Miche2]

Hi Vidhya,

Yes, you could have a system that checks all recent duplicates to verify that the other email with the same email address is blacklisted. This would require some web services, again.

The hard point is to avoid calling the web service or all leads that enter your system. You need to do it for every lead that enters the system and has a duplicate. Detecting this can only be done "post" creation. So use a SC that does the following:

• Trigger: lead is created
• flow
  • Wait 10 minutes (wait until all other lead management, data management, etc...) is done
  • Call webhook, with the following choice:
    • if member of "possible duplicates" -> call the web hook
    • default: do nothing

Then the webhook web service will have to connect back to Marketo and search for all dupes of the new email and, if one of the dupe is black listed, return a True value that Marketo will use to update the new record.

Pay attention to one one point: if you have marge volume, the above architecture may consume a lot of API call. In this case, you may prefer a batch system. This means that, when a new lead comes in, you will have to just add it to a static list, then you will need a web service to connect to Marketo once every hour or every day and process all the leads in the list.

-Greg

[Rachel_Noble]

Sri- Greg's right, I spoke too quickly. We do this with a non-emailable field because we don't email to duplicates for a number of reasons. Might not work out if you want to email to dups in other scenarios. Hope you found a solution!