Lead DeDupe Options with Marketo Forms

[Anonymous]

Hello Everyone,

      Our company is currently considering the move away from using Salesforce Web-2-Lead forms, to using Marketo web forms (or another form with an API REST integration directly into Marketo). Today, we currently have a Salesforce plug-in that handles our deduplication rules in Salesforce.com when it receives a new lead.

The issue we are coming up against, is that Marketo only looks at an email address for deduplication and has no ability to modify any of the deduplication rules. This has our sales team concerned because in some countries we do business in, it is quite common to use departmental shared email addresses (e.g. sales@abccompany.com or marketing@abccompany.com). The concern is that without the ability to consider other fields, such as first / last name, we will have leads overwriting each others' data in our database.

Unfortunately, this is becoming a make-or-break situation for using Marketo as the primary entry point for our leads, which now reduces our ability to use many features, such as the "filled out form" triggers / filters, progressive profiling, form prefills, etc.

Has anyone else here already come across this and came up with a solution we haven't thought of to get around this? Ultimately, I would like to use Marketo, but it is looking less likely, now.

Thanks!

Jason

[Anonymous]

By the way, I have seen that there are companies such as Leadonance and M-Clean in Marketo's launchpoint site but have no experience with either of these tools. I've seen them mentioned in other Dedup discussion but have not seen anyone who has used them and can recommend one over the other. If anyone has experience with them and could recommend one, I'd also be interested in hearing more.

Thanks

[SanfordWhiteman]

The fundamental problem is that the tools that exist to do exactly what you want -- namely the REST API Push Lead endpoint -- are severely rate-limited and too dangerous to expose via a web form without an intermediate layer of DoS protection. (Witness what happens when you attract a spammer to your form, or a newbie hacker drips a few thousand hits in a half-hour and renders not only this integration, but all your integrations, useless.)  No professional organization can afford to do this in 2017.

Further proof of the direct API model's insufficiency: Marketo forms allow 43,200 form posts per day per source IP to allow legit usage patterns while protecting the system, while the absolute maximum post attempts you can send via REST -- and on newer instances only -- is 50,000 per day, period. Obviously the former limit has been tuned to balance real-world needs and DoS protection, while the latter has not been designed for one-by-one API use from an uncontrolled source.

Additionally underscoring the difference is that the Form Editor is (naturally) geared toward regular Forms 2.0 form submissions. You actually can use Marketo forms (as built in Form Editor by users) and submit them via API, but it requires substantial rejiggering using JS. And it doesn't solve the rate issue. For that, you need to pass your API calls through an additional mitigation layer.

A primitive layer that at least protects your other integrations will cap the API calls at, say, 25% of your total. That'll make sure your other integrations run unhindered (which is indeed vital) but won't stop a malicious actor from easily stopping this one integration from working. 

A more proper approach (which we do using AWS Lambda but could be built using a variety of platforms) is having a per-source-IP rate limit and daily total limit, so you protect your other integrations and also protect against script kiddies who have only a few machines at their disposal. You still can't stop a concerted attack from a large number of IPs, though: that's the thing about only having 50,000 calls to spread around per day.

So ultimately, a solution is possible that allows the Form Editor to continue to be used while allowing you to select your own dedupe key, but you will have to allocate development time.

[Anonymous]

Good analysis, Sanford. Not sure about the other software mentioned, but I’d like to stress, that Deduplicator uses a different approach: Deduplicator is not designed for “one-by-one API use from an uncontrolled source”, we support a list-based approach instead. 

It allows us to work independently from triggers as long as we know which static list to pull leads from for deduplication. We could easily add to that a 15-minute wait period to have Deduplicator query Marketo automatically at given intervals, resulting in an “almost real-time ongoing deduplication”. In my opinion, it  doesn’t have any serious shortcomings but prevent possible DoS attacks.

Besides that, list-based approach allows a customer to set up and store custom lead deduplication and "field-level conflict resolution" scenarios. So you can have a scenario like:

1. Identify duplicates by First name + Last name

2. Pick a winner by creation date

3. Inherit all field values from a winner lead excluding “lead address” – for which store the value from the oldest lead and “First Name”.

Deduplicator allows to do it quite easily.