Webhook to create purposeful duplicates in Marketo is unreliable

Highlighted
Level 10 - Champion Alumni

Re: Webhook to create purposeful duplicates in Marketo is unreliable

I hear you, Trish - very frustrating.  It almost sounds like a cop-out reply since they can't figure out why this is happening.  I'm hoping Kristen or John can chime in here.  If anything, this is the most basic type of webhook - nothing fancy here.  I replied back to the support ticket asking what they mean by "manage duplicate creation on the front end".

Paul Wilson - do you use this approach with any of your customers?

Highlighted

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Hi Dan - Kristen left Marketo last year, unfortunately. I agree that this just isn't a good enough answer. Can you ask that it be escalated further? (Thank you!)

Highlighted
Level 10 - Champion Alumni

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Yeah, I know - she's now at BlueWolf.  But she's still active on the community and has an active profile.  She published a series of blog posts recently here.

Highlighted
Level 4

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Quick question for you Dan Stevens do you have duplicate detection rules active in DCRM?

Highlighted
Level 10 - Champion Alumni

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Hi Paul.  We do.  Very basic setup (notification pops up if a duplicate - based on email address - is created per entity (lead/contact).  When dupes are created purposefully in Marketo, they are not synced back to CRM.

Highlighted
Level 4

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Good day Dan Stevens​ I wonder if there might be a "behind the scenes" issue at play here. A couple of weeks ago I learend the following from Marketo support:

"When Marketo attempts to pass either an insertion or an update into the CRM through the API, we adhere to the deduplication rules in both cases and this can cause the API request to fail. In Marketo we have the option to suppress deduplication configuration when we make the API request for inserts for Leads, updates on Leads or Contacts, or a combination. We adjusted the configuration in the Marketo instance to suppress deduplication configuration for updates on both the Lead and Contact entities."

So if you add a case comment on your open issue and mention that Praveen had pointed the above out on another case then this may help if the de-dupe rules may be affecting how the API calls are happening.

Hope this helps some!

PW

Highlighted
Level 10 - Champion Alumni

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Thanks for the reply Paul.  We actually learned of this a while ago - as it contributed to several sync issues we were experiencing.  We have since suppressed the dupe detection from Marketo to CRM - just like you did - and that solved the issue.  For this issue, I think Sandy's nailed it - now we just need to figure out the best alternative approach to use that will provide reliable results.

Highlighted
Level 10 - Community Moderator

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Let me explain why something like this would be unreliable unless specifically part of their infrastructure plan (as opposed to a coincidental byproduct of certain infrastructure choices, which is why it seems to work sporadically).

Webhooks originate from servers inside Marketo's network. Those servers may or may not be the same pod that your instance is on.  Before the webhook connects, it resolves the hostname using DNS.  In most organizations, one or more of the following is true:

  • a public hostname (like app-zz.marketo.com) does not resolve internally at all, so the DNS lookup fails
  • a public hostname resolves to a private IP address from inside the network (i.e. using internal DNS servers) but that IP addy doesn't accept inbound connections and/or is a different actual server from the intended destination
  • a public hostname resolves to the usual public IP address, but due to firewall rules that IP address isn't translated to a private IP or isn't accessible from inside. (Some firewalls technically can't make connections that go both in and out of the same internal port, other cases they're manually set up to block such connections.)

The sum of all the above is that you can't literally connect to "app-zz.marketo.com" from servers inside Marketo because of a combination of DNS and TCP/IP routing problems.

But you can try to connect to the same server that's referred to by "app-zz.marketo.com" but using a different name. Let's say the canonical name of that server is DANSERVER and its private addresses are 1.2.3.4 and 127.0.0.1 (the second is the loopback address present on all machines).

We customers wouldn't know the internal address 1.2.3.4 (Marketo wouldn't publish this), and we wouldn't even know the official name is DANSERVER, but we would know that if we were making an outbound connection from DANSERVER to DANSERVER then "localhost" would resolve to 127.0.0.1 and 127.0.0.1 is one of the local IPs. This is knowable because it's an industry standard that's true 99.999% of the time. For example, on your personal PC or Mac, "localhost" points to your machine. Think of it as the universal hostname.

But this only works if you going from a machine, to that same machine.  We know we want this webhook to connect to your pod (that's where a form listener is to make these duplicates). If the webhooks are also going from your pod, then connecting to "http://localhost/index.php/leadCapture/save" is going both from and to that same server, so it should work.

But there's no guarantee that webhooks will be sent from your pod server itself -- nor could there be, if Marketo's architecture is going to scale. It stands to reason that some or all webhooks will be sent from other servers. On those servers, "localhost" is no longer your pod, it's the specialized webhook server itself, which doesn't do form processing. Hence the call would fail, since it would be connecting toi a server that can't do what you want it to do.

Other aspects which can confound connections to localhost are

  • when the local server requires SSL for all connections: unless the server is very deliberately set up to accommodate this scenario, the secure connection will fail, since a public SSL cert can't cover the name "localhost"
  • when the local webserver doesn't bind (a.k.a. attach/listen) to 127.0.0.1 at all, deliberately excluding it, or if the service bound to 127.0.0.1 isn't the one you want.

When Marketo talks about using a proxy, they mean a server outside Marketo that connects back in using your public pod name (and public IP address). In contrast to the localhost connection, this kind of connection will always work, since from Marketo's perspective it's just a form post from some outside address. The syntax of the webhook is exactly the same, it just goes to a special outside hostname. We actually have several such proxies already in place for different pods. If you tell me which pods you guys are on you can just use our proxy.

Highlighted
Level 10 - Champion Alumni

Re: Webhook to create purposeful duplicates in Marketo is unreliable

Thanks for the thorough explanation, Sandy. So much better than getting a "this is not a supported method" like we received from Marketo.

We're on the San Jose pod: app-sj08.marketo.com.  If we use this alternate approach, would this open us up to any data privacy/security issues by using a proxy like this?

i would still like to hear Kristen and John's perspective on this - especially if they're still using this solution with their customers.

Highlighted
Level 10 - Community Moderator

Re: Webhook to create purposeful duplicates in Marketo is unreliable

If we use this alternate approach, would this open us up to any data privacy/security issues by using a proxy like this?

You're quite explicitly passing data out of Marketo via webhook and back in. It's an SSL connection in both directions, though I could be intercepting the data because the connection is terminated on the proxy (of course I'm not, but that may be a concern).

Anyway, you can try:

     https​://marketo-app-sj08.api.figureone.com/index.php/leadCapture/save