Marketo Webhooks With Marketo API

[Jon_Wu]

Hi Sanford,

That's great advice. I almost had something like that working with an old localhost trick on the Marketo end, but I think that stopped working way back and it seemed error-prone as if we were being too sneaky. I'm glad you mentioned multiple cookies since I had this concern as well.

If I'm understanding correctly, you'd have a long string lead field that could hold multiple cookies and you can just delimit them by something and accumulate them in the Salesforce lead. Then, when the lead is created or the field changes in Marketo, you'd use a webhook from Marketo to call a custom API that we could build, that would call the REST API (http://developers.marketo.com/rest-api/endpoint-reference/lead-database-endpoint-reference/#!/Leads/...​) for each cookie value seen in the lead. Is that the right idea? Then, you could remove all those cookies from the Salesforce lead so the field doesn't fill up and we know that those cookie values have been associated.

If I'm on the right track so far, what happens in these 2 cases?

1. Visitors come back to the site on a new device, but we want to associate their tracking cookie

If we go through the whole process again and wait for the cookie value to change, we end up using 1 API call for every new visitor. Is this would you had in mind?

It seems like we could get around this by having our Salesforce lead creation API let us know if the lead was already processed, synced, and associated into Marketo, so we'd know it'd be safe to call associateLead from Munchkin without creating a duplicate. This still uses 1 Salesforce API call per visit, but there's not a big way around that, and adding capacity there is fairly cheap. However, I wanted to see if you had other ideas.

2. What happens if somebody is filling out a Marketo form? It seems like there's no way to avoid them getting associated and created in Marketo immediately, unless I basically don't use Marketo forms.

We just started using Marketo forms and it's causing a lot of headache with sync conflicts. However, unless we don't actually use a Marketo form eve,  and just build our own forms and APIs that insert into Salesforce directly, I wasn't sure how to avoid creating leads in Marketo and Salesforce at the same time. In this case we also create the Salesforce lead immediately because we load a promotion into their Salesforce / e-commerce account once they fill out the form.

Thanks so much for your help!

Jon

[SanfordWhiteman]

That's right, Jon, you have the idea. I use a JSON array because I'm like that, but could be a semicolon-delimited list (since these are cookies, you can't have a delimiter collision w/semicolons).

Then I put the already-processed cookies in a history field so they don't run through again.

Like you, I'm not convinced the loopback form post is here to stay. Not that it's deprecated exactly, but that it's not documented enough internally and that it could go away due to network/DNS changes, etc.  But you can do the same thing (and ensure that it'll still work) by bouncing off a reverse proxy and back in. Again, if you wanted to go that route, which does have a certain appeal since it doesn't use API calls.

To the 2 cases you mention:

Jon Wu wrote:

1. Visitors come back to the site on a new device, but we want to associate their tracking cookie

If we go through the whole process again and wait for the cookie value to change, we end up using 1 API call for every new visitor. Is this would you had in mind?

It seems like we could get around this by having our Salesforce lead creation API let us know if the lead was already processed, synced, and associated into Marketo, so we'd know it'd be safe to call associateLead from Munchkin without creating a duplicate. This still uses 1 Salesforce API call per visit, but there's not a big way around that, and adding capacity there is fairly cheap. However, I wanted to see if you had other ideas.

Like you said, don't see a way around this without using at least one SFDC API call, since that's the only way you can then save on Mkto calls. Check to see if the lead has made the roundtrip into Marketo and back to SFDC.  Otherwise, pass the cookie again over to Marketo.

2. What happens if somebody is filling out a Marketo form? It seems like there's no way to avoid them getting associated and created in Marketo immediately, unless I basically don't use Marketo forms.

We just started using Marketo forms and it's causing a lot of headache with sync conflicts. However, unless we don't actually use a Marketo form eve, and just build our own forms and APIs that insert into Salesforce directly, I wasn't sure how to avoid creating leads in Marketo and Salesforce at the same time. In this case we also create the Salesforce lead immediately because we load a promotion into their Salesforce / e-commerce account once they fill out the form.

You can use a Marketo form merely as a transport mechanism. That is, you pass a random GUID as the Email and put the real Email (along with everything else) in custom fields. Then a webhook calls the SFDC API to create the actual lead first over there, and you delete the temporary lead in Marketo.  It's... messy but works.

[Jon_Wu]

Wow super awesome advice! It's so refreshing to hear these creative solutions, and so quickly. I've spent days or weeks in the biggest chains with support and Marketo engineers without getting anywhere (other than our current brute force deduplication) for years, so thank you thank you thank you! I like the idea of using a JSON array too as delimiters seem pretty brittle and inexact

The only thing we can do with the loopback method is to post a form, right? Basically we can only update fields or pass in a Munchkin cookie via _mkto_trk, but we can't merge things. If so, I like this idea for saving API calls on lead updates and we could easily set up a reverse proxy, so I think I'll look into that in case they block the instance hostname (which if I understand right, is now required instead of localhost).

That form hack is pretty darn crazy, but it definitely sounds like it'd work. The bummer there is that it makes form logic very convoluted (and hard to debug for eng and marketing), and makes direct form events useless for things like lead scoring, or triggering an email. I could imagine that we store the name of the form in a custom field (perhaps as a JSON array in case there are multiple forms submitted quickly), send that to the webhook, have Salesforce write that to a lead, and have a separate trigger that looks for that custom form name field to be on a new lead, or updated on an existing lead, but whew that's messy. You sound pretty familiar with the ins and outs of Marketo though, so if that's your suggestion, I trust the advice much more just anybody getting that creative.

Overall, are you basically saying that we should never create any lead in Marketo and funnel everything through Salesforce?

It seems like this implies that we can't import lists into Marketo either - at least not without making sure all the leads are in Salesforce AND synced back to Marketo already. Is that the idea?

If so, this seems like a pretty dramatic change for us, but I could see it working out. Using the forms API instead of the REST or SOAP API to associate the cookies after the leads are synced from Salesforce to Marketo seems like one of the key tricks here.

Without that trick, it seems like we'd need to call the REST API once per pre-Salesforce-to-Marketo-sync session, and would therefore have a limit of creating 10k new leads per day (or fewer with multiple sessions), which isn't enough for us.

It seems like it's impossible to use the API for anything that might apply to many leads at scale, which seems to indicate that we can only use it for reporting or occasional things. This a quite a bummer since with Salesforce you can add 10k API calls for ~$20 / month, and use bulk API calls to do 50k things at once, whereas with the Marketo REST API, it seems like many operations are 1:1 and this person said it costs $15k / year to simply add 10k API calls. If I'm getting that right, that's madness!

Anyways, please let me know if I'm saying sane things, and thanks so much for your help!

[John_M]

Actually, you cannot do this as far as I'm aware. Specifically, you cannot call a specific instance of Marketo using SOAP or REST using the webhook in that same instance.

[SanfordWhiteman]

I've heard (via a friend) that Professional Services had a secret recipe for the same instance. But you could just pass through a dumb reverse proxy in either case, so that's not the problem. The problem is still gonna be managing it all using only Flows... I'm sure you'll agree.

[Elliott_Lowe1]

A Marketo Professional Services consultant did this with me a couple of years ago, but I haven't had a chance to implement this yet.

[John_M]

Im in professional services on the enterprise team, and my co workers and I are unaware of his currently being possible. You can try it out simply enough. Youll get a timeout.

[John_M]

If you Do get it working, I'll be happy to be prove wrong! Be sure to post how you did it!

[Anonymous]

Hi John,

Is it possible to exchange lead information between two separate Marketo instances using webhooks and APIs?

thanks,

Danny

[Anonymous]

Up until very recently (like, middle of last week) I was actually using exactly this structure and it was flawless.

We get custom events from our SaaS platform sent via Segment.io, and they show up in our lead activity log as "visited web page: domain.com/events/Name-of-Custom-Event". I then used a smart campaign triggered by the visit to that web address to call a webhook which submitted form in our instance using a POST request. So the lead activity record would subsequently have a corresponding "filled out form: capture - name of custom event" line item.

We did this in order to store events longer term, as web page visits are archived after 90 days. And of course we could still filter via date, number of occurrences, etc as needed. Then starting last week (with no warning or heads up from Marketo, I might add) they disabled this as an option. Now of course we're scrambling to find an alternative solution.

Danny Essner​ I believe it may be possible to pass data between instances still using only webhooks - You'd need to get the app id and munchkin id for the Marketo instance you want to send data into, but I've done this in the past (please let me know if you'd like more info).