Odd question but I am wondering if its possible to use a marketo webhook to call and return data with either the soap or rest api for the same instance in marketo.
I have a unique use case where I want to merge two leads but need to do it in a smart campaign and I'm wondering if I can do that all contained within marketo or if I will need to use an external script to accomplish this.
If anyone has code examples that would be super as well I'm not very proficient with api syntax and webhooks.
Hi Josh thanks for your comment.
I am creating a contact record in salesforce via the rest api and using the data from a new lead in marketo to generate it. Once the interval sync occurs that will cause there to be two leads in marketo with the exact same information as marketo doesn't allow me to force a relationship between those records at the point of creation.
So I need a programmatic way to cause a merge of the original marketo lead that triggered the contact creation webhook and the new marketo lead that was generated from the interval sync so I don't have duplicate records in marketo. Does that help clarify? Our company creates contacts first in salesforce which is confusing I know but this would be a workaround that would allow that to happen.
Thanks,
Dave.
David, what are the conditions that prohibit you from waiting < 5m for the Marketo lead to sync to SFDC?
Not to say I haven't seen a similar situation -- in fact, we are dealing with one now. But that's because we have a form creating users in SFDC and we can't switch that form to be a Mkto form.
I'd like to know the particulars of your situation because there may be a better approach.
While it is possible, after a fashion, to call Mkto APIs from Marketo 'hooks, it is clumsy, not particularly scalable, and if the flows are not managed tightly will lead to infinite loops.
Hey Sanford,
We have a similar problem and I'm interested in how you dealt with this.
We are actually using a webhook to get sessionId from SFDC SOAP, and then a second webhook to create a contact. SFDC then deals with attaching to account. (i know this isn't the best way, but I currently don't have the time/budget or approval to create an intermediary).
Problem is, Mkto creates a duplicate lead as soon as the contact appears in SFDC.
I have no access to SFDC ID in Mkto and can't see any other way to prevent the dupes.
Thoughts?
You'll have to give more of an explanation for why you're in this predicament.
You say you don't have access to the SFDC ID -- is this because the Marketo lead in whose context you're running the webhook isn't the same person you're trying to create in SDFC?
The Marketo contact is the same person that I am creating in SFDC
I don't have access to the SFDC ID in a flow step or as an available field to map a Webhook response to.
Shortly after I use Webhook to create contact in SFDC Maketo creates dupe record.
I'm kind of at a loss as to how to prevent this from happening. My org doesn't use lead records in SFDC, we've investigated creating leads and converting, but that creates new account and opportunity.
My org doesn't use lead records in SFDC, we've investigated creating leads and converting, but that creates new account and opportunity.
You can use Apex to short-circuit the Account + Oppo. I think that's the way to go. Easier all around.
Otherwise you would have to run a merge of the initial Marketo Lead and the SFDC-created Marketo Contact, which is going to be more fragile.
Sanford, I have this exact issue, and with some help from Marketo, we solved this in a rather brute force way. I'm wondering if you have any more efficient or reliable ways to handle this.
We create leads in Salesforce and Marketo simultaneously in many cases from our e-commerce site. Any time you enter your email on our site (usually not in a Marketo form), we add you to Salesforce and our other account systems for CRM and tracking, and call associateLead in Munchkin so that all the user's activity is associated with them immediately in all systems.
Of course, since Marketo doesn't deduplicate Salesforce synced leads, this will result in a duplicate within 5 minutes when the Salesforce lead is synced back to Marketo.
To ditch the duplicate as quickly as possible to avoid conflicts in behavioral scoring or unsubscribe preferences, or other fields, we use a Smart Campaign to fire a webhook to our own endpoint, which deduplicates the lead by finding all leads with the same email, and merging them all into the Salesforce one. This in theory works decently (not really, see below), but it still uses 2 API calls and isn't ideal from that perspective, because we'll hit API limits if we suddenly appear on national TV or have some unexpectedly large event, but perhaps the limits issue is just unavoidable with the way Marketo limits work.
The worst part is, this doesn't actually work reliably due to data consistency bugs on Marketo's side. When we first implemented this, our find dupes, then merge behavior wasn't working consistently. After talking with support, we determined that the API to find leads didn't consistently return all the leads with the same email address if the leads were recently created. To get around this, we fire a deduplication webhook as normal, but then wait 3 minutes and do it again, in case the first one didn't work.
In rarer cases, a lead may appear in Salesforce, but not appear in Marketo, so we also have to deduplicate all new leads that appear in Marketo, that aren't from Salesforce. Again, we have the same consistency bug with the API not returning all leads, so we try once, wait 3 minutes, then try again.
In all, this means every lead that gets created results in 4 API calls to find duplicates (1 for the lead starting in Marketo, 1 for the Salesforce lead synced into Marketo, and double those for retries), and 1 API call to merge them. It's been working fairly reliably (except when we hit API concurrency limits), but the number of API calls is way too many.
While we could always wait 3 minutes, this doesn't bode well for users on multiple devices doing things really quickly in real time. The longer a lead is a duplicate, the more behavioral and scoring data will be lost during the merge.
There's got to be a better way... Any suggestions?
Jon, there's certainly a more efficient way! You're in a not-rare -- not to say "common," I guess -- situation, where the lead is known to SFDC first, but you want to associate the Munchkin session. This happens with Web-2-Lead and 3rd-party forms and other environments as well.
The answer is: don't call associateLead from the client (nor from your server). Let the sync create the person in Marketo. But include on the lead a field that stores the Munchkin cookie value (_mkto_trk). When the lead is created or when this field changes, use a webhook from Marketo to associate the cookie with the lead. Takes one API call total.
We actually have a field that holds multiple Munchkin cookies (in case the person goes in from more than one device before the next sync, unlikely but possible). But you can work with just one at a time in most cases.
Hi Sanford,
That's great advice. I almost had something like that working with an old localhost trick on the Marketo end, but I think that stopped working way back and it seemed error-prone as if we were being too sneaky. I'm glad you mentioned multiple cookies since I had this concern as well.
If I'm understanding correctly, you'd have a long string lead field that could hold multiple cookies and you can just delimit them by something and accumulate them in the Salesforce lead. Then, when the lead is created or the field changes in Marketo, you'd use a webhook from Marketo to call a custom API that we could build, that would call the REST API (http://developers.marketo.com/rest-api/endpoint-reference/lead-database-endpoint-reference/#!/Leads/associateLeadUsingPO…) for each cookie value seen in the lead. Is that the right idea? Then, you could remove all those cookies from the Salesforce lead so the field doesn't fill up and we know that those cookie values have been associated.
If I'm on the right track so far, what happens in these 2 cases?
1. Visitors come back to the site on a new device, but we want to associate their tracking cookie
If we go through the whole process again and wait for the cookie value to change, we end up using 1 API call for every new visitor. Is this would you had in mind?
It seems like we could get around this by having our Salesforce lead creation API let us know if the lead was already processed, synced, and associated into Marketo, so we'd know it'd be safe to call associateLead from Munchkin without creating a duplicate. This still uses 1 Salesforce API call per visit, but there's not a big way around that, and adding capacity there is fairly cheap. However, I wanted to see if you had other ideas.
2. What happens if somebody is filling out a Marketo form? It seems like there's no way to avoid them getting associated and created in Marketo immediately, unless I basically don't use Marketo forms.
We just started using Marketo forms and it's causing a lot of headache with sync conflicts. However, unless we don't actually use a Marketo form eve, and just build our own forms and APIs that insert into Salesforce directly, I wasn't sure how to avoid creating leads in Marketo and Salesforce at the same time. In this case we also create the Salesforce lead immediately because we load a promotion into their Salesforce / e-commerce account once they fill out the form.
Thanks so much for your help!
Jon
That's right, Jon, you have the idea. I use a JSON array because I'm like that, but could be a semicolon-delimited list (since these are cookies, you can't have a delimiter collision w/semicolons).
Then I put the already-processed cookies in a history field so they don't run through again.
Like you, I'm not convinced the loopback form post is here to stay. Not that it's deprecated exactly, but that it's not documented enough internally and that it could go away due to network/DNS changes, etc. But you can do the same thing (and ensure that it'll still work) by bouncing off a reverse proxy and back in. Again, if you wanted to go that route, which does have a certain appeal since it doesn't use API calls.
To the 2 cases you mention:
Jon Wu wrote:
1. Visitors come back to the site on a new device, but we want to associate their tracking cookie
If we go through the whole process again and wait for the cookie value to change, we end up using 1 API call for every new visitor. Is this would you had in mind?
It seems like we could get around this by having our Salesforce lead creation API let us know if the lead was already processed, synced, and associated into Marketo, so we'd know it'd be safe to call associateLead from Munchkin without creating a duplicate. This still uses 1 Salesforce API call per visit, but there's not a big way around that, and adding capacity there is fairly cheap. However, I wanted to see if you had other ideas.
Like you said, don't see a way around this without using at least one SFDC API call, since that's the only way you can then save on Mkto calls. Check to see if the lead has made the roundtrip into Marketo and back to SFDC. Otherwise, pass the cookie again over to Marketo.
2. What happens if somebody is filling out a Marketo form? It seems like there's no way to avoid them getting associated and created in Marketo immediately, unless I basically don't use Marketo forms.
We just started using Marketo forms and it's causing a lot of headache with sync conflicts. However, unless we don't actually use a Marketo form eve, and just build our own forms and APIs that insert into Salesforce directly, I wasn't sure how to avoid creating leads in Marketo and Salesforce at the same time. In this case we also create the Salesforce lead immediately because we load a promotion into their Salesforce / e-commerce account once they fill out the form.
You can use a Marketo form merely as a transport mechanism. That is, you pass a random GUID as the Email and put the real Email (along with everything else) in custom fields. Then a webhook calls the SFDC API to create the actual lead first over there, and you delete the temporary lead in Marketo. It's... messy but works.
Wow super awesome advice! It's so refreshing to hear these creative solutions, and so quickly. I've spent days or weeks in the biggest chains with support and Marketo engineers without getting anywhere (other than our current brute force deduplication) for years, so thank you thank you thank you! I like the idea of using a JSON array too as delimiters seem pretty brittle and inexact
The only thing we can do with the loopback method is to post a form, right? Basically we can only update fields or pass in a Munchkin cookie via _mkto_trk, but we can't merge things. If so, I like this idea for saving API calls on lead updates and we could easily set up a reverse proxy, so I think I'll look into that in case they block the instance hostname (which if I understand right, is now required instead of localhost).
That form hack is pretty darn crazy, but it definitely sounds like it'd work. The bummer there is that it makes form logic very convoluted (and hard to debug for eng and marketing), and makes direct form events useless for things like lead scoring, or triggering an email. I could imagine that we store the name of the form in a custom field (perhaps as a JSON array in case there are multiple forms submitted quickly), send that to the webhook, have Salesforce write that to a lead, and have a separate trigger that looks for that custom form name field to be on a new lead, or updated on an existing lead, but whew that's messy. You sound pretty familiar with the ins and outs of Marketo though, so if that's your suggestion, I trust the advice much more just anybody getting that creative.
Overall, are you basically saying that we should never create any lead in Marketo and funnel everything through Salesforce?
It seems like this implies that we can't import lists into Marketo either - at least not without making sure all the leads are in Salesforce AND synced back to Marketo already. Is that the idea?
If so, this seems like a pretty dramatic change for us, but I could see it working out. Using the forms API instead of the REST or SOAP API to associate the cookies after the leads are synced from Salesforce to Marketo seems like one of the key tricks here.
Without that trick, it seems like we'd need to call the REST API once per pre-Salesforce-to-Marketo-sync session, and would therefore have a limit of creating 10k new leads per day (or fewer with multiple sessions), which isn't enough for us.
It seems like it's impossible to use the API for anything that might apply to many leads at scale, which seems to indicate that we can only use it for reporting or occasional things. This a quite a bummer since with Salesforce you can add 10k API calls for ~$20 / month, and use bulk API calls to do 50k things at once, whereas with the Marketo REST API, it seems like many operations are 1:1 and this person said it costs $15k / year to simply add 10k API calls. If I'm getting that right, that's madness!
Anyways, please let me know if I'm saying sane things, and thanks so much for your help!
Actually, you cannot do this as far as I'm aware. Specifically, you cannot call a specific instance of Marketo using SOAP or REST using the webhook in that same instance.
I've heard (via a friend) that Professional Services had a secret recipe for the same instance. But you could just pass through a dumb reverse proxy in either case, so that's not the problem. The problem is still gonna be managing it all using only Flows... I'm sure you'll agree.
A Marketo Professional Services consultant did this with me a couple of years ago, but I haven't had a chance to implement this yet.
Im in professional services on the enterprise team, and my co workers and I are unaware of his currently being possible. You can try it out simply enough. Youll get a timeout.
If you Do get it working, I'll be happy to be prove wrong! Be sure to post how you did it!
Hi John,
Is it possible to exchange lead information between two separate Marketo instances using webhooks and APIs?
thanks,
Danny
Up until very recently (like, middle of last week) I was actually using exactly this structure and it was flawless.
We get custom events from our SaaS platform sent via Segment.io, and they show up in our lead activity log as "visited web page: domain.com/events/Name-of-Custom-Event". I then used a smart campaign triggered by the visit to that web address to call a webhook which submitted form in our instance using a POST request. So the lead activity record would subsequently have a corresponding "filled out form: capture - name of custom event" line item.
We did this in order to store events longer term, as web page visits are archived after 90 days. And of course we could still filter via date, number of occurrences, etc as needed. Then starting last week (with no warning or heads up from Marketo, I might add) they disabled this as an option. Now of course we're scrambling to find an alternative solution.
Danny Essner I believe it may be possible to pass data between instances still using only webhooks - You'd need to get the app id and munchkin id for the Marketo instance you want to send data into, but I've done this in the past (please let me know if you'd like more info).
I don't understand what you're saying stopped working. Unless there's something buggy in your Marketo instance in particular, a loopback form post from a Marketo webhook to the same instance still works just fine.
This isn't, of course, actually using an API call (that was the open question). You're just using the standard forms endpoint (http://localhost/index.php/leadCapture/save) not an API. And, like I said, still works.
The response I got from Marketo Support:
I had a chance to speak with one of the Senior Engineers and was informed that using webhooks to POST into a Marketo Form in this fashion is no longer supported. Using a webhook in this fashion has the potential to create an infinite loop of activity and can become resource intensive.
So either it's truly no longer supported to webhook to your own instance, or they disabled it for us. Realistically I can think of a couple of ways to take an infinite loop of activity within Marketo without involving a webhook so I'm not sure why that would be a concern. I realize that it's an unsupported feature but "unsupported" shouldn't mean "kill it at random with no warning" which is what happened in our case. My ticket is still open though so maybe I'll hear more from them.
So either it's truly no longer supported to webhook to your own instance, or they disabled it for us.
That's overly punitive, if so. Almost bizarre given that such webhooks have not only been used in the wild for a while, but were endorsed in a blog post from 2 months ago that's still up! I know I was at a client last month and saw one that someone had implemented in the past. I just take it as a matter of course that people still do this, even if I don't generally use it.
Realistically I can think of a couple of ways to take an infinite loop of activity within Marketo without involving a webhook so I'm not sure why that would be a concern.
I agree, I find that justification unconvincing. But more important, all that's being addressed (if it has been disabled for your instance, it hasn't for us) is a webhook that literally uses the loopback address to post without leaving the Marketo network. This says nothing about leaving the network and echoing the request back in, which would be extremely easy to set up. In 5 minutes, you could put up a reverse proxy that does just that. You can PM me about it if you want.
Also, is it possible that there was something in your implementation that never should've worked in the first place? I ask because you weren't using localhost in the the snippet you posted in another thread. In this case, maybe you're seeing a coincidence: Marketo locked down communication from inside their network to the external IP of your pod/instance and thus broke your webhook, but it would still work with actual loopback. Just an idea.
Speaking of ideas, though, why not use an Interesting Moment for this? Or a datetime field on the lead LastCustomEvent{EventName}?
Ok, quick update (I know David Horne found a different solution but this may help other people in the same situation. Big thanks to Sanford Whiteman for walking through this with me.
The following structure for a webhook appears to work (at least for now):
URL: set to http://localhost/index.php/leadCapture/save
Request type: POST
Template: include required parameters (munchkinId, formid) and any values or tokens for other fields that you want to include in the POST
Encoding: set to Form/URL
We wound up taking another route that was more scalable. Thanks everyone for your help. After consulting with Marketo experts we found out that webhooks do not operate in a batch function so large imports would take too long.
I believe there is an example on RingLead's site if you look for their webhook guide.
What precisely do you want to do? Marketo dedupes emails before it sends any emails out. What you want may already exist with Ringlead or another tool.