Jon_Wu

Jo_Pitts1 · ‎02-16-2022

@NadineThomas , There are a number of posts floating around on this topic. In many ways, you are better off adopting the approach I discuss in this post: https://nation.marketo.com/t5/submit-form-api-beta/looking-to-added-contacts-to-marketo-from-a-3rd-party-form/m-p/321718 The reasons why this is a good approach are also listed in that post. ALSO - If your non Marketo forms happen to be Gravity forms, and you are using Wordpress for your site, there is now a plug in that will take the gravity form data, pass it along to a Marketo form and submit it for you - no code required. I'm in the process of trialling said plug in for a couple of clients, and will do a post here as soon as I am comfortable with its reliability and performance. This confers all the benefits I list in the post above, with zero coding burden! Cheers Jo

kh-lschutte · ‎02-06-2020

kh-lschutte · ‎02-06-2020

kh-lschutte · ‎02-06-2020

Jon_Wu · ‎03-08-2019

Anonymous Lead Promotion to Known Lead – Munchkin V2 Behavior outlines that only some certain steps, including Change Score, run when an anonymous lead is promoted / associated. My question is, within a Flow, is there a condition I can add within a choice step to only run Change Score if the events are NOT being replayed? This relates to potential workarounds for How can you decrement scores of (previously) anonymous leads after a wait w/ Munchkin v2?

Jon_Wu · ‎03-12-2019

Darn that's what I figured in terms of Marketo being unhappy with high peaks. We'll have to find a way that doesn't overwhelm it and slow everything down. Or if thousands to 10s of thousands per hour is fine usually, maybe we just suffer on peak days, but that's not ideal. For this particular scoring use case, I don't technically need the scores to be always up to date, but then the problem is that if they aren't, it becomes difficult to know if it's safe to use the latest score or not. For example, one strategy I have now is to use a webhook to update the category of a lead's highest score, but then do it at most once every 2 hours, then wait 3 hours and run the webhook again. This is an approximation of throttling and is much better than firing webhooks on every page view for performance. The hard part is that it makes it more difficult for automation to work since you can't really trust whether the score is up to date, or whether it's a few hours old.

Jon_Wu · ‎03-07-2019

Hi Josh. Not sure I follow. Are you saying that maybe I could prevent the decrement from happening if I could tell that events were being replayed during association? Even if I could prevent it from happening at that moment, I'm not sure how I would actually decrement the score by the appropriate amount in 14 days. Thanks for your help

Jon_Wu · ‎02-28-2019

Hi Sanford. It seems like what I have down gives some balance between adding a little safety and not increasing complexity too much, adding external dependencies, or slowing down performance by more than a couple hundred ms, but I'd love to hear more about your thoughts on compare-and-swap via webhook. First, I think what I have now in the screenshot reduces the race condition window to a reasonable size. There's probably a 50-100 ms window between where we check to remove them from the flow and when we actually get them removed from the list. Not ideal, but I don't expect too much concurrency except when a cron / scheduled Smart Campaign overlaps with triggers. As I'm typing this, another way I could reduce the likeliness of a race condition is to add a wait step before putting people into a cron list, so that the cron is truly more of a fallback and won't compete with triggered campaigns that execute immediately, but that would add yet another step. When new leads are created OR when leads are associated, I try to kick off actions to send emails that were triggered by activity that was logged while the lead was anonymous. The ideal scenario is that I can use a trigger on a server-side form post that I use to associate leads, but if that fails or if I'm associating via REST API, I depend on 24 scheduled batches (running once per hour) to try and flush the queues out as a fallback. If any of those scheduled campaigns run at the same time as the lead association, that's where the race is most likely to happen. The super ugly part with all of this is that I'd have to have our marketing team implement all of these things every time the want to set a triggered email, which seems tedious. If I was using a webhook to do a lock externally, I'd need to set up another service, have a key/value store with tracking for every campaign that's running, and then I think I'd be writing back to a Marketo field with the result by campaign, so I'd need a new lead field for every campaign that I run. Or this kind of what you're thinking with a compare-and-swap via webhook? If so, I think that's probably too much. Worst case if the race condition happens, is somebody gets 2 emails. If that happens 1 in 100,000 times that's probably not a huge deal. While I agree the smart thing to do would be to never use an inequality query, but I was imagining if that were the case, the Smart List Best Practices doc wouldn't need to recommend use positive over negative operators, but this is just speculation of course. Either way, I think these lists will probably be small so Denise's advice to go with more readable solution may be the most practical if I was keeping this strategy.

Jon_Wu · ‎08-04-2017

Yeah who knows, but glad it's working now. Thanks for sharing about the broken pattern matcher bug. It's good to know what kind of things can and have been going wrong, so we know it's a possibility before we go crazy ourselves.

Jon_Wu · ‎08-03-2017

I escalated to our account manager who said somebody would call me yesterday morning, but then instead I got an email saying that they won't support us customizing the URL (or having a fake URL as they put it), despite the docs saying: You can customize the URL and querystring which are sent with the data object in the second argument. They also gave a bogus reason that they filter out stuff for abuse and spam prevention and said stuff has to be on our domain (which it is). This also isn't true since all our random dev domains record actions anyways. Since they seem to be so caught up that me logging page views at /_event/... isn't cool, I just made the URL an actual page, and submitted another screencast to them showing that this still doesn't work, even if it's a real page. Waiting to hear back from support escalations now. Separately, another support rep just told me that the case sounded like something they'd be able to handle, but that I just need to keep hammering escalations until I'm heard.

Jon_Wu · ‎07-10-2017

Exactly what I was thinking in general for cases where we didn't have an event In our specific case, we write info to Salesforce first via our own backend, then that API only returns a hash to the frontend if the lead was already synced from Salesforce to Marketo. This way, new leads have a deferred association through Salesforce to prevent duplicate creation, but existing leads can be associated before events we want to take action on after email collection, are fired on the frontend.

Jon_Wu · ‎07-06-2017

I've confirmed that the flow choices are historical as you mentioned, which means my above strategies won't work. The best seems to be to reformat upfront, use a webhook with {{trigger.Web Page}} to process and make decisions elsewhere such as FlowBoost or a custom RESTful service. It's a bummer you can't use {{trigger.Web Page}} in choices, but you could also probably use it to log an interesting moment, then set up a trigger on one of those instead of using a webhook.

Anonymous · ‎06-01-2016

Can you actually run this every hour? When trying to set this up, I saw you could only schedule daily at most. We could make 24 separate campaigns, but then exclusion wouldn't work (and that would suck).

SanfordWhiteman · ‎05-26-2016

Don't trust [null] to really be ANSI SQL NULL because there are some datatypes that will set the default even though they say [null]. Let me see what we're doing with clearing a field. I can't believe we aren't doing that in any of our webhooks.

Mike_Reynolds2 · ‎12-08-2015

Included in this article Summary How Leads Become Known Anonymous Web Activity Logging Triggered Smart Campaigns Revised 2/19/2016 Replaced screen shots with corrected versions Summary When a person visits a web page with your Munchkin tracking code on it, that web activity is tracked for that anonymous lead. When a lead takes action to become known, a lead record is created in your Marketo lead database and the anonymous web activity that lead had prior to becoming known is included in that new lead record. Any smart campaigns that would have triggered off of that web activity will then fire in the order of when the web activity occurred. How Leads Become Known There are three main ways that an anonymous web activity record can be converted into a known lead in your Marketo lead database. 1. Fills out form . If the anonymous lead fills out a Marketo form, that will create a new known lead record. The anonymous web activity that lead had before becoming known will be associated to that new known lead record. 2. Clicks a link in an email . Hyperlinks in emails are automatically decorated with a tracking code that is used to associate an anonymous record back to a known lead record. If you have a lead in your database that hasn’t had any web activity yet (like a lead that was added from a list import), when the lead clicks the hyperlink in the email, that tracking code in the hyperlink will associate their Munchkin cookie to your known lead record and all of that anonymous web activity will be logged into that known lead record. 3. API call . You can use an API call to associate an anonymous web activity record to a known lead record. This can be done through a SOAP API syncLead call, or by a REST API Associate Lead call. Anonymous Web Activity Logging Anonymous lead activity is housed in a different database. This doesn’t create a lead record until the lead is promoted to a known lead. All activity a lead had prior to becoming known will be logged chronologically in the new lead record with the correct corresponding date/time stamps once the lead becomes known. Triggered Smart Campaigns Anonymous leads can’t trigger smart campaigns. However, when a lead becomes known, the active smart campaigns that would have triggered from the anonymous activity will be triggered. All trigger campaigns currently active at the time the lead becomes known will be triggered if the lead took the action being triggered on while anonymous. Only campaigns currently active at the time the lead becomes known will be triggered. The campaign definition at the time it is triggered is what will be executed. For example, if the campaign has a Score Change of +10 but is changed to +5 just before the lead becomes known, the new lead created will be given a score of 5, not 10. The campaigns that are triggered by the anonymous activity will be processed in order of the activities that occurred, oldest to newest. In the lead’s activity log, all campaign activities (with one exception below) will be logged with the time stamp of when the lead becomes known, right before the New Lead activity. Interesting Moment flow step activities will be backdated to the date and time when the corresponding anonymous activity occurred. Not all flow actions in the campaign will be executed. When leads flow through campaigns as a result of the promotion to known lead, only specific flow actions will be processed. All other flow actions will be ignored. The flow actions executed will be: 1. Change Score 2. Interesting Moment 3. Change Data Value 4. Add to List 5. Remove from List Secondary campaigns (campaigns triggered off of activities performed by the campaigns triggered by the lead promotion) will not be triggered. Pruned activities will not trigger campaigns. Anonymous web activity is not stored forever—it gets pruned over time, so any activity that has been pruned is no longer there to be triggered off of.

Jon_Wu · ‎04-27-2017

Wow super awesome advice! It's so refreshing to hear these creative solutions, and so quickly. I've spent days or weeks in the biggest chains with support and Marketo engineers without getting anywhere (other than our current brute force deduplication) for years, so thank you thank you thank you! I like the idea of using a JSON array too as delimiters seem pretty brittle and inexact The only thing we can do with the loopback method is to post a form, right? Basically we can only update fields or pass in a Munchkin cookie via _mkto_trk, but we can't merge things. If so, I like this idea for saving API calls on lead updates and we could easily set up a reverse proxy, so I think I'll look into that in case they block the instance hostname (which if I understand right, is now required instead of localhost). That form hack is pretty darn crazy, but it definitely sounds like it'd work. The bummer there is that it makes form logic very convoluted (and hard to debug for eng and marketing), and makes direct form events useless for things like lead scoring, or triggering an email. I could imagine that we store the name of the form in a custom field (perhaps as a JSON array in case there are multiple forms submitted quickly), send that to the webhook, have Salesforce write that to a lead, and have a separate trigger that looks for that custom form name field to be on a new lead, or updated on an existing lead, but whew that's messy. You sound pretty familiar with the ins and outs of Marketo though, so if that's your suggestion, I trust the advice much more just anybody getting that creative. Overall, are you basically saying that we should never create any lead in Marketo and funnel everything through Salesforce? It seems like this implies that we can't import lists into Marketo either - at least not without making sure all the leads are in Salesforce AND synced back to Marketo already. Is that the idea? If so, this seems like a pretty dramatic change for us, but I could see it working out. Using the forms API instead of the REST or SOAP API to associate the cookies after the leads are synced from Salesforce to Marketo seems like one of the key tricks here. Without that trick, it seems like we'd need to call the REST API once per pre-Salesforce-to-Marketo-sync session, and would therefore have a limit of creating 10k new leads per day (or fewer with multiple sessions), which isn't enough for us. It seems like it's impossible to use the API for anything that might apply to many leads at scale, which seems to indicate that we can only use it for reporting or occasional things. This a quite a bummer since with Salesforce you can add 10k API calls for ~$20 / month, and use bulk API calls to do 50k things at once, whereas with the Marketo REST API, it seems like many operations are 1:1 and this person said it costs $15k / year to simply add 10k API calls. If I'm getting that right, that's madness! Anyways, please let me know if I'm saying sane things, and thanks so much for your help!

Jon_Wu · ‎01-13-2015

Hi Jep, Thanks for the info. I'm now trying to split up batches by using both sinceLatest and sinceOldest with a 1 day span. I will then try to parallelize this by getting leads from each day in batches, then combining all those into 1 big list to deduplicate everything. However, I'm getting unexpected results. I'm also looking for an automated solution that I can keep running so ideally I'm not going to do any manual export since that would be hard to do continuously. We need batch deduplicaton because it looks like there are some platform bugs that cause our webhooks to fail. We try to deduplicate leads in real time as duplicates are created, but the API has some bugs that cause this to fail some small percentage of the time. A few questions: 1. What date is the LastUpdateAtSelector selector using? I'd expect it to use the updated date that I see in the Marketo UI. However, for something like this: sinceOldest.set(2014, Calendar.AUGUST, 1); sinceLatest.set(2014, Calendar.AUGUST, 2); I get leads with dates such as the following: Created: Apr 8, 2014 11:01 PM Updated: Dec 21, 2014 10:36 PM Created: Apr 8, 2014 11:00 PM Updated: Aug 12, 2014 9:30 PM Created: Aug 11, 2014 8:37 PM Updated: Aug 11, 2014 9:24 PM What's going on here? The filter is definitely working in that it filters out a few leads at once and some dates return nothing, but the dates on the leads themselves don't make sense. 2. Is there a limit to how many concurrent connections the API can handle at once? For example, if we've had Marketo running for 500 days, could I make 500 requests at once? I seem to be hitting some limits now but it's really trial and error. Thanks for all your help!

Anonymous · ‎12-08-2014

Marketo has the ability to see and pull data from Salesforce Formula fields, there is however a catch which will be explained in this article. Everytime the Marketo Sync connects to Salesforce it will scan records and look at the " SytemModStamp " (salesforce system field) for each one of them. It will compare this value with the stored value, which was pulled at the last scheduled sync. If the values match, Marketo will move on to the next record. If the values are different (new value later date than previous value) , then Marketo will do a compare and contrast of all fields on that record in both systems and update the information as needed. When a normal non-formula field is updated and changed on a Lead/Contact record in SFDC, the SytemModStamp value is updated. This is how on next sync Marketo knows to do a compare/contrast check and pull updates. Formula fields do not behave the same way. A formula field is calculated based on data in fields called upon in the formula; this means that the formula field calculation itself will not update the SytemModStamp in Salesforce. Chances are you already have existing records in SFDC and Marketo. If you were to create a formula field today in your instance of SFDC and have it sync down into Marketo, the data calculated for the formula field in SFDC will not come into Marketo right away. The reason for this is, the formula field has created data based on already existing data, this does not result in a SytemModStamp change. Typically formula fields will be a calculation of data from fields which are somehow related to the lead/contact record. This means that moving forward, any change in the normal field, will result in a SytemModStamp change as well as a recalculation of the formula field. In this case, Marketo will see the updated SytemModStamp due to the normal field change. Marketo will do the compare/contrast excercise and find that the formula field also needs updating. If you create a formula field in SFDC and would like to have all the historical data for the formula field to come into Marketo, you can force an update on the records in SFDC to update the SytemModStamp. This way, on next sync, Marketo will see the formula data and pull it in. Alternatively, you can simply allow for natural SytemModStamp updates in SFDC to occur which should result in a slow trickle of historical data from SFDC into Marketo for the newly created formula field. You can only use data from a formula field in Marketo to segment data and filter. If you try to do a change data value, Marketo will accept the change, tries to sync it to Salesforce and fails to update there. Eventually the Salesforce calculated value will come back into Marketo. Is this article helpful ? YesNo

Jon_Wu · ‎09-14-2015

Unfortunately, the only possible solution in my case is to have a webhook that calls an API you write to deduplicate leads. You would call this webhook any time a new account is created where the email address isn't empty and the source wasn't Marketo. For example, in my case, leads must be created both through Munchkin and Salesforce (so data could be synced back) so we were getting duplicates since the Salesforce sync doesn't dedupe (which is really lame). One thing to caution you against is calling the webhook to deduplicate leads without an email address as they will become "known leads" in Marketo even though they are really anonymous (no email). This will cause your account to fill up because these leads without emails won't get automatically deleted after 90 days. You can actually use this issue to your advantage, but in general, you probably don't want this.

Anonymous · ‎10-29-2014

I also would prefer the "mkt_tok" were not appended to the URL as a GET parameter. As previous users have mentioned, this breaks caching and that's doing way more harm than good. Appending as a hash parameter would be better.

Jon_Wu · ‎08-11-2014

Hi Murtza, We've had this running for a while, and it turns out it's not quite working as expected. QUESTION: In what scenarios can duplicate leads be created? When you fill out your email on our site, I call associateLead from Munchkin. We are seeing many duplicates, but from the Munchkin documention for associateLead, I see "Marketo will automatically de-duplicate the lead if one already exists in your database". However, we also create a contact in Salesforce which then syncs back to Marketo, then we use a webhook to call mergeLeads with the SOAP API to merge the original Marketo lead to the Salesforce one synced one. It was my impression that only the synced lead from Salesforce could result in a duplicate since associateLead is supposed to deduplciate automatically. Instead, I can just keep calling associateLead from a fresh session and I get duplicates like in this screenshot: https://www.dropbox.com/s/6mrsiy7nby7pfl3/Screenshot%202014-08-11%2016.26.30.png. Any ideas? Thanks, Jon

Jon_Wu · ‎04-21-2014

Thanks Calvin. I'll go with the suggestion of adding my own info to track whether the lead is associated or not since anonymous lead tracking is important to me. Even better, I'm now looking for _mkto_trk before Munchkin initializes so I can know to call associateLead immediately if I know the user's email and this is the first time Marketo has seen them.

Online Status	Offline
Date Last Visited	‎07-26-2021 11:33 PM

Date Registered	‎04-08-2014 12:26 PM
Date Last Visited	‎07-26-2021 11:33 PM
Total Messages Posted	84
Likes Received	24

Top Badges Earned

Re: Submit Form API beta

Re: Highest Lead Score - Status changed to: Open I...

Re: Fix invalid for attribute on label tag in form...

Re: New trigger filter value on "Score is Changed"...

How can you skip Flow steps if they're being repla...

Re: Changing a lead score based on another score

Re: How can you decrement scores of (previously) a...

Re: Can I avoid race conditions when using static ...

Re: Are there known gotchas where Smart Campaign t...

Re: Are there known gotchas where Munchkin page vi...

Re: When a lead is merged / associated how can I m...

Re: For a Visit Web Page trigger, how do you match...

Re: Can you trigger on score changes 24 hours ago ...

Re: How can I set a data field to NULL from a webh...

Anonymous Lead Promotion to Known Lead – Munchkin ...

Re: Marketo Webhooks With Marketo API

Re: Java SOAP API - getMultipleLeads hangs even wi...

How does Marketo see and pull data from Salesforce...

Re: Can you use a Marketo lead ID instead of Email...

Re: Possible to add mkt_tok tracking as anchor ins...

Re: Avoiding duplicates with both associateLead an...

Re: Possible to detect if a lead is already associ...