Webhook making an anonymous lead known

[Grégoire_Miche2]

Dear all,

We had a weird surprise today. A customer of ours has its database full of empty leads (no names, no emails). About 3000 of them. e.g.:

When looking at the activity logs of these leads, they all have in common to have navigated the web site and suddenly the score triggers have fired, following the ones that computes total score, which is done through a webhook (as you will see, all this happens in the same second...):

The very weird thing is that the detail of the "new person" activity shows that the lead was created by the webhook:

The smart campaign that call the webhook fires on the following triggers:

I just cannot make any sense of this!

All of these smart campaigns are not supposed to fire on anonymous leads in the first place, but only when the lead becomes known...

we have opened a support ticket, I'll let the community know.

-Greg

[Mike_Reynolds2 · Accepted solution]

Hi Grégoire Michel

Can't speak officially for something I'm not personally familiar with, and webhooks were never really my personal strong suit. I'd have to defer to the Support ticket on this one.

As for the lead promotion part, the smart campaigns not triggering off of anonymous leads will only apply after the instance has been moved to Munchkin V2. Not all instances have actually been moved to Munchkin V2 yet. There was a rolling deployment of Munchkin V2. The removal of access to anonymous leads was done already, but the next step after that is the changeover to Munchkin V2, which is a more complicated upgrade being made and is taking time to implement. So if you're not seeing the Munchkin V2 behavior (anonymous leads go through trigger campaigns while still anonymous, not through the campaign replay activity used in Munchkin V2), then your instance likely hasn't been switched over yet.

[Grégoire_Miche2]

Hi Mike Reynolds,

Any insight and official statement here would be welcome, and probably necessary.

-Greg

[Mike_Reynolds2 · Accepted solution]

Hi Grégoire Michel

Can't speak officially for something I'm not personally familiar with, and webhooks were never really my personal strong suit. I'd have to defer to the Support ticket on this one.

As for the lead promotion part, the smart campaigns not triggering off of anonymous leads will only apply after the instance has been moved to Munchkin V2. Not all instances have actually been moved to Munchkin V2 yet. There was a rolling deployment of Munchkin V2. The removal of access to anonymous leads was done already, but the next step after that is the changeover to Munchkin V2, which is a more complicated upgrade being made and is taking time to implement. So if you're not seeing the Munchkin V2 behavior (anonymous leads go through trigger campaigns while still anonymous, not through the campaign replay activity used in Munchkin V2), then your instance likely hasn't been switched over yet.

[Grégoire_Miche2]

Hi Mike,

The issue is that we have no clue whether or not a given instance has been upgraded. And honestly, more than a year after the announcement, I am very surprised that not everyone has been upgraded yet. Furthermore, there is no calendar available, and we find out the hard way, which is, IMHO, not the best of customer friendliness

-Greg

[Anonymous]

Hi Greg,

I just got bitten by this, too. We have over 5K leads now with no contact information. And it's only the Webhook that "promotes" the anonymous leads to known. We don't see them in any other campaigns so I don't think this has anything to do with Munchkin V1 vs V2. AND... I have found the same thing that Jon Wu reported - which is that this create duplicates. My initial interaction with Support on this issue seems to indicate that they think this is normal. It strikes me as major failure on Marketo's part that they do nothing to prevent duplicate creation when a Webhook adds the lead to the database.

When I discovered all this was happening last week I added a filter for Email Address is not empty to the campaign that calls the Webhook so I hope that solves the problem going forward. But I don't yet know what to do with the 5K records with no contact info already created since they are a sitting invitation for dupe creation (not to mention bumping up the DB size).

Denise

[SanfordWhiteman]

It strikes me as major failure on Marketo's part that they do nothing to prevent duplicate creation when a Webhook adds the lead to the database.

You can avoid the duplicate creation by ensuring that associated sessions get a form that doesn't have the Email Address field on it. That is, use Known Lead HTML ("If Known Lead, show Custom HTML" in Form Editor) to decide what form they see.

Already-associated sessions shouldn't be able to enter a new Email Address (which creates a new lead). They should only be able to enter a proxy field (like Email Address 2) that then updates their Email Address in a Flow step.

When I discovered all this was happening last week I added a filter for Email Address is not empty to the campaign that calls the Webhook so I hope that solves the problem going forward.

Yes, that works around the problem, but then you're no longer scoring leads in the same way.

[Anonymous]

Interesting idea on the forms.

Re: "Yes, that works around the problem, but then you're no longer scoring leads in the same way."

It seems to me we are still scoring leads the same way because all the webhook does is add behavior score + demographic score and store the result in a combined score field. So as soon as the lead (really) becomes known, that formula will run and the result will be the same as it would have if the combined score field had been populating behind the scenes while the lead was anonymous.

If I think harder about this I guess there are some rare (I think) exceptions where the lead becomes known via a non-scored activity. The only one that comes to mind is if the lead is created in SFDC. And the form treatment you outlined above wouldn't solve that one.

[Grégoire_Miche2]

Here is the answer from support:

"In order to prevent anonymous leads from going through campaigns that call webhooks, which in turn convert the lead records to "known" leads, I would recommend adding a filter to the smart list of any campaign calling a webhook. This filter can be anything like Email or First name > Is Empty. You'll want to use some filter based on a field that you know anonymous leads will have empty, so that adding the filter to your campaign's smart list will keep them from getting to the webhook flow step."

The fix is easy, but this is really amazing to me since nowhere I have read that webhooks could make an anonymous lead known!

-Greg

[Jon_Wu]

I had an issue like this too and came to the same conclusions after long threads with support. We had a high score webhook writing to the lead that was causing leads without an email to be considered known. This also caused our total number of "known" leads to grow a lot since they'd never be deleted.

Although I hit this years ago and worked around it, I recently got hit again by this on a related issue where webhooks were causing anonymous leads to become known, thus breaking automatic lead deduplication. Since the webook would make people "known" the deduplication by email address wasn't kicking in, even for leads directly added to Marketo via form or REST API (not via Salesforce where we'd expect duplicates to come from).

Support said this issue would be fixed with Munchkin v2, but also said we weren't fully migrated and didn't know when... Very frustrating as you've noted.

[DeniseGreenberg]

Hi @Jon_Wu  - I know this was a long time ago but do you happen to know if the issue of Webhooks promoting anonymous leads to known was indeed fixed in Munchkin v2? 

[Jon_Wu · Accepted solution]

@DeniseGreenberg yes it's been a while, but IIRC this is not an issue anymore. FWIW I did just search our database for ones without emails and nothing came up also.

[Dan_Stevens_]

Thanks for sharing this with us, Greg.  I, too, didn't think this was possible to create a new lead from an anonymous lead.  I need to ensure this isn't happening to us!

[Grégoire_Miche2]

That would be a funny way to come back to Marketo previous behavior, when the anonymous leads were accessible. Just create a dummy webhook that updates a field and call it on a "visits web page" trigger. That would make all web page visitors known... And also probably kill the performance of you instance if you have some volumes.

The database size would increase dramatically, though, but if I remember well, the contract says that only the leads who can be reached (email, phone, address) really count. So these ones would not...

-Greg

[Dan_Stevens_]

The database size would increase dramatically, though, but if I remember well, the contract says that only the leads who can be reached (email, phone, address) really count. So these ones would not...

We're currently going through contract negotiations as we speak, and I was surprised to learn that ANY RECORD in Marketo will now go against your database size (even if email, phone, address are blank).  So be careful with this.

[Anonymous]

Originally, the database count was based on marketable records but it was at least 3 years ago now that Marketo starting counting all Known records in the database for the contract limits. So you weren't imagining that it was only marketable leads... it changed.

Regards,

Sheila

[Grégoire_Miche2]

Hi Sheila,

The problem with the issue raised in this post is that it turns anonymous leads (that are not supposed to count) into known leads. If we delete these leads, we loose the historical information. If we keep them, they count, although they are not supposed to be there, since Marketo created them erroneously...

-Greg

[Grégoire_Miche2]

Hi Dan

Thx

-Greg

[Casey_Grimes]

So, uh...

If you theoretically wanted to create anonymous records in controlled circumstances, am I reading correctly that Marketo isn't going to patch this? Because there is absolutely a use for this that has pained me since anonymous records were removed a while back. However, I don't want to build this solution just to see this patched next release.

[Grégoire_Miche2]

Hi Courtney,

I do not have the slightest idea on whether this will be patched or not... This is so unexpected that I would not bet that it will last as long as income taxes

The very surprising part is that support seems to be considering this as perfectly normal. Here is the second answer we had when we requested the records to be sent back to anonymous:

It's not that the webhooks convert the leads per se, it's more that the webhook sends data about the lead out to whatever service it calls, and when the data comes back to Marketo is has to have somewhere to put it so it creates the new lead.

Unfortunately there is no way to turn these records back into anonymous records, so if you must have them removed from the lead list the only way would be deletion.

as a result, we have a database with 3000+ empty leads that might never convert and we now have the choice between deleting them (and loosing all the historical data) and keeping them and cluttering the database with them, expecting an hypothetical form fill out.

-Greg

[SanfordWhiteman]

It's not that the webhooks convert the leads per se, it's more that the webhook sends data about the lead out to whatever service it calls, and when the data comes back to Marketo is has to have somewhere to put it so it creates the new lead.

Sidestepping the question of how Anonymous leads are going through trigger campaigns without explicit promotion.

This is really frustrating because people like us are getting the wrong info, and we're relied upon to be in the know about "what not to do or attempt"...

[SanfordWhiteman]

Yeah, this is totally messed up.  This behavior was specifically supposed to be turned off.  I don't understand if this is a regression or whether we're now resuscitating old behavior w/r/t Anonymous leads.