Bot or Not? – Are you suffering from ‘bot clicks’?

[Ronen-Was-SRpro]

If you are a marketer, you probably have heard about the somewhat new and emerging enemy: The Email Security Bots. The war is on and we have been standing clueless for a while, watching from the side while the Bots were messing with our numbers, with no real solution available.
Well, it’s time to fight back!
It all started a couple of years ago when some of our customers noticed a surge in email click metrics and they also pointed out some interesting and strange behaviors in their data:

• Seconds after email is delivered, they see a high volume of clicks
• They noticed a high volume of clicks from the same account/company
• The activity log shows ‘clicked a link in an email’ before the ‘email open’ event and in many cases without visiting the actual page

Our research concluded those behaviors are typical for Bots, and NOT for humans. The bots' role is to click each link in emails, sent to the domain they protect, to prevent harmful clicks that can harm the company by flagging them as a phishing scam.
The implications of those Bots clicks can be devastating for marketing teams worldwide.
All your marketing numbers could be way off. It means you’ve been counting clicks completely wrong in your marketing automation program. Not to mention the impact on your scoring, interesting moments and nurturing campaigns and obviously your reports.
We, at eDigital.Marketing, did an extensive research and came up with a solution that we would like to share with you. We implemented it in our customers' instances and our customers are surprised by the findings and are satisfied with the results.
We started by running a test on an email that was part of a nurturing campaign already built in Marketo.
The test was using a Smart Campaign that was listening to page visits and email delivery.
The program was running for about five days to allow enough time for prospects to actually click on the link in the email.
Five days later we ran a new Smart Campaign just to collect data from Marketo about "clicked the link in the email" without any filters at all.
We downloaded both lists to excel and checked for clicks in Marketo that were NOT in the list we have created. Here are the shocking numbers:
Marketo counted 327 clicks WHILE the Smart Campaign only identified 91 of those clicks as real people who clicked the link and actually visited the page.

So at that point, it was pretty simple to calculate that approximately 72% of those clicks were fake and were made by ‘Bots’, we then identified and created a list of the companies that are using bots as part of their IT security infrastructure.
To make things even more complicated, we then went ahead and made some additional research on the list and found that a third of the remaining ‘humans’ can NOT be counted as clicked anyway since they visited the webpage in the past and NOT by using the email we've tested. The Smart Campaign was flagging them since the email was delivered to them and indeed in the past, they visited the page.
BUT since we compared the Marketo clicks to the Smart Campaign clicks, those ‘humans’ (that didn't click but visited the page) where excluded and therefore no extra calculating was needed.
To make sure the data is correct, we sampled some ‘Bot’ leads and checked their logs in Marketo. All leads from all companies who were suspected to be using ‘bots’ were showing the activity of a ‘bot’ - clicked but no open nor visited page.
In conclusion, out of 327 clicks identified by Marketo, only 91 were Humans.
Now all was left for us to do is to add a few Smart Campaigns to neutralize the Bots and stop them from disrupting the scoring system, the interesting moments and all reporting.
We now have a Bot system running in the background making sure all our numbers are correct and not just making us look good.

[Rafael_Santoni1]

What did you do when you say at the end...

"Now all was left for us to do is to add a few Smart Campaigns to neutralize the Bots and stop them from disrupting the scoring system, the interesting moments and all reporting.

We now have a Bot system running in the background making sure all our numbers are correct and not just making us look good."

Did you manage to make the Marketo native reports be accurate by excluding the bots, or are you now reporting using a different system/method?

[Dezie_Okpoebo]

I have also experienced this, and while I can't speak to the percentage of clicks that were "false" as I haven't run that full research, it has caused us many issues; Sales reps following up on false leads, bad & misleading reporting, etc. The solution highlighted above is a good one, but as some have noted, bots can also "view webpages." We were also having a hard time when we simply sent people to a pdf from an email. You can't see webpage visits to a pdf, unfortunately.

So we implemented a honeypot to our emails. A honeypot is also seen as bait for a bot that humans cannot see or click. We added a 1x1 pixel image that blended into either the header or the footer of our emails and hyperlinked it to a specific page. Now a human cannot see that link and therefore will not click the link, but bots will click every link within an email (as far as I know). We then created a smart list of clicks to that honeypot link, and used that to suppress the bots from reporting, Salesforce, interesting moments, lead scoring, you name it. I've heard back from Sales reps that the leads they are seeing have all improved significantly since we implemented the honeypot.

[Dan_Stevens_]

Thanks for sharing, Dezie - but the problem with this approach is many bots will eventually deliver the email to the intended recipient after all scans/clicks have been made and the email has been deemed safe.  And if you're filtering out those that click on the honeypot link, you may be filtering out legitimate/human-based clicks.

[Denise_Greenb12]

Hi Dan (and other click-bot combatants) -

I was re-reading this thread and just had an idea to try to mitigate the problem. After going through various of the exercises above to identify the companies in your database that almost certainly deploy link scanners, what about requiring a minimum of 2 clicks (using the min. number of times constraint) on a given link to count as a click if the recipient is flagged as from a link scanner company? That way you wouldn't lose the legitimate recipient who clicks after the bot has deemed the email safe.

[Emily_Salus1]

Exactly! You definitely want to include legitimate clicks, even if a bot does the first click (or set of clicks - some of them click on multiple or all links. How annoying!). Otherwise your reporting (and results) end up skewed.

I don't see a great way to solve this just using Marketo programs, triggers, campaigns, etc. and I've spent some time thinking about it. I had this problem at a previous company and it was a serious problem for our results and making decisions about where to put spend.

We have a way to solve this where I work, but I'm not allowed to do promotions here. You can search for me on LinkedIn with a note saying you want to discuss if it's relevant to you.

[Dezie_Okpoebo]

Hi Dan - yes that is something that we've thought about at length and in the long run we decided that was ok. At least until we can figure something else out. Also, if those folks really want to get in touch with us, they'll get pushed to Salesforce through other methods (ie form submits, etc.).

[Chris_Wilcox]

This is also a great method, but can you elaborate on specifically how your reporting is suppressing the link? We have a lot of ongoing pushes to our CRM when activities take place, how are you able to filter out only the bot clicks? Or, are you simply filtering any clicks from that email address if you see the honeypot interaction? Our goal is to suppress the bot click, but still try to capture any human engagement even if a bot had initially reviewed/clicked the email.

[Dezie_Okpoebo]

Hi Chris - Being able to differentiate the bot click from the human interaction is an issue we haven't been able to solve just yet, unfortunately. Though I've considered trying to combine the honeypot method with the "visited webpage" method to decipher between bots and human engagement. But again, that initial engagement with the honeypot remains on the contact activity. Anyone else have ideas on this?

[Yaj_Rathakrishn]

Unfortunately, this may not be accurate either. We tried that in our organization and what we found was - Yes we would be able to identify which ones were not the bots. However, if a bot does click on it, it doesn't mean the CTA page click wasn't legitimate. With an 80% bot click ratio, the number of people we would exclude would be astronomical in the long run.

We've noticed this issue for years and raised it with Marketo, but they aren't able to help and keep saying no one has heard of it which I 100% do not believe. But, we may have to stop looking at clicks as a determinant of success for campaigns and maybe start looking at the opportunities and quotes being generated per campaign and go from there.

[Dezie_Okpoebo]

We definitely don't look at clicks as a success metric anymore as a result of this. Like you said, we look more to product usage or opps created as success metrics and those get pushed back into Marketo for reporting.

I've worked with another automation tool and they said the same thing "bots don't click emails." I BEG to differ...

[Grégoire_Miche2]

Vote here friends:

[Danielle_Wong]

The test was using a Smart Campaign that was listening to page visits and email delivery.

how exactly did you set this test up?

[Frank_Elley]

Here's another place where this can bite you: reports. We email a daily "known visitors" and a weekly "anonymous visitors" report for specific geos/segments to the relevant sales teams. After a nurture or promotional email drops, the sales team sees a "too good to be true" surge of visitors from some accounts. So many things are wrong at this point. First, the sales team wastes time getting excited about a sudden spike in interest from an account that turns out to be a waste of time. This in turn damages the credibility of these reports (and thus of Marketo and Marketing Ops in general). And worse, as you can imagine, the sales teams who really embrace these reports are of course the most effective and good partners for marketers; so it's doubly a shame when these reports send false positives. Would also love to see some schematics or smart list/flow screenshots that give us a head start. These metrics are still useful, so we haven't discontintinued the reports; but you really just can't tell your sales team "don't believe them under XX conditions until we have time to re-rationalize them."

[Dan_Stevens_]

To ensure you’re truly dealing with human activity, you need to measure one additional step - and that’s identifying those that are web page visits are also clicking on a link (or submitting a form) on the landing page. Today, bots can trigger web page visits as well.

[Chris_Wilcox]

It's really interesting you're seeing bots trigger page views, I really haven't seen that correlation in our data yet. We operate in a pretty specific B2B sector, so I actually think it more likely has to do with the most commonly used technology in our sector. Barracuda seems to be our biggest problem, and from my research and own testing, it does not appear that Barracuda specifically downloads any javascript when it gets to the landing page, as I'm not seeing broad bot-click activity in Marketo, GA, or other web analytics tools, but I AM seeing the corresponding click activity in Marketo. The honeypot idea someone posted below seems like a decent way to avoid that, but only if you're batch reporting OR mass-suppressing known bot-click addresses.

Dan -- When you track that additional activity on the landing page, is that trigger based? Do you have a way you associate that activity with the program which the user originally clicked? That's my biggest gap that I'm trying to solve for at this point.

[Dan_Stevens_]

Chris - the way we used to do this (until we saw false-positives with page visits) was to add a "Visited web page" choice to the "change program status" flow step.  You can't add it as a filter in the smart list since it can sometimes take several minutes for Marketo to register the page visit when it has already triggered off of the email click.  So we add a 15 minute wait step and then something like this:

But now that we're faced with a third variable - and since Marketo doesn't support conditional "choice" logic (where we would need a second level choice of "clicked link on specific web page") - we're kinda stuck. Heck, there's not even a choice that allows you to select the link constrained by a web page.  I believe Sanford has been toying with a custom activity that would identify when all three of these actions have occurred, but nothing's been published as of yet.

Obviously, if the person has visited this page in the past - and not part of the email click session - then this too will result in a false-positive.

[Grégoire_Miche2]

Hi Dan,

You can use smart list and "member of smart list" in your choices if you need some advanced logic.

-Greg

[Dan_Stevens_]

Hi Greg - yeah, we use this approach for some of our smaller requirements.  This would not be a scalable approach here though - given the number of different types of emails we send and landing pages where we're driving traffic to.

[Grégoire_Miche2]

That makes sense... Unfortunately

-Greg

[Jay_Jiang]

Wouldn't you just use request campaign and add the filters in the requested campaign's smart list?

RE: detecting link clicks after clicking through from an email AND visiting a webpage, doable using munchkin API:

if(location.search.indexOf('mkt_tok') != -1){
$("a").addClass("tracklink");
$("a.tracklink").click(function(){
    Munchkin.munchkinFunction('visitWebPage', {
        'url': '/email-visit-click'
    });
});
}

Will add the class "tracklink" to all <a> tags if mkt_tok is present in the url search string. adds a listener to a.tracklink so when it's clicked it fires a munchkin visitwebpage to Marketo. You can then use the visited webpage filter for everyone who's visited "/email-visit-click"

[SanfordWhiteman]

1. Visit Web Page uses async XHR by design, so you can't make this event finish before the person navigates away from the page.
2. This requires an additional interaction with the page, which means it isn't tracking the page view only.
3. Why even bother with this, as the click event is already going to be logged as a Clicked Link which can be constrained by Web Page?

People seem to think this is easy to solve using only the tools available in the browser. It's not.