Bot-checks in emails

[Robb_Barrett]

Hi all – I was just having a conversation where I had to explain to someone that often times we register email clicks that aren’t actually human clicks but rather an email scanning program. It’s often confusing, but when I see clicks on privacy policy and other footer links I assume it’s a bot.

So I had a thought – what if we put invisible GIFs or a hidden character link in email templates that direct to a Bot-Check page?  With this, the tally of clicks on that link would equal the number of non-human clicks, which would make it easier to explain and verify that these aren’t real clicks.  Have any of you ever tested something like this?

[Robb_Barrett]

So far it's working for me.  I sent an email out this morning with that piece of code buried in the bottom corner of the email and I've already seen about 10% of my total clicks had hits to that page. It's a very small sample size so far and I want to give it more time to see how it works but it's helping me already provide more accurate reporting.

[SanfordWhiteman]

Yeah, I agreed with your concept that this method would tell you how many leads were using mail scanners, so it is useful.

But that isn't the same as "reversing the charges" for scanner-driven clicks...

[Anonymous]

Rob - Thanks for bringing this subject up

The email link scanning/checking issue and resulting false clicks impacts all Marketo customers to some degree. To me, it seems like this issue should be handled/solved at Marketo for all customers.

Here is why I believe that. Marketo already has the data to solve this issue. The activity log shows link scanning activity in an easily identifiable way. In each case, the link scanning software shows up as a click prior to the delivery of the email. See examples below in red. Although this data is view-able in our Marketo instance, there is no way to programmatically exclude these clicks. Normal email delivery and opens are shown in black.

Marketo can solve this issue for all customers by creating a back-end program with logic that looks for emails that report the Click Email status before the Email Delivered status and then ignoring those clicks for reporting purposes. Marketo could show this link checking software activity in the activity log with a status saying that this click will not be counted for reporting purposes.

The above is a simple example. With better data, I am sure Marketo could programmatically account for all of the different scenarios link scanning software creates. In the end, doing this will provide all Marketo customers with better, more accurate email performance data.

I would love to hear the the thoughts of others in the community.

[SanfordWhiteman]

This isn't the way the really sophisticated mail scanners work (which is probably why these are the only ones you noticed!).

They don't all scan mail before returning a 200 OK ("Delivered") because that makes it too easy for malicious actors to determine that they're being checked, respond with an innocent page, then go back to their malicious page after scanning is over. Accepting the mail for delivery and then scanning it is how you maintain accuracy.

I think people overstate the ease of detecting activity that by definition is supposed to be undetectable, or else it is foiled. Not to say there aren't ways to filter, but they aren't obvious!

[Anonymous]

Hi Sanford,

I think you are a few steps ahead of me. The sample data I provided above was from the Marketo results tab, I wanted to show everyone that this data it is accessible by Marketo today. I also wanted to show everyone that this behavior is readily identifiable. If other link scanning software doesn't cause this behavior, great, it doesn't impact my metrics. 

This challenge is not going away. First, as customers of a top tier, best in class, Marketing Automation Solution, we need Marketo to recognize that this behavior is an issue for customers.  Second, we need Marketo to start working on a solution to this issue. Maybe I am out of the loop, however, I am not seeing anything from Marketo.

[Nicholas_Manojl]

I'm not sure if this is a supported feature or not, but try emailing support and asking them to switch on the "Add hidden link to detect email bots" feature within the feature manager.

[Dan_Stevens_]

Hi Robb - the thought here makes sense, but don't forget, these bots will often click on the links and then - if the scanner doesn't throw the email into quarantine/junk - the email may actually get delivered to the recipient, who may generate valid clicks.  How would you distinguish the two?

[SanfordWhiteman]

the email may actually get delivered to the recipient, who may generate valid clicks. How would you distinguish the two?

Right, you can't say "this recipient is only a mail scanner" you can only say "this recipient uses a mail scanner." Otherwise you discard real clicks from the same lead.

However, Robb seems to only want to count the number of times a scanner-only link was clicked, so he can prove to other people that such automated clicks do indeed happen. And in that respect it will be useful.

[Josh_Hill13]

We are experimenting with a top link hidden in the HTML that goes to a Honeypot page. The spam bots we see click only the first link and even if they click ALL links, then anyone who clicks Link 1 we know is a bot.

No real results yet though, but that's the plan.

[SanfordWhiteman]

The spam bots we see click only the first link and even if they click ALL links, then anyone who clicks Link 1 we know is a bot.

Mail scanners have to click all links to be useful.  If something is clicking only one link it's not a good example.

Also, if "someone" clicks a presumed scanner-only link, that doesn't mean that all clicks registered to that someone are from the scanner. A human click a minute later would be discarded in that scenario.

[Keith_Nyberg2]

Haven't tested this but adding a honeypot to an email to track bot clicks does sounds cool. Only question I have is how do you maintain the hyperlink as tracked by Marketo while also being hidden? Unless you don't plan to hide it which could cause issues with real people clicking the link.... i mean who wouldn't click a link that says "Don't click this link!" Could be a cool idea to submit to Marketo to automatically embed a bot link in the email that would only get triggered by a bot click based on being hidden, but again, not sure how to accomplish this task.... make the hyperlink color match the email background?

[Robb_Barrett]

You can HREF a space or a random character in the footer that's the same color as the background and no underline, so in theory... <a href="blahblah.com" style="color:{{my.background color}}; text-tecoration: none;">&nbsp</a>

[Prachi_Chheda]

Hi Robb - Thanks for sharing this post, we are facing similar issue, we have updated the above code at the footer of our email, my question is the link suppose to be hidden from human eye as well as not clickable by humans Our code is as follows <p><a href="https://www.xyz.com/?" style="color: {{my.background color}}; text-tecoration: none;"></a><br /></p>, when we send test email we cannot click this link, is this an expected behavior?

[SanfordWhiteman]

The link is intended to be hidden. Of course this means it also looks like a clickjacking attempt. (Also, it will not be hidden in all mail clients, no matter what you do.) I wouldn't place much stock in this method.