11 Replies Latest reply on Jan 18, 2018 1:20 PM by Sanford Whiteman

    Bot-checks in emails

    Robb Barrett PRD

      Hi all – I was just having a conversation where I had to explain to someone that often times we register email clicks that aren’t actually human clicks but rather an email scanning program. It’s often confusing, but when I see clicks on privacy policy and other footer links I assume it’s a bot.

       

      So I had a thought – what if we put invisible GIFs or a hidden character link in email templates that direct to a Bot-Check page?  With this, the tally of clicks on that link would equal the number of non-human clicks, which would make it easier to explain and verify that these aren’t real clicks.  Have any of you ever tested something like this?

       

        • Re: Bot-checks in emails
          Keith Nyberg

          Haven't tested this but adding a honeypot to an email to track bot clicks does sounds cool. Only question I have is how do you maintain the hyperlink as tracked by Marketo while also being hidden? Unless you don't plan to hide it which could cause issues with real people clicking the link.... i mean who wouldn't click a link that says "Don't click this link!" Could be a cool idea to submit to Marketo to automatically embed a bot link in the email that would only get triggered by a bot click based on being hidden, but again, not sure how to accomplish this task.... make the hyperlink color match the email background?

          • Re: Bot-checks in emails
            Dan Stevens

            Hi Robb - the thought here makes sense, but don't forget, these bots will often click on the links and then - if the scanner doesn't throw the email into quarantine/junk - the email may actually get delivered to the recipient, who may generate valid clicks.  How would you distinguish the two?

              • Re: Bot-checks in emails
                Josh Hill

                We are experimenting with a top link hidden in the HTML that goes to a Honeypot page. The spam bots we see click only the first link and even if they click ALL links, then anyone who clicks Link 1 we know is a bot.

                 

                No real results yet though, but that's the plan.

                  • Re: Bot-checks in emails
                    Sanford Whiteman

                    The spam bots we see click only the first link and even if they click ALL links, then anyone who clicks Link 1 we know is a bot.

                    Mail scanners have to click all links to be useful.  If something is clicking only one link it's not a good example.

                     

                    Also, if "someone" clicks a presumed scanner-only link, that doesn't mean that all clicks registered to that someone are from the scanner. A human click a minute later would be discarded in that scenario.

                  • Re: Bot-checks in emails
                    Sanford Whiteman

                    the email may actually get delivered to the recipient, who may generate valid clicks. How would you distinguish the two?

                    Right, you can't say "this recipient is only a mail scanner" you can only say "this recipient uses a mail scanner." Otherwise you discard real clicks from the same lead.

                     

                    However, Robb seems to only want to count the number of times a scanner-only link was clicked, so he can prove to other people that such automated clicks do indeed happen. And in that respect it will be useful.

                  • Re: Bot-checks in emails
                    Nicholas Manojlovic

                    I'm not sure if this is a supported feature or not, but try emailing support and asking them to switch on the "Add hidden link to detect email bots" feature within the feature manager.

                    • Re: Bot-checks in emails
                      Eric Price

                      Rob - Thanks for bringing this subject up

                       

                      The email link scanning/checking issue and resulting false clicks impacts all Marketo customers to some degree. To me, it seems like this issue should be handled/solved at Marketo for all customers.

                       

                      Here is why I believe that. Marketo already has the data to solve this issue. The activity log shows link scanning activity in an easily identifiable way. In each case, the link scanning software shows up as a click prior to the delivery of the email. See examples below in red. Although this data is view-able in our Marketo instance, there is no way to programmatically exclude these clicks. Normal email delivery and opens are shown in black.

                       

                      Link Scanner Image 1.png

                      Marketo can solve this issue for all customers by creating a back-end program with logic that looks for emails that report the Click Email status before the Email Delivered status and then ignoring those clicks for reporting purposes. Marketo could show this link checking software activity in the activity log with a status saying that this click will not be counted for reporting purposes.

                       

                      The above is a simple example. With better data, I am sure Marketo could programmatically account for all of the different scenarios link scanning software creates. In the end, doing this will provide all Marketo customers with better, more accurate email performance data.

                       

                      I would love to hear the the thoughts of others in the community.

                      1 of 1 people found this helpful
                        • Re: Bot-checks in emails
                          Sanford Whiteman

                          This isn't the way the really sophisticated mail scanners work (which is probably why these are the only ones you noticed!).

                           

                          They don't all scan mail before returning a 200 OK ("Delivered") because that makes it too easy for malicious actors to determine that they're being checked, respond with an innocent page, then go back to their malicious page after scanning is over. Accepting the mail for delivery and then scanning it is how you maintain accuracy.

                           

                          I think people overstate the ease of detecting activity that by definition is supposed to be undetectable, or else it is foiled. Not to say there aren't ways to filter, but they aren't obvious!

                        • Re: Bot-checks in emails
                          Robb Barrett PRD

                          So far it's working for me.  I sent an email out this morning with that piece of code buried in the bottom corner of the email and I've already seen about 10% of my total clicks had hits to that page. It's a very small sample size so far and I want to give it more time to see how it works but it's helping me already provide more accurate reporting.