76 Replies Latest reply on Apr 27, 2018 2:09 PM by Sanford Whiteman

    Spam filters registering clicks?

      Has there been any problems with spam filters scanning emails and registering clicks as they follow the links in the email?  We are getting false positives on our email clicks.
        • Re: Spam filters registering clicks?

          There are several posts here on Marketo about this issue, and my firm has been digging into it a lot over the last few days. The short answer is that yes, this does indeed happen - spam filters (like Barracuda) / bots / junk mail algorithms do indeed click on links in emails (see this interesting blog post from 2013 regarding the issue - Barracuda calls this "multilevel intent analysis"). The spam filter is looking for redirection or malware or something like that. There isn't a whole ton that we marketers can do about it, though. Here is what we've done and found:

          • First thing we did was download the entire Marketo activity log using the API, put it in a database, and started dissecting the "Click Email" event types. We also sat down with the system administrator here to review some of this data. In short: there is nothing in the User Agent, Platform, Device, etc. that will help spot these.
          • Then we started looking at the timing: what about people who click before they open? What about people who click really quickly after the "Send Email" activity is logged? Well...the "Send Email" event isn't indicative of when, exactly, the email leaves Marketo's servers, so that's not really accurate - you can't spot bots based on that.
          • The best way we've found right now is to include a one-pixel picture / link on the email - invisible to just about everyone (as suggested here). Anything that clicks on such a tiny little pixel you can consider a bot. True; someone might not load images and see a box, but most people won't see it at all.
          • Another possibility: see if you have a bunch of clicks that all happen at the same time (or people clicking every link in an email, every week - would a real person really need to read your Privacy Policy week-in and week-out?). Those are probably bots...but I personally would want to download the data into a real database before attempting this kind of query.
          • One more (really complex) possibility: when we went to our sysadmin (the guy who runs our own company's Barracuda machine) about a lot of these issues, he started to "ping" some of the IP addresses included in the suspicious "Click Link In Email" activities. One or more of them shot back a response indicating that it was a Barracuda box. If you are really, super-duper concerned with this problem, it should be possible to download all Marketo activities via the API and write some custom script / code to extract the IP addresses from the Marketo "Click Email" events and then to periodically ping all these servers to see if you can get them to self-identify as a spam filter (parse the text-strings of the responses for incriminating evidence).

           

          We have not done this last thing, as our "one-pixel" solution has indicated (at least over the last two weeks) that it's likely not a major issue. Perhaps some day, when our organization has unlimited resources (heh), we will pursue this last option, but the reality is that we have a lot going on and better things to do to add more value to our marketing efforts.

           

          I would also like the data to exist in a perfect world - one where our Users validate our TRON Data Discs and we can take down the evil Master Control Programs while we're on our light-cycles on the grid - but that gleaming world of perfect, neon data does not exist. For most of us, I would guess this statistical aberration will not significantly affect our analysis of content effectiveness.

           

          Hope this helps.

          17 of 17 people found this helpful
          • Re: Spam filters registering clicks?
            Chris Saporito

            We just ran into this same issue today. After reading through Matt's response, not exactly sure what our next step should be. Lots of good info though!

            1 of 1 people found this helpful
            • Re: Spam filters registering clicks?
              Courtney Grimes

              Matt gives a lot of great advice here, but I did want to just add as a footnote/call to action for anyone else annoyed by the current situation: I've been talking to a few different filtering companies about adding some unique, filter service-only string to their UA when checking links (they normally spoof specifically as IE/Win7) in order to correctly differentiate human vs. machine clicks. The problem affects more than marketing automation platforms; I find myself continually explaining this to transactional email provider users, for instance.

               

              I'd highly encourage people to go bother Cloudmark/Symantec/Barracuda as well so I don't seem like a lone weird geek on this point.

              1 of 1 people found this helpful
                • Re: Spam filters registering clicks?
                  Sanford Whiteman

                  filter service-only string to their UA when checking links

                  Doesn't make sense though.  That would make their service worthless because it's supposed to be prechecking for hostile sites.  All a site would need to do is UA sniff and return a non-malicious payload.  They need more randomness, not less (my experience is it isn't the same UA at all).

                  1 of 1 people found this helpful
                  • Re: Spam filters registering clicks?
                    Beth Rotach

                    I'd be happy to reach out to Barracuda - that's the culprit for us about 90% of the time. What do I need to do/say?

                      • Re: Spam filters registering clicks?
                        Sanford Whiteman

                        "Please don't protect your clients against phishing and malware attacks"?

                         

                        Seriously, I don't see why any anti-spam software that offers this functionality in the first place would offer to turn it off.  It's obviously by design that it follows links, even if it seems like a fiasco from an analytics standpoint. It might also be better to not contact them because according to the research here, there is a way of avoiding spurious clicks, but they could break that if they wanted to (by starting to follow JS redirects).

                    • Re: Spam filters registering clicks?

                      One of my big concerns is that we're passing this activity into SFDC, which our inside reps are using to follow up on what seemingly look like responses to email campaigns. Have others resulted to just disabling that type of activity from being passed to SFDC?

                        • Re: Spam filters registering clicks?
                          Sanford Whiteman

                          One of my big concerns is that we're passing this activity into SFDC, which our inside reps are using to follow up on what seemingly look like responses to email campaigns. Have others resulted to just disabling that type of activity from being passed to SFDC?

                          If you mean replicating all activities (well, one per lead per activity type per day, technically), yes, you would have to turn that off if you want to perform any filtering.  If you create a Smart Campaign that seems to work, like the one Conor proposes above, you can use that to create SFDC tasks and/or Interesting Moments that correspond to the filtered activities/sequences of activities.

                        • Re: Spam filters registering clicks?
                          Beth Rotach

                          Just a quick note - we've been emailing pretty aggressively with Marketo support regarding this issue. We also found out that MANY other ESPs provide this "click filtering" as part of their service because it happens so often to folks. We recently spoke with about 8-10 other ESPs that automatically (and very easily) filter these clicks for their clients. Apparently the ESP can easily filter clicks by IP/known barracuda IPs and code. Marketo deliverability team assured that their are now working with the product side to try to implement this ASAP, especially because so many people are asking about it - KEEP ASKING!

                          11 of 11 people found this helpful
                            • Re: Spam filters registering clicks?
                              Sanford Whiteman

                              Apparently the ESP can easily filter clicks by IP/known barracuda IPs and code.

                              Nope, they cannot (and are not) doing it this way. It's a preposterous claim, and any ESP that claims to reliably filter automated clicks this way is lying.

                               

                              Rather, they are using a mechanism that is closer to what I have described in this thread and elsewhere.  It is possible for Marketo to attempt the same and achieve a high degree of coverage.  But to the degree that it works, it is because of the defensive coding used by the mail scanner (to prevent an amplification attack against the scanner itself) and not because of any special brilliance or detective work by the ESP.

                            • Re: Spam filters registering clicks?
                              Robb Barrett

                              OK, here's a situation I was presented with this morning:

                              We use Marketo for our Contact Us page and we have workflows that fire off alerts. One of the alerts has two links: I've Handled This, or I Need to Re-Route This. There is a follow up workflow that is triggered on the I've Handled This click.  One of my colleagues asked for help because a click is firing off the follow-up workflow 4 times.

                               

                              One of the logs I looked at shows the initial alert delivered Sunday night.  On Monday morning, the alert was clicked at 10:34am and there was a corresponding VWP.  Then, also at 10:34 I see two more Clicks Link and only one VWP.

                               

                              My first thought is that I have a double-clicker. I created a lead for myself following the process.  I was very slow about waiting to click on the link the first time. I did, it registered one click, one VWP, then nothing more. I put in a filter for Not Clicks Link In Email in Past 1 minute to see if that would help.  Then, about 5 minutes after my first click I double clicked. It registered two clicks and two VWPs.

                               

                              A minute later, it registered 3 clicks and 1 VWP. These were not by me or anyone else.

                               

                              Now, it's work noting that we have a URLDefensePoint system in place. All links in emails are re-coded by the server with DefensePoint to check. I'm thinking that it's testing the link for us to see what happens prior to allowing the browser to go to the link.

                               

                              Thoughts?

                               

                              Sanford Whiteman

                                • Re: Spam filters registering clicks?
                                  Sanford Whiteman

                                  Now, it's work noting that we have a URLDefensePoint system in place. All links in emails are re-coded by the server with DefensePoint to check. I'm thinking that it's testing the link for us to see what happens prior to allowing the browser to go to the link.

                                  I think you're correct.

                                   

                                  And this is a case where, unlike inbound scanners I know of, the outbound/opt-in service can afford to perform deep scanning because they only see a subset of links. That is, they are actually following the JS redirect, so they generate a Visit Web Page as well as a click. (Inbound scanners can't afford to do this because from a defensive programming standpoint they could tie up their own resources.)

                                  2 of 2 people found this helpful
                                • Re: Spam filters registering clicks?
                                  Robb Barrett

                                  This is the same thing I'm seeing on spam traps too. You get a flurry of clicks and few VWPs.

                                  • Re: Spam filters registering clicks?
                                    Carmi Lopez-Jones

                                    Thanks to Kiersti Esparza, Manager of Privacy/Deliverability at Marketo, who has just posted a community article on this topic.  Understanding a Spike in Click Activity.

                                     

                                    Cheers!

                                    Carmi

                                    1 of 1 people found this helpful
                                    • Re: Spam filters registering clicks?
                                      Devraj Grewal

                                      I provided a couple of workarounds for this issue on my discussion topic: Email was clicked before it was delivered? It's a link scanner

                                      • Re: Spam filters registering clicks?
                                        Robb Barrett

                                        Spam spam spam eggs spam spam spam

                                         

                                        Here's something I put together on how to find link scanners / Spam traps.

                                        1 of 1 people found this helpful
                                        • Re: Spam filters registering clicks?

                                          Hi Matt,

                                          Thanks for the solution! I wasn't able to find the url link for the 1 pixel picture. Is there a specific link that needs to be added or can it be any link we set?

                                           

                                          Thanks,
                                          Julia

                                          • Re: Spam filters registering clicks?
                                            Robb Barrett

                                            Sanford Whiteman - just had a thought I'd like to bounce off of you:

                                             

                                            Somewhere near the bottom, create a link with style="display: none;" so no one would see it or find it without reading the code. This would mean any click on the link or visit to the page would be a machine.

                                            1 of 1 people found this helpful
                                              • Re: Spam filters registering clicks?

                                                That's a really cool idea! I thought the link has to be on top of the page though?
                                                I can give it a try and see what the result's like.

                                                 

                                                Thanks!

                                                Julia

                                                • Re: Spam filters registering clicks?
                                                  Sanford Whiteman

                                                  Unfortunately, that'll have the same problem as every similar approach, such as an <A> wrapping a 1x1 image or an <A> with no contents at all --  or for that matter an <A> that says, "Don't click this under any circumstances, it's a virus."

                                                   

                                                  Yes, if it's the first link clicked by a mail scanner (it won't necessarily be, since they can click in any order) then a click within the next, say, 10 seconds is probably (again, not necessarily) that same scanner.

                                                   

                                                  But Marketo's granularity (Filter: Not Clicked Link in Email, Link: {your-scanner-lure-URL}, Date: In past 10 seconds) won't catch this.

                                                   

                                                  The problem has never been building a link that could only be clicked by a non-human. It's how to action that activity in the context of other activities. Like I say every time... until Marketo actually changes their tracking logic so that closely contiguous activities for the same lead are scanned as a batch before committing them to the activity database, there's no way to situate the probably-automated traffic in context.

                                                   

                                                  And bear in mind that anything Marketo might try to do, an attacker hosting malware can do as well (read: if it looks like a mail scanner, send back innocent results). So even though link scanning is half-measure anyway, it is not supposed to be trivially avoidable, since the bad actors have anything but trivial skillz.

                                                  2 of 2 people found this helpful
                                                • Re: Spam filters registering clicks?
                                                  Dan Stevens.

                                                  Just a quick note - we've been emailing pretty aggressively with Marketo support regarding this issue. We also found out that MANY other ESPs provide this "click filtering" as part of their service because it happens so often to folks. We recently spoke with about 8-10 other ESPs that automatically (and very easily) filter these clicks for their clients. Marketo deliverability team assured that their are now working with the product side to try to implement this ASAP, especially because so many people are asking about it - KEEP ASKING!

                                                  It's been about 16 months since the post was made.  Can anyone from the Marketo product team chime in and let us know if an enhancement to more accurately track valid email clicks?

                                                  • Re: Spam filters registering clicks?
                                                    Robb Barrett

                                                    Not an official person, but if you see a "Clicks Link" without a "Visits Web Page" you've got yourself a spambot.

                                                    • Re: Spam filters registering clicks?
                                                      Benjamin Ortiz

                                                      I see what you're saying - I guess my question should be, is using the above filter/trigger combo the most reliable solution known today to attempt to filter out bot clicks? and are there any known issues with tracking marketo hosted pages where I would miss out on real clicks by using this filter/trigger combo?