SOLVED

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Go to solution
tcthanawan
Level 2

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Hi Karina, 
We don't have a practice of purchasing a list, but we found that our audience has a security gate on their end. Once an email passes through that security gate, it is often marked as clicked or opened. Marketo recommended we turn on the bot activity filter in the admin setting. However, some still came through.

It is really great to hear you got the automation that cleans up the leads, do you use Marketo form as well? 

gergana
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Hi everyone,

We're encountering issues with bot activities. Although we haven't purchased any lists, our target audience is B2B, which means we're seeing a lot of bot activities coming from email security systems that 'click' every link in the email and this is causing issues on report level and could also influence some operational programs that update some values/scoring etc.

To address this, we've activated the bot activity filter in the admin settings (using both the IAB list and Proximity pattern). Since enabling these filters, a significant amount of bot activity has been filtered out. We've decided to keep a log of bot activity for troubleshooting and analysis purposes.

However, I've noticed that the filter isn't perfect. Sometimes, a contact shows a sequence of 10 consecutive clicks, but not all of these clicks are marked as bot activity (even though the timestamps are the same for all actions). While we can't see the exact timestamps (till the second), it's clear that this is bot activity, especially when there's an open email an hour later with real clicks following it.

Another issue we're seeing is with the Email Performance Report, particularly for newsletters with many links. Even when we apply a filter in the smart list (for "Clicked link in the respective email" with the constraint "Is bot activity" = True), the report still doesn't provide accurate numbers:

For example:

  • Email link performance report with a smart list filter set on "clicked link in email (Email1)" and a constraint "Is bot activity" = False -> Link 1: 616 people & 1.311 clicks
  • Email link performance report with no filter in the smart list -> Link 1: 634 people & 1.333 clicks
  • Smart list with "clicked link in email (Email1)", constraint "Link" is = Link 1, constraint "Is bot activity" = False: 431 people
  • Smart list with "clicked link in email (Email1)", constraint "Link" is = Link 1, constraint "Is bot activity" = True: 235 people

The sum of the two smart lists doesn't match the 634 people in the report, as there are 32 contacts that are present in both smart lists.

This is very confusing because we can't rely on the email performance report to provide accurate numbers for email link performance. If we need to report on the real clicks for each link in the email, we have to create smart lists for each link individually.

If I understood well, in the case of Email Link performance, when we apply the mentioned filter in the smart list, the logic seems to be as follows: from all the people who have clicked on the email, consider only those who have at least one non-bot activity, then display this in the report. However, as noted, many people have activities that are sometimes detected by the filter as bot activity and sometimes not. This inconsistency is why we see almost no difference in the email performance report, regardless of whether we include the bot activity filter or not. (I am not sure I have explained myself well :))

Has anyone else experienced similar issues? Do you have any suggestions?

 

tcthanawan
Level 2

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Hi Gergana, 

 

It sounds like we are both experiencing the same issue in the B2B space, although we did not add the "Is bot activity" constraint to our smart list and did not keep the bot log. 

 

Similar to you, we found that the email performance report did not provide an accurate number as it included bot activity. So, we are using the smart campaign to help us pull the list of people who clicked on a certain link, and on top of that, we used the Visited web page smart campaign just to ensure the click is genuine.

 

Please note that if you are using a trigger smart campaign like the one below, sometimes Marketo doesn't trigger through the audience even though they have visited web page activity on their Activity log. Therefore, I often pressure test with the filtered smart campaign after 1 day or 2 of sending the email. 

The visited page smart campaign includes: Visited page with "Link" and "member of Email Sent or click" smart campaign:
Visit web page.jpg

Our process seems pretty manual when it comes to reporting accurate email engagement. If anyone experiences a similar issue to Gergana and me, I would love to hear about it. Thank you!

KarinaVidal
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

@tcthanawan , yes, we use Marketo forms across our website and on some social ads. 

The issues you two are experiencing are very interesting. I'm fortunate not to be experiencing them at my organization. This one is challenging because you seem to have real users with bot activities due to security parameters set by their org/IT. I'm thinking about validating a user's engagement by setting up hidden links that only a robot would catch, like an email honeypot.

 

I suggest having a text link the same color as the background, perhaps in the footer. If that link is clicked, you can use it as another exclusion in your list to auto-remove that engagement from your report, similar to what @gergana mentioned about the bot exclusion criteria native to MKTO. Another criterion that can be used is to use an "Any" logic and include links that have been clicked 2+ times since bots will click links only once, but the actual recipient will click a link once, allowing an engagement of 2+ clicks to be recorded. Those two measures, I believe, will be an additional barrier to assist in determining authentic engagement over false ones. 

Things like this require creative thinking since we have no control over the inbox security set by the recipient. But, I'm curious to know your thoughts on the above ideas. If implemented, do share, as I'd love to follow this brewing case study. ๐Ÿค“



gergana
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Thanks for all the insightful suggestions so far ๐Ÿ™‚

 

@KarinaVidal, I find the honeypot link in an email to be an interesting approach. However, I'm concerned it might be problematic. From what I recall, using the same color for links and the background can trigger anti-spam algorithms in various email clients, which makes it a bit risky. I should look into this further to confirm, even though these algorithms are often a black box.

 

Another challenge with this method is that it might exclude legitimate clicks. For instance, if thereโ€™s a spike in clicks right after sending, including on the "honeypot" link, and then a contact opens the email two hours later and clicks a link for real, this contact would be excluded from my smart list if I use the "honeypot" click as a filter. It would be useful to have a constraint like "XX seconds duration between clicks," similar to the bot activity filter in admin. This would permit us do better troubleshooting.
For sure the honeypot is a great solution for forms, but in emails, it's a little bit more complicated, I think.

Regarding the second approach, there are a few points to consider:

  • Creating a smart list with "link is any" and the constraint "min number of times" = 2 will result in various situations:
    • Contacts with 1 bot activity and 1+ real click(s)
    • Contacts with 2+ real clicks
    • However, this approach will exclude all contacts (without any email security system in place that clicks all the links in a mail) that have in the log only one single real click.

Depending on our goals, this method could be useful for generating a list of potential bot clicks for manual review. But if we need it for reporting purposes, the situation becomes more complicated. ๐Ÿ™‚

  • I agree that logically a bot clicks a link only once, but upon examining bot activities in our instance, I found that this is not always the case. Here is a screenshot of a contact showing bot clicks, with the highlighted rows indicating clicks on the same link. Although you can't see the details from the screenshot, I assure you it's the same link, and it appears only once in the email. Strangely, the "higher" one has "Is bot activity" set to False, while the "lower" one has "Is bot activity" set to True.
    gergana_0-1717771122307.png

 

Looking forward to hearing more of your thoughts on this and if anyone else, besides me and @tcthanawan, has experienced similar issues.

KarinaVidal
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

All of your points are valid. Also, thanks for sharing screenshots; the visuals helped me better understand what you're experiencing. This issue is more convoluted than I imagined. I've heard about texts and links with the same color potentially causing server issues, but I've never heard, seen, or read the actual problems experienced by someone. I'll need to research and find real cases because the "honeypot" links are HTML codes focused on text design (CSS). @laurenmccormack , with the many organizations your teams have worked with, do you have any insights or experience in mitigating an issue with hidden links that match an email's background? If so, how has a sender's email been impacted?

 

I'm still curious about this issue because the links will appear clearly in the HTML email version of an email, which, from my understanding, is how these email servers scrape emails to find spam/malware. I wonder, too, if a developer can create a module or add a script in the Email editor to leverage a built-in email "honeypot." 

 Please continue to share your experience. If I discover anything that is low-risk to test before calling upon a developer, I'll continue sharing my findings @gergana .

laurenmccormack
Level 3 - Champion

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Great dialogue so far, team! Yes, bots are an ever-present challenge. I personally have used honeypots, Recaptcha, webhooks, and a sundry assortment of bot filtering capabilities that Marketo has tried to launch in the platform itself. It's an ongoing issue. I personally have an existential crisis when I fill out Recaptchas, wondering if I indeed am a bot myself when I struggle with a corner of a bicycle tire in a square. Plus Recaptcha tends to lower conversions. 

 

Anyhow, on another topic, how are you all doing with the study guide materials? Are you working your way through the content? Do you have any related questions?

 

Also, here's our Friday question of the week:

Which of the following is a special feature of the Engagement Program in Marketo?

A) Smart Campaigns: A tool for building complex marketing workflows.

B) Nurture Streams: A mechanism to automate and personalize ongoing communication with leads over time.

C) Email Deliverability: A feature to ensure emails reach the inboxes of recipients.

D) Program Tokens: Variables that can be used to customize content dynamically.

 

Lauren McCormack
KarinaVidal
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Thanks for your input, @laurenmccormack!

 

Random fact: last year, I discovered that Recaptcha only knows if we are not robots based on our cookie history. That is a way for the forms/sites to know when to have it appear or bypass the requirement on the next visit. 

 

The study guide is going very well for me. I'm learning small details I didn't know, such as how long Marketo retains information. Something like a data retention policy may seem minute, but it is super valuable when retroacting programs, values, program memberships, or auditing. 

 

As for today's answer, I believe it's:

B) Nurture Streams: A mechanism to automate and personalize ongoing communication with leads over time.

 

The stream's ability to set a cast and cadence, move users to other streams within the program, and add unique programs/emails are the distinctive components of this specific type compared to others.

 

Question:
You mentioned Email Deliverability as a feature. Is this a tool we can edit to optimize email sends, or is it related to the Email Engagement Report, where we can see deliverability and engagement stats?

gergana
Level 3

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

I agree with @KarinaVidal regarding the question of the week. The answer is B) Nurture Streams
Personally, I prefer Marketo's Engagement Programs over the "visual" journeys offered by other marketing automation tools as the Engagement Programs provide extensive flexibility with nurture streams and allow for the inclusion of smart campaigns instead of just simple emails, enabling more complex logic when needed.

Even though the Engagement Programs are my favorite, I find it very challenging to explain their logic to new team members. Most people new to Marketo are accustomed to the visual representation of a journey of other MA tools.


Regarding the Email Deliverability, if you meen the Deliberability Tools "add-on" it's a very interesting feature. Please note that this is a paid add-on (I think for all the Marketo packages). Here is some documentation.

gergana_0-1717854085774.png

The Email Deliverability tool in Marketo offers real-time tracking of email delivery and inbox placement, providing insights into email performance and deliverability issues. It also includes features for spam diagnostics and reputation monitoring, helping to optimize email campaigns and ensure they reach the intended audience. To utilize it, you have to download a "seed list" from the tool and import it into your Marketo instance. Include this seed list in your email campaigns, and you'll gain access to valuable deliverability data.

Additionally, it has a feature that allows you to create email previews, which is very useful when creating a new email template. This feature lets you test the email with the majority of email clients and on desktop vs. mobile, similar to Litmus.

laurenmccormack
Level 3 - Champion

Re: ๐Ÿ—ฃ๏ธ [Mentor Lauren McCormack] ๐ŸŒŸ Adobe Marketo Engage Community Mentorship Program 2024 ๐ŸŒŸ

Marketo offers a Deliverability Package with instances these days, I believe. I am old school and like to look up and monitor my SenderScore for my IP a few times a year to be sure I'm doing well and have not hit any blocklists: https://senderscore.org/

Lauren McCormack