Filtering Email Bot Activity Feature V2

Sreekanth_Reddy
Marketo Employee
Marketo Employee

May 2022 release details: https://nation.marketo.com/t5/product-documents/filtering-email-bot-activity-feature-may-2022-releas...

 

With a focus to address bot's impact on Email engagement, in 2021 we have launched our first version of filtering email bot activities. We have seen that this version of the solution is capturing an average of 2 Million bot activities per week for nearly 1000 Marketo customers. The impact is 3% of all email clicks and opens logged, while the industry average is ranging from 15% to 30%.

 

We got feedback from a lot of customers and from the Marketo community. We have categorized the feedback into the following themes:

 

Themes

Description

More Transparency

The current version identifies the bot and doesn't log activities identified as bots.
Almost every customer and community expressed they would like to log the activities and do their own analysis

More Robust filtering

The current version completely relies on the IAB list and we need to introduce more sophisticated filtering to identify bot activities

More Control

As a customer, I find few user-agents/IPs that are creating bot activities and would like to include them in bot activity filtering criteria



We have been working on enhancing the feature that is more sophisticated patterns and gives more control to customers. As part of this effort, in the March release we have made the following enhancements:

 

  • Now Marketing admins have the option to opt for logging activities that are identified as bot
    Sreekanth_Reddy_0-1658838279460.png
  • We have added 2 new attributes to email click, email open activities
    • “Is Bot Activity”, is a boolean value (True/False) and will be marked as true if the activity is identified as bot activity
    • “Bot activity pattern”: To highlight the pattern respective activity matched with. In this current release, all activities that are identified as bot will have value “match with IAB bot list”
      Sreekanth_Reddy_1-1647419972936.png
  • These attributes are available as constraints in filters and trigger of these respective activities i.e Clicked link in email, Clicks link in email
    Sreekanth_Reddy_2-1647420156043.png

     

Useful Links:

17342
33
33 Comments
Sreekanth_Reddy
Marketo Employee

Hello @Aslihan_Ayan_Ha , instead of focusing on updating all reporting interfaces we are working towards making the filtering algorithm sophisticated enough so that all customers can completely eliminate logging activities by bots. Do you think this is better approach or we should consider providing an option in all possible reports that are being impacted by bots?

Katja_Keesom
Level 10 - Community Advisor + Adobe Champion

@Sreekanth_Reddy I would definitely provide the option in reporting to either in- or exclude the bot activities. That makes it easier to assess the impact of the bot activities and compare against previous results.

Sreekanth_Reddy
Marketo Employee

Hey @Katja_Keesom, Interesting! I am curious to know that since bot activities are junk data why do we want to analyze them rather exclude them out completely?

Katja_Keesom
Level 10 - Community Advisor + Adobe Champion

@Sreekanth_Reddy there will always be a cut over from the reporting with all bot activities included to the new reporting. In order to understand sudden trend breaches better it would be good to be able to in/exclude the activities flexibly to see whether they are the cause of the change or not. Also, it could be good to understand which customers have active mail scanners to judge behaviour better. Finally, it would be good to analyze the bot activities and make your own judgement on whether it was correctly flagged as such so we can judge which types of bot activities to drop from being logged altogether. Hope this helps.

hpavlac
Level 2

@Sreekanth_Reddy  to follow up on some of the previous conversations - if we are logging Bot Activity, do bot clicks show as part of Email Link Performance Reports and Email Program Summary Views? Is there a way to filter them out?

Sreekanth_Reddy
Marketo Employee

Hey @hpavlac , we haven't made any changes to any reporting yet. So, yes all activities will continue to be part of the report but you can consider using new constraints we added at email open and email link click activities.

Screenshot 2022-08-17 at 10.06.38 PM.png

CarrieTaylor
Level 2

When was this newer feature for bot activity rolled out?  We have seen an increase around the week of July 24th for a couple of our newsletters.

Joshua_Patnod
Level 1

Hi @Sreekanth_Reddy - 

This new feature is going to be key to weeding out bot activity from scoring related to email opens and clicks so thank you! 

Are there plans to include the "Is Bot Activity" field as a filter option in email performance reports? Similar to the standard columns for Soft and Hard Bounces, it would be great if a similar column could created for Bot Opens and Bot Clicks. 

  • Marketing Activities - Email Performance Reports
  • Analytics - Email Performance Reports
  • Revenue Cycle Explorer - Email Performance Reports

Thanks again for the great overview. 

Josh

Sreekanth_Reddy
Marketo Employee

@CarrieTaylor IAB list was launched in May 2022 and Proximity pattern (log mode only) - July 15th 

Sreekanth_Reddy
Marketo Employee

@Joshua_Patnod Awesome! Thank you for the detailed feedback. We are as of now focusing on improving our bot activity detection and post this I will have your reporting feedback as consideration for the roadmap. 😃