Filtering Email Bot Activity Feature V2

[Sreekanth_Reddy]

May 2022 release details: https://nation.marketo.com/t5/product-documents/filtering-email-bot-activity-feature-may-2022-releas...

With a focus to address bot's impact on Email engagement, in 2021 we have launched our first version of filtering email bot activities. We have seen that this version of the solution is capturing an average of 2 Million bot activities per week for nearly 1000 Marketo customers. The impact is 3% of all email clicks and opens logged, while the industry average is ranging from 15% to 30%.

We got feedback from a lot of customers and from the Marketo community. We have categorized the feedback into the following themes:

Themes	Description
More Transparency	The current version identifies the bot and doesn't log activities identified as bots. Almost every customer and community expressed they would like to log the activities and do their own analysis
More Robust filtering	The current version completely relies on the IAB list and we need to introduce more sophisticated filtering to identify bot activities
More Control	As a customer, I find few user-agents/IPs that are creating bot activities and would like to include them in bot activity filtering criteria

We have been working on enhancing the feature that is more sophisticated patterns and gives more control to customers. As part of this effort, in the March release we have made the following enhancements:

• Now Marketing admins have the option to opt for logging activities that are identified as bot
  
• We have added 2 new attributes to email click, email open activities
  • “Is Bot Activity”, is a boolean value (True/False) and will be marked as true if the activity is identified as bot activity
  • “Bot activity pattern”: To highlight the pattern respective activity matched with. In this current release, all activities that are identified as bot will have value “match with IAB bot list”
    
• These attributes are available as constraints in filters and trigger of these respective activities i.e Clicked link in email, Clicks link in email
  

Useful Links:

• Product Docs
• Email bot activity filtering update 2021 
• Email bot activity filtering FAQs

[Sreekanth_Reddy]

Hello @Aslihan_Ayan_Ha , instead of focusing on updating all reporting interfaces we are working towards making the filtering algorithm sophisticated enough so that all customers can completely eliminate logging activities by bots. Do you think this is better approach or we should consider providing an option in all possible reports that are being impacted by bots?

[Katja_Keesom]

@Sreekanth_Reddy I would definitely provide the option in reporting to either in- or exclude the bot activities. That makes it easier to assess the impact of the bot activities and compare against previous results.

[Sreekanth_Reddy]

Hey @Katja_Keesom, Interesting! I am curious to know that since bot activities are junk data why do we want to analyze them rather exclude them out completely?

[Katja_Keesom]

@Sreekanth_Reddy there will always be a cut over from the reporting with all bot activities included to the new reporting. In order to understand sudden trend breaches better it would be good to be able to in/exclude the activities flexibly to see whether they are the cause of the change or not. Also, it could be good to understand which customers have active mail scanners to judge behaviour better. Finally, it would be good to analyze the bot activities and make your own judgement on whether it was correctly flagged as such so we can judge which types of bot activities to drop from being logged altogether. Hope this helps.

[hpavlac]

@Sreekanth_Reddy  to follow up on some of the previous conversations - if we are logging Bot Activity, do bot clicks show as part of Email Link Performance Reports and Email Program Summary Views? Is there a way to filter them out?

[Sreekanth_Reddy]

Hey @hpavlac , we haven't made any changes to any reporting yet. So, yes all activities will continue to be part of the report but you can consider using new constraints we added at email open and email link click activities.

[CarrieTaylor]

When was this newer feature for bot activity rolled out?  We have seen an increase around the week of July 24th for a couple of our newsletters.

[Joshua_Patnod]

Hi @Sreekanth_Reddy - 

This new feature is going to be key to weeding out bot activity from scoring related to email opens and clicks so thank you! 

Are there plans to include the "Is Bot Activity" field as a filter option in email performance reports? Similar to the standard columns for Soft and Hard Bounces, it would be great if a similar column could created for Bot Opens and Bot Clicks. 

• Marketing Activities - Email Performance Reports
• Analytics - Email Performance Reports
• Revenue Cycle Explorer - Email Performance Reports

Thanks again for the great overview. 

Josh

[Sreekanth_Reddy]

@CarrieTaylor IAB list was launched in May 2022 and Proximity pattern (log mode only) - July 15th 

[Sreekanth_Reddy]

@Joshua_Patnod Awesome! Thank you for the detailed feedback. We are as of now focusing on improving our bot activity detection and post this I will have your reporting feedback as consideration for the roadmap. 😃