Email Bot Activity constraints impacting Email Performance Report Metrics

[John_Potter1]

Hello Marketo Nation Community,

I'm an Admin in a client's Marketo subscription where we have been trying to understand a couple of things on how the "filter bot activity" setting impacts Email Performance Reports (EPR).

Just for context, since implementing the Match with IAB List / Match with Proximity Pattern features, the client decided to "Filter Bot Activity" for both options as they wanted as much of this bot activity removed from activity logs as possible. Done. However, we are still seeing some activity that may look bot related such as several clicks in email footer links that the proximity pattern is just not catching. It's expected that it will not be 100% accurate, right? But over time, this will improve with the technology.

But, knowing we are "filtering out all bot activity" at the admin level, why are we seeing a significant impact to "Sent/Delivered" metrics and slight discrepancies within "Opened/Clicked" numbers when we experiment with adding "Is Bot Activity is false" constraints to Opened/Clicked filters within the EPR Smart List? If these activities are being filtered out, then adding such a filter and constraint combinations should have no bearing on the resulting counts, correct?

Here's an example:

A standard EPR where ONLY a Region filter is being used to target AMER - notice the 10k+ counts for Sent/Delivered as well as the counts for Opened:

Now, when adding the 2 Smart List filters: "Opened Email Is ...Is Bot Activity False" or "Clicked Email Is...Is Bot Activity False" as seen below....:

The Sent/Delivered is a quarter of the count it originally was and the Opened has increased by 2 - and yes, the 4 emails listed in the filters are the same 4 emails shown in the report:

Here's another example for the same campaign but for the EMEA region:

Before adding the bot filter constraints:

After adding the same 2 bot filter constraints seen in the example above:

One would think the resulting numbers should be identical if bot activity is being filtered from the activity logs. It would be super helpful if someone could possibly shed some light into why the significant and slight discrepancies we are seeing

Much appreciated.

Thanks!

[Darshil_Shah1]

Well, you are technically correct! One should see no difference if they filter out the bot activities from their instance; however, did you enable filtering bot activities from the start, or were you initially on log bot activities and then switched to the filtering option? This could impact the results, as there could be a case wherein Marketo is referencing the activities from before the options were switched. Additionally, you're referencing the email names and links in the multi-value chooser, which operates in the OR condition, i.e., if any email open/click activities would qualify for the smart list criteria.

[John_Potter1]

Hi Darshil_Shah1,

Appreciate your reply. The "Filter Bot Activity" Admin setting for BOTH Match with IAB List and Match with Proximity Pattern have been toggled on for more than 1+ year. This particular campaign has only been running for 1-month. This Marketo subscription is only tracking "Is Bot Activity is False". I know this because, when creating a Smart List with the below criteria, the results come back as 0.

Now, if this Marketo subscription was logging the bot activity, as uditmathur mentions below, yes, I would see a drop in Opens & Clicks when applying such filters & constraints, however, in this case, the Opens and/or Clicks marginally increase by +1/+2 or so when adding the "Is Bot Activity is False" Smart List constraints, when there should be absolutely no impact whatsoever. If you have any insight on this, it would be great to know why things are changing ever so slightly.

Regarding your secondary comment around referencing the email names and links in the multi-value chooser, yes silly me, it explains why the Sent/Delivered counts are so drastically different. For example, in the 2nd EPR screenshot I posted after applying the filters, the 915 people sent the 1st email (line 1) have at least opened OR clicked in ANY of the 1st 4 emails that have deployed. The Sent/Delivered numbers in this scenario will never be higher than the overall "Opened" count, which in this case is 1269. That makes sense. Thanks for helping me think through this one.

I'm going to keep this open a little while longer to see if anyone replies on the marginal increments of Opened/Clicked metrics when using these filters when bot activity filters are indeed enabled and have been for some time.

Thanks for the support!

JP

[SanfordWhiteman · Accepted solution]

Is Bot Activity=false includes those log entries where the Is Bot Activity property is not set at all, i.e. undefined.

[John_Potter1]

Hi Sanford,

Appreciate the feedback and this makes sense. There very well could be companies not using filtering services for their inbound/outbound email trafiic in which case Marketo cannot define their "Is Bot Activity", however, as you mention, they will appear as if they were "false" when such filters/constraints are used. Good to know!

Thanks

JP

[uditmathur]

Hi,

Email send/delivered is nothing to do with Bot Activity filter as @Darshil_Shah1 mentioned it might be because you switched from log activity to filter activity.

But definitely your clicks & open will drop if you choose to Filter Bot Activity.