How does your team calculate email metrics?

[stacenic]

Hi All!

We recently ran a team meeting & realized there were two schools of thought for looking at email metrics within Marketo, and I wanted to see how others have approached this. 

When looking at a program, you can create a report within the program to look at email performance. Do people look at the bolded numbers at the very bottom for their average? Or do you calculate on your own? For instance: 

               Delivered        Opens       % Opened

Email 1:         5795         1850              31.9%

Email 2:         5357         1350              25.2%

Email 3:         5330         1051              19.7%

TOTAL         16,482        4251              25.8%

Do you look at your average open rate as 25.8% OR do you look at it as 25.6% (the average of the three open rates)?

When looking at industry benchmarking - how is that typically calculated? 

[Dave_Roberts]

Im not sure about industry standard benchmarks or how to compare, but it looks like there's some funny math going on here that's leading to some confusion (and this kind of thing happens a lot in reporting stats in my experience). 

The % Opened seems to be a rounded quotient of totaled Opens divided by totaled Delivered.

The Average posted in the TOTAL line seems to be a rounded quotient of TOTAL Opens divided by TOTAL Delivered (0.25791772843 or about 25.8% rounded up). 

The 25.6% number seems to be a "derived" value that is based on the rounded average quotient of each line-item so I'd expect it to be a little off compared to the "real numbers" b/c it's rounding all the add'l numbers off to make a percent and then taking an average of the rounded numbers.

In this case, the difference is quite small (0.2%) but that can be a BIG difference when looking at a database with a large number of leads. 

While it's not something that I'm normally responsible for (and admittedly a bit outside my area of expertise) I'd suggest using as many "real numbers" to perform calculations like this -- especially if 0.2% is enough to make an impactful difference based on your audience size. It seems to me that the 25.6% number is "close" but not exact, so if you're looking for exact, I'd just check the math in these situations and go with the one that is most "true" (or the least derived) to get to a workable number. 

Sometimes it's just a matter of using the "real numbers" (Opens, Delivered) to do the math rather than the "derived numbers" (percent open column) to do the math even though it seems like it saves you a step.

I think it's always good when working with numbers to double check stuff like this - it's really easy to get bad info based on bad math and then make bad decisions based on artificial numbers (email clicks comes to mind). Kudos for looking into this to get your team aligned and on the right track!

[Robert_Villanue]

As for our team, we would go with 25.8%.

If you check the internet, there's an existing topic about why taking an average or averages is wrong.

"These numbers rarely match because taking an average of averages is wrong. The reason an average of averages is wrong is that it doesn't take into account how many units went into each average."

Reference:

https://tinyurl.com/v9un4k2t

[Katja_Keesom]

It is extremely dangerous to work with an average of the (indeed already rounded) percentages. In your example each of the three emails has a similar amount of emails delivered, so the two results only differ slightly as expected.

But imagine one or two of those emails are sent to roughly 500 instead of 5.000 people? Averaging the three open rate percentages would then give those smaller emails an equal weight in the final result, which is entirely unjustified by the volume sent.

For that reason alone I would ALWAYS recommend to use the mathematical approach and take the average based on total delivered and total opens, as the bottom line in your table shows.