SOLVED

Honeypot humbling experience for anyone else?

Go to solution
44MattWolf44
Level 3

Honeypot humbling experience for anyone else?

Hi all

I recently added a honeypot to our newsletter emails in order to filter out any false clicks that may have been from bots... Our true results, after removing the honeypot clicks, are significantly lower, about 90% lower if not more. It's been a jarring experience, has anyone gone through anything similar?

It even makes me question if the industry benchmarks we see for standard open rates, click rates, etc. are trustworthy when bot and/or spam filter clicks are taken into account

1 ACCEPTED SOLUTION

Accepted Solutions
Dave_Roberts
Level 10

Re: Honeypot humbling experience for anyone else?

 

I think this depends on your specific instance so it's probably different from place-to-place but as someone who sees a lot of different instances from mid-to-large companies across the world this doesn't surprise me at all. In my experience there is a pretty significant gap between the numbers that are reported on and the truth in those numbers -- especially when it comes to advertising the efficacy of certain tactics b/c it's sounds better to report a bigger number than an accurate one. 

 

🔥Hot take:

My personal point-of-view on this sort of scenario is that there's a lot of copy-cat type of stuff out there and when you're doing the same thing that everyone else is doing (comparing to industry standards or benchmarks) you'll inevitably fall into league with everyone else who's using similar tactics. The internet world and attention economy is moving pretty fast these days so by the time you hear about "that new things that's creating all the waves" it's already too late to capitalize on joining the moving, but that's almost never how the information is presented in webinars and self-reporting.

 

I think the other angle of this (and take it with a grain of salt, I'm a developer and not a marketer) is that there's usually an incentive to make things appear to perform better than they actually do and a lack of effort when it comes to challenging "the way it's always been done". I've worked with hundreds of companies over the past decade and this seems to be true mostly when it comes to reporting on marketing numbers (conversions, clicks, etc) b/c it just looks like you're doing your job better unless you really dig into the details which presents a sort of reverse incentive b/c it only makes it look like the stuff you've been doing has been working worse than reported. 

The good news here tho, is that it sounds like your honeypot might be working really well and that's refreshing b/c it's such a simple solution to something (bot clicks) that gets a lot of to-do made about it. There's probably more air-tight solutions like Captcha that make the user experience a bit more abrasive so I'd say if you're able to find this kind of result with a honeypot (even though the results are a bit jarring) then I'd chalk that up for as a win for the sake of not introducing unnecessary complexity into your system. Also, although it's a significant drop-off I think it's better to know that something isn't working and fix it than to carry on into the future pretending it was doing 10x what it is actually doing  -- this is the kind of thing that erodes the trust between marketing and sales alignment in the big picture.

View solution in original post

4 REPLIES 4
Dave_Roberts
Level 10

Re: Honeypot humbling experience for anyone else?

 

I think this depends on your specific instance so it's probably different from place-to-place but as someone who sees a lot of different instances from mid-to-large companies across the world this doesn't surprise me at all. In my experience there is a pretty significant gap between the numbers that are reported on and the truth in those numbers -- especially when it comes to advertising the efficacy of certain tactics b/c it's sounds better to report a bigger number than an accurate one. 

 

🔥Hot take:

My personal point-of-view on this sort of scenario is that there's a lot of copy-cat type of stuff out there and when you're doing the same thing that everyone else is doing (comparing to industry standards or benchmarks) you'll inevitably fall into league with everyone else who's using similar tactics. The internet world and attention economy is moving pretty fast these days so by the time you hear about "that new things that's creating all the waves" it's already too late to capitalize on joining the moving, but that's almost never how the information is presented in webinars and self-reporting.

 

I think the other angle of this (and take it with a grain of salt, I'm a developer and not a marketer) is that there's usually an incentive to make things appear to perform better than they actually do and a lack of effort when it comes to challenging "the way it's always been done". I've worked with hundreds of companies over the past decade and this seems to be true mostly when it comes to reporting on marketing numbers (conversions, clicks, etc) b/c it just looks like you're doing your job better unless you really dig into the details which presents a sort of reverse incentive b/c it only makes it look like the stuff you've been doing has been working worse than reported. 

The good news here tho, is that it sounds like your honeypot might be working really well and that's refreshing b/c it's such a simple solution to something (bot clicks) that gets a lot of to-do made about it. There's probably more air-tight solutions like Captcha that make the user experience a bit more abrasive so I'd say if you're able to find this kind of result with a honeypot (even though the results are a bit jarring) then I'd chalk that up for as a win for the sake of not introducing unnecessary complexity into your system. Also, although it's a significant drop-off I think it's better to know that something isn't working and fix it than to carry on into the future pretending it was doing 10x what it is actually doing  -- this is the kind of thing that erodes the trust between marketing and sales alignment in the big picture.

44MattWolf44
Level 3

Re: Honeypot humbling experience for anyone else?

Thanks for the thoughtful response here Dave. It really resonated with several of my teammates and I

Jo_Pitts1
Level 10 - Community Advisor

Re: Honeypot humbling experience for anyone else?

@44MattWolf44, there is a lot of truth in what @Dave_Roberts has said.

 

Out of interest, if you use Marketo's built in bot filtering, what kind of results do you see?

 

Regards

Jo

44MattWolf44
Level 3

Re: Honeypot humbling experience for anyone else?

We've used it on-again-off-again throughout 2023. The results are pretty similar, but what's nice about the honeypot method is, as time goes on, we can track who specifically has clicked ONLY the honeypot link throughout several months, vs. who has clicked the honeypot link AND other links as well (meaning they are showing bot activity but also real activity and should continue receiving content).