The test is complete and I've revised this post to reflect the results. Overall, the test results were as expected. We have about 150K records that we sent to over the last month. Out of 215 emails that Marketo marked "email invalid = true," 30 are indeed invalid and 35 are good (delivered to). 31 of the 35 good emails opened any email sent in the past 30 days. 11 clicked a link (not unsubscribe, but articles). The remaining 150 out of 215 records are still being tested (in a wait step).
So now I'm going to batch update and re-validate all the records that Marketo marked invalid. The time period will be since we started with Marketo and before this test. I'll do this in small increments so deliverability won't be an issue. By the way, I had to do this exact same thing in Exact Target. At that time I found that 33K out of 100K records that E.T. marked bad were actually good and they had typical open/click rates.
Here's the background that I originally posted:
A customer recently complained that they stopped receiving emails. I checked their record and Marketo marked their email invalid due to a hard bounce. However, they reached out to me via their "hard bounced" email.
I wonder how prevalent these false positive hard bounces are, so I set up the following campaign and will continually monitor. I'm hoping this 30-day test period will be enough time for recipients to clear out their full inbox so false positives won't occur.
Our sender reputation is excellent, so we can withstand sending to bounced emails without hurting our reputation as long as it's done in small quantities.
Here's the test ...
Campaign: Test invalid records for 30 days
Description: Marketo marks emails invalid via hard bounce, spam bounce or technical bounce. Some people get a false-positive hard bounce and stop receiving emails they want. This campaign tests recent invalid emails who have recently opened or clicked, who haven't already gone through this test process and excludes category 1 (spam) bounces. We "Email Invalid" to true for these records and test for 30 days to be sure they're bad. After the 30-day wait step the "Mark bad records" campaign will request all the members from this campaign and check records sent to but not delivered to and then the flow step will mark these emails as legitimately invalid. Anyone delivered to will be ignored and continue receiving emails.
Smartlist
Notes:
- I filtered out people who haven't opened or clicked in 3 months because I only want people who are engaged, but I might extend that to 6 months.
- "Not email bounced" is just filtering out spam bounces. A category 1 bounce is spam so obviously, I want to ignore these records.
- Step 4 is essentially doing the same thing as step 3 — just a redundancy to be safe.
- Step 5 basically says exclude bad records that have already been tested. The "Tested invalid record for 30 days and sent email was not delivered" value is set in another smart campaign.
- Step 6 is another redundancy. It excludes people who have already been tested. If they received the email (step 7 in the flow below) they've already been through the cycle and can be ignored.
Everyone else who meets the criteria goes to the...
Flow
Step 5 sends them to the "Mark good records as valid" campaign which changes their "Email Invalid Cause" field to "False-positive bounce. This email is good." if an email was successfully delivered to in the past 30 days.
Step 7 sends an email at 6pm (a time of leisure) to those who haven't already been marked good from step 5 and who didn't happen to receive any other emails during the past 30 days. If they didn't receive any emails in the past 30 days, we need to send them a test email to determine if it gets delivered. So they get this email copy:
Do you want to continue receiving our emails?
Our records indicate that your email bounced (i.e. wasn't delivered to). You may have had a full inbox. It could be that your email client was down, but either way, we're just double checking to see if you would still like to receive our messages. Do nothing and we'll keep sending. If you'd like to scale back or discontinue receiving messages, feel free to update your preferences now.
If this email is delivered, then step 8 will send them through the "Mark good records as valid" campaign which is shown below:
"Mark good records as valid" smart list:
"Mark good records as valid" campaign flow
Everyone else goes to step 10 which sends them to the "Mark bad records as invalid" campaign.
"Mark bad records as invalid" smart list:
Flow
.png)