Table of contents

Background

This document provides additional details for customers that were impacted by the June 11, 2019 Communication Limits issue. “Send Email” activities are generated in real-time. These activities are stored in a queue, where our back-end process removes them on a first-in, first-out basis. The process then converts them into an aggregate count of sent emails on a per-lead basis, with an associated timestamp. We process these activities at 2,000 activities per batch.


Frequently Asked Questions (FAQ)

What email smart campaigns could have been affected?

During the impacted timeframe, both trigger and batch email campaigns could have been affected. A batch campaign with more than 2,000 leads is likely to have been affected. Trigger campaigns and batch campaigns with less than 2,000 leads could have been affected, but not as likely, as described below.

Why are batch email campaigns with more than 2,000 leads likely to have been affected?

When a large batch campaign is run, a large influx of Send Email activities are sent to the queue. For example, if a campaign sent out 10,000 emails, the back-end process would take the first batch of 2,000 activities and keep track of the email send count on a per-lead basis. After finishing the 1st batch, the tracking process normally moves on to count the next 2,000 activities. However, this issue caused the first 2,000 activities to be re-processed multiple times (up to 5 times). In this scenario, the first 2,000 leads who were sent an email would have a count of up to 5 emails sent when it should have only been 1 email. For the remaining 8,000 leads, they would not have been processed, resulting in a count of 0 emails sent to each of these remaining leads.


Why are batch email campaigns with less than 2,000 leads not as likely to have been affected?

This is because we process a batch of 2,000 Send Email activities at a time. If there were no further Send Email activities in the queue at the time of the back-end processing, this issue would not have caused activities to be re-processed.


Why are trigger email campaigns not as likely to have been affected?

Trigger campaigns only initiate under specific conditions. In the most common uses of trigger campaigns, it is unlikely that there would be an influx of trigger campaign execution that would have resulted in a large number of Send Email activities during a short period of time. For example, consider a trigger campaign that is set to send an email every time a lead fills out a form on a specific landing page. It is unlikely that there would be such a surge of leads filling out more than 2,000 forms in a short period of time, which would lead to the processing of more than 2,000 Send Email activities unless there was a big marketing effort to drive these leads to such landing page.


Can you provide a list of the trigger email campaigns that were affected?

Unfortunately, there is no way for us to obtain this information. The queue that stores such data has already been processed and the content was removed after processing.

Can you provide a list of the batch email campaigns that were affected?

We cannot recover this information due to the same reason listed above. However, the data can be obtained manually by following the steps below:

1. If you know what campaigns were run during the impacted time-frame, you can check each campaign and see whether there were more than 2,000 emails sent.

2. The Email Performance report can also be used to identify the possibly affected campaigns.

a. Navigate to Analytics > Email Performance report

b. Under Setup, set the Sent Date to the impacted time-frame based on the data center your instance is in.

London Data Center: June 11, 2019 – June 24, 2019

Sydney, San Jose, and Ashburn Data Centers: June 14, 2019 – June 24, 2019

setup.png

c. Run the report and Sort the Sent column in Descending Order to identify email sends greater than 2,000.

report.png

d. Locate the email in question and click on the Used By tab to identify the campaign(s) that were used to send this email. In addition, please ensure that this is a non-operational email (operational emails were NOT affected by this issue).

used by.png

e. Navigate to the smart campaign and Click on the Email tab to confirm that more than 2,000 leads were sent the email. 

smart campaign.png

f. Click on the Run History tab to confirm that more than 2,000 leads were sent this email in a single campaign run during the impacted time-frame.

run history.png

If you have additional questions or experience any issues, please contact Marketo support at https://support.marketo.com, or through any of the methods listed here.