This document provides an overview of the recent lead Change Score issue that impacted a subset of customers across our data centers beginning on June 11, 2019. All customers that were impacted were emailed details of the incident on June 20, 2019. If you are unsure if your instance was impacted, please reach out to Marketo support at https://support.marketo.com.
Impacted Timeframe by Data Center:
- London data center, June 11, 2019, 4:00 PM PDT - 5:00 PM PDT
- Sydney data center, June 14, 2019, 3:30 PM PDT - 4:00 PM PDT
- San Jose & Ashburn data centers, June 14, 2019, 6:30 PM PDT - 7:00 PM PDT
How to identify what data center your instance is located in:
You can deduce the server locations by looking at the URL of your instance. The CNAME=the pod or server your instance is on.
The example above indicates that this instance is in our Ashburn data center.
The abbreviations for all our data centers are as follows.
lon = London
sn = Sydney
sj = San Jose
ab = Ashburn
Service affected:
A subset of Marketo customers may have experienced lead score changes that were not being updated. Impacted customers would have received campaign failure notifications in their Marketo subscription. Change Score flow steps that were set to update lead scores would not have been triggered during the impacted time-frame, however, no other flow steps were affected and the remainder of the campaign would have executed as expected.
What happened:
This issue originated from an internal logic error during a recent release. This error caused a delay between the database schema update and the code release, resulting in lead score change triggers to be missed.
Remediation:
Once the cause was identified, our team immediately began to identify the root cause and potential fixes. Our team created this documentation to educate customers on how to replay the impacted flow steps to update the lead score as needed.
Campaigns that were not triggered during the impacted timeframe will not be automatically replayed. However, customers can create their own smart campaign to update the lead scores as shown below.
Replay Impacted Smart Campaign:
1. Navigate to the Notification center in your instance.
2. Once in the Notification center, look for any campaign failures during the timeframe that the data center your instance is in was impacted as listed above.
3. If the failed campaign contains score changes, then only the Change Score flow steps failed. Every other flow step in that campaign would have worked correctly. You will need to re-run the Change Score flow steps for the leads listed in the campaign failure notification.
If you experience any issues with replaying the impacted smart campaigns, please contact Marketo support at https://support.marketo.com, or through any of the methods listed here.
.png)