externalCompanyId Service Issue

AnneMarie_C · ‎08-20-2020

This document provides an overview of the recent service issue that impacted Marketo customers beginning on July 28, 2020, or July 31, 2020, depending on the data center they are located in. If you are unsure if your instance was impacted, please contact Marketo Engage Support at https://support.marketo.com.

A bug introduced in the 2020 Q3 release inadvertently caused duplicate externalCompanyIds to be created in customers’ Marketo Engage instances. When a lead was reparented to a different company via externalCompanyId, the losing company would have also been updated with the same externalCompanyId. This would have caused two companies in Marketo to have the same externalCompanyId. This behavior violates the uniqueness of such value.

Below are two scenarios that should help depict how your data may have been affected by this issue.

Scenario 1

A lead may not have been initially associated with a company with externalCompanyId.

Create a new lead using API, form fill, import, or any other method with example email (i.e. testmarketo@adobe.com) without an externalCompanyId. Assume that this lead would have the following values. Please note a Marketo Company is always created behind the scenes even if no externalCompanyId or company fields are present.
1. Marketo Company (aka Org) Id=2
2. Marketo Lead Id=2 associated with Marketo Company/Org Id=2
3. ExternalCompanyId=NULL (i.e. Empty Value)
Associate the lead to a known company (Adobe) using create/update Lead API where an externalCompanyId is specified. Adobe company has the following values:
1. Marketo Company/Org Id=1
2. ExternalCompanyId= ABCDEFG
3. Name = Adobe

Payload of create/update lead

{
"lookupField":"email",

"input":[

{

"email":"testmarketo@adobe.com",

"externalCompanyId":"ABCDEFG"

}

]

}

The lead would successfully be reparented from Marketo Company Id=2 to Marketo Company Id=1 which is the expected behavior. However, it would also create the same externalCompanyId for the losing organization with Marketo Company Id=2.
1. Marketo Company/Org Id=2 with externalCompanyId=ABCDEFG (Incorrect)
2. Marketo Company/Org Id=1 with externalCompanyId=ABCDEFG
Let’s say you update another lead under Marketo Company/Org Id=1 with the externalCompanyId specified. It’s possible that the lead would be reassigned from Marketo Company/OrgId=1 to 2 because that company was the most recently updated. Our logic picks the most recent company/organization to be updated when there are duplicate externalCompanyId.
If you update the Company fields (ex. City=San Jose) along with externalCompanyId=ABCDEFG using the create/update Companies API call, the new updates would go to Marketo Company/Org Id=2 rather than 1. This behavior is incorrect. This is again due to Marketo Company/Org Id=2 being the most recently updated company/organization.

Below is an outline of what the data would look like in the Marketo Engage database.

Initial Dataset

Mkto Org Id	Name	City	External Company Id	Lead Id	Lead Email Address
1	Adobe	San Mateo	ABCDEFG	1	johndoe@adobe.com
2	NULL	NULL	NULL	2	testmarketo@adobe.com

Final Incorrect Dataset caused by the bug (after Step 5)

Mkto Org Id	Name	City	External Company Id	Lead Id	Lead Email Address
1	Adobe	San Mateo	ABCDEFG	2	testmarketo@adobe.com
2	NULL	San Jose	ABCDEFG	1	johndoe@adobe.com

What would have been the correct behavior?

Mkto Org Id	Name	City	External Company Id	Lead Id	Lead Email Address
1	Adobe	San Jose	ABCDEFG	1	johndoe@adobe.com
				2	testmarketo@adobe.com

Scenario 2

A lead might have been reassigned from one company to another due to job change. As an example, let’s say the following data is in your system.

Mkto Org Id	Name	External CompanyId	Lead Id	Lead Email Address
1	Company A	123	1	test@companyA.com
			2	test2@companyA.com
2	Company B	234	3	test@companyB.com

Assume Lead Id=2 now works for Company B and the lead was updated to be associated to company/organization with externalCompanyId=234 and a new email address. The lead would have been reparented successfully. However, Company A would have their externalCompanyId updated to 234 as well, which would have resulted in the following.

Mkto Org Id	Name	External CompanyId	Lead Id	Lead Email Address
1	Company A	234 (Wrong Value)	1	test@companyA.com
2	Company B	234	2	test2@companyB.com
			3	test@companyB.com

How to possibly fix the data discrepancy?

As indicated in the examples above, it is not possible for Marketo to determine an appropriate data fix since the correct externalCompanyId is no longer available. In addition, it’s difficult to say which leads should be associated with a certain company or create a formula to determine the winning company. Your company’s CRM system should be the source of truth for this information.

While we can’t fix this data ourselves, we can provide the list of affected leads, companies, and Change Data Value activities for the field Contact Company. This list can be obtained by submitting a case to Marketo Engage Support.

There are three files that will be provided to you.

Activities Data – Leads that were reparented to a different company/organization during the affected timeframe. This spreadsheet will contain the following columns.
1. lead_person_id = Marketo unique ID of the lead
2. old_org_id = Original Marketo unique ID of the company/organization
3. new_org_id = New Marketo unique ID of the company/organization
4. old_org_ext_company_id = Original company/organization’s externalCompanyId
5. new_org_ext_company_id = New company/organization’s externalCompanyId
Companies Data – All the companies that might have been affected as a result of this issue. This spreadsheet will contain the following columns.
1. id – Marketo unique ID of the company/organization
2. name – Name of the Company
3. created_at – When the company was created
4. updated_at – When the company was last updated (at the time the data was generated)
Leads – These are all the leads that belong to the affected organizations. NOTE: Not all leads were affected, but we are providing as much data as possible to assist you.
1. id – Marketo unique ID of the lead
2. email – Email address of the lead
3. link_company_id – This is the Marketo unique company/organization ID that points the ID field of the companies data file (1)
4. created_at – When the lead was created
5. updated_at – When the lead was last updated (at the time the data was generated)

In order to perform the data fix on your own, you will first need to determine which company should be the winner of the duplicates and reparent leads to the correct company/organization, if needed. This can be done using a combination of Marketo Leads and Companies API.

In cases where the loser organization shouldn’t have an externalCompanyId, the data fix could follow the steps below.

Reassign the leads under the loser organization to a dummy organization.
Delete the loser organization using the Delete Company API by idField. The IDs are provided in the CSV files Support will provide. NOTE: Deleting the organization doesn’t delete the associated leads’ activities since activities are recorded at the lead level.
Reparent the leads from the dummy organization to the winner organization.
Resync the organization’s data from your CRM to Marketo Engage in case some fields are not up to date.

In cases where the loser organization’s externalCompanyId was updated to have the same value as the winner organization, the loser organization is an actual company that should be retained in the system.

Recreate the loser organization using the right externalCompanyId from your CRM system.
Reparent the leads from loser organization to this new organization.
Delete the loser organization using Marketo idField. NOTE: Deleting the organization doesn’t delete the associated leads’ activities since activities are recorded at the lead level.

If you have any additional questions, please Contact Marketo Engage Support.