"Merge Fields" Feature - The Art of the Possible and How to Execute Well

[Chris_Willis1]

If you have migrated to the new Marketo IMS (Identity Management), you may have started to notice something: Many new features are getting unlocked to you as an admin user.  I am certain that you, like me, are excited about some of these new features we are seeing from Adobe that are making our lives as admins and marketers easier.

One of these features that has been unlocked is a “Merge Fields” feature.  

When I first discovered this new Merge Fields feature, I became instantly excited because one of the issues that we have had as Marketo Engage Admins is with duplicate and/or extraneous fields.  Duplicate fields can result from a number of sources, such as errant Salesforce field setup, lax Salesforce admin field level security settings (many of which are unnecessary and can cause sync backlog issues), bespoke field additions, or fields created for prior use cases that are no longer needed.  

Prior to this feature being launched, we faced two key limitations with regards to managing our data models in Marketo Engage.  

• Fields could not be removed, but could only be “hidden” - meaning that technical debt is allowed to persist in our orgs. 
• Field duplicates resulting from CRM mapping issues - a prime example being the “Country Code” (SFDC Lead) and “Mailing Country Code” (SFDC Contact) mapping issue that results from the country/state picklists being turned on in your integrated Salesforce org.  These field duplicates are either “managed” by workflow hacks or must be merged through creating a Support ticket with Adobe/Marketo.  I have submitted my fair share of these support tickets over the years, and the level of detail required to provide support for a field merge is such that many of us opt for often broken workflow hacks.  

Enter stage left, the Merge Field capability.  With this feature, Marketo Engage admins now have an UI-driven, out-of-the-box solution for executing database field cleanup without having to submit a Support Ticket.  

With this new feature, you can as a Marketo Engage admin:

• Clean up field mapping for duplicate fields created where Salesforce lead conversion rules map the fields (such as Lead Address fields and Contact Mailing Address fields), as well as other fields that are linked via Salesforce lead conversion rules but aren’t immediately recognized by the Marketo Sync due to differences in field API and/or label names in Salesforce. 
• Remove/Reduce (read:  delete) “extra fields” in your database, by type (this will be discussed in more detail in a bit).  Field merges do not look at the field name or API name when it validates merges, which gives you a lot of flexibility when you understand the feature’s rules.  

This sounds great, but what validation rules or safeguards exist?  

I don’t expect that this list is fully exhaustive.  However, what I have found in using this feature is that Marketo Engage will return an error when you execute a field merge in these conditions.

• You can only merge fields that have the same field type, meaning you cannot merge a date/time field with a string field, or a string field with a textarea field.  To merge, the fields must be the same type (string, textarea, boolean, integer, float, score, date, datetime, etc.)

• Fields can only be merged if they exist on the same Marketo Engage object.  We don’t generally talk about “objects” much in Marketo Engage, but the database does leverage an object-based database structure (this is apparent if you export Marketo Engage data into a BI platform such as DOMO or Tableau).  What this means for field merges is that you cannot merge a company field with a person field (and vice versa).  Marketo Engage, under the hood, sees these fields on different database tables, and therefore can’t merge them.  

Fields included under “Company Info” will be linked to the Company object.  Fields included under the “Info” tab will be linked to the Person (lead) object, and SFDC Custom/Standard fields will be linked to the Marketo Engage object aligned to the Salesforce object the field is linked to in Field Mapping.  Fields linked to Leads and/or Contacts in Salesforce will be linked to Person records and those linked to Accounts will be linked to Company records.  

• Fields cannot be merged if the merge action results in errant Salesforce field mappings.  If you have one field that is mapped to a Salesforce lead/contact record, it cannot be merged into a field that has different Salesforce lead/contact field mappings.  As a best practice, I recommend that you only merge in these two scenarios.  
• The merge (non-surviving) field in your field merge is not mapped to any Salesforce fields, OR
• The merge (non-surviving) field in your field merge is mapped to a Contact record that is linked in the Salesforce lead conversion mappings, and the target (surviving) field is mapped to the Salesforce Lead version of the field (meaning, the resulting merge will link the Salesforce Lead field with the resulting converted Contact field).  For example, this field merge of Longitude (SFDC Lead) and Mailing Longitude (SFDC Contact) resulted in this fixed Salesforce field mapping.

IMPORTANT NOTE:  Salesforce field mappings are NOT removed when your Salesforce admin removes Field Level Security (FLS) access to the field from Marketo Engage.  You will see the CRM Field Map be removed in Field Management.  However, the Field Mapping metadata from the initial sync will persist in the Field Mapping module.  It is this mapping data that Marketo Engage uses to determine whether a merge will be allowed, versus the real-time data in the Field Management module.  Marketo Engage uses the Field Mapping as the source of truth for field mappings, and this data is not updated when a field is no longer visible by the Salesforce Sync.  

Field mappings can be removed by Adobe Support via a support ticket.  If you have a large number of fields you removed from the sync and want to clean via a mass-merging project, you can submit a list to Support to remove the field mapping.  In your support ticket, you will need to include:

• The precise field names (I recommend including friendly labels and API names) you want to unmap
• You will need to remove field references from all assets (Smart Lists, Smart Campaigns, email tokens, etc.)
• If you wish to retain data, make a backup of the field data (either offline or in a backup field) as Adobe Support cannot guarantee that data loss will not occur in the unmapping exercise.  This may not be necessary for fields you wish to destroy through the merging exercise. 

Once you have confirmed that the fields you are merging do not have conflicting field types, objects, or Salesforce field mapping, you are ready to execute your merge.  To execute the merge, first navigate to the Field Management module in Admin.

Then, search for the field you wish to merge.  This field will be the “merged” (or non-surviving) field.  For this example, we will be merging TEMP_String_PersonField6 DupeField1 (merge) into TEMP_String_PersonField6 (target). 

Select the merge (non-surviving) field in field management.  Then navigate to Field Actions, and choose the Merge Fields option.

You will receive a dialogue window.  It will ask you to identify the target (surviving) field and choose a winner in the event that a data conflict exists.   

Once you have confirmed your choices, click the Merge button.  Note, that once you click this button, you cannot undo, so be sure that the instructions you provide are your final answer, because once you click the orange button, like Neo taking the “red pill” –  there is no turning back. 

What happens now? 

If an issue exists which prevents a merge, you will get an error message.  In the case of a field mapping issue, the error will look like this (naming the Salesforce objects causing conflicts): 

In the case of an object conflict issue, the error will look like this (naming the Marketo Engage objects causing conflicts):

If no conflicts exist, Marketo Engage will start to execute on the field merge and will work to reconcile any data conflicts that exist between the two fields being merged.  The amount of time required to execute the merge depends on the data contained in the merged (non-surviving) and target (surviving) fields and the number of data conflicts that Marketo Engage needs to reconcile.  

What should I expect for data conflicts?

Let’s run through a merge conflict scenario:

In this example, Peyton Manning has conflicting data in the fields being merged.  

When TEMP_String_PersonField6 DupeField1 is merged with TEMP_String_PersonField6 and the merge (non-surviving) field is chosen as the winner for data conflicts, the resulting surviving value will be “Indianapolis Colts”

Conversely, if TEMP_String_PersonField6 DupeField2 is merged with TEMP_String_PersonField6 and the target (surviving) field is chosen as the winner for data conflicts, the resulting surviving value will be “Papa Johns.”  

If the value of TEMP_String_PersonField6 was instead blank, the resulting merge would result in the value being “Denver Broncos.”   

In the event that one field has data, but not the other, the resulting merged field will retain the completed field data

In this case, regardless of the surviving field chosen, the resulting merge will place “Obi Wan Kenobi, you’re my only hope” in TEMP_String_PersonField6.  

In any scenario where you merge fields with conflicting values and you may want to retain data destroyed via merge conflicts, always create a backup of the non-surviving data so you can reconcile and backfill any data you want to retain.  I do especially recommend this action in the event that you merge fields to fix Salesforce lead/contact lead conversion mapping issues (such as Country Code, Mailing Country Code). 

You will want to note that when data is updated in a surviving field as a result of merge, you will NOT see a record in the activity log indicating a Change Data Value.  You will need to evaluate by reviewing a set of records to confirm the results of your field merge.  If you consider that the Activity log is a database table in Marketo Engage, you can quickly see that having the merge add activity log records for field merges would add significant processing time to your merges.  Be aware of this fact. 

Again, here’s the good news… It is true that the Field Labels and API Names for fields do NOT have to match in order to merge, only the field type and the CRM field mappings for the final outcome need to align.  You can, therefore, delete unwanted fields via this feature by merging a field you don’t want to retain with a field you wish to retain (such as a general purpose “TEMP” field for campaigns, as the example above shows)... this action will delete the non-surviving field.  

When using the Merge Field function to delete fields, I do recommend that you clear out all legacy data (as well as asset references, which is required) from the field prior to executing the merge so that the merges are executed quickly and that legacy, technical debt, data does not persist into your target field(s).  As a best practice, create a Marketo-Only field in your database that can serve as a virtual “trash can” field (create one by type), in which you will merge/delete all of your unwanted fields into this “trash can” you are using.  The example I shared above is an example of creating a field for the express purpose of destroying unwanted fields in your database.  

I hope this instruction serves to enable you and give you confidence with executing field merges.  This is a very useful feature when used well, and can create great value.  To all of my fellow Marketing Ops nerds who are excited about Kon-Mari’ing your org, I wish you success!