Big Marketo, Pt 2: Use static lists instead of creating fields

Robb_Barrett
Marketo Employee
Marketo Employee

Here's a piece I recently wrote up to share with some colleagues who felt that they needed to add a lot of custom fields to our Marketo DB for one particular product line in one particular region.  The problem here is that we have so many product lines in so many regions that giving this access to one opens up the floodgates to all.  The problem is that I'm not a fan of sparsely populated data fields - fields that have only a small records who will have something other than the default value.

If the field can't work for everyone everywhere, I'm not interested in adding it to the database.  With that, here's my justification for using static lists instead of custom fields.

Using Flex Fields and Static Lists to act as custom fields

This document will show how to use temporary fields, “Flex Fields”, on forms in place of creating custom fields and then store the data in either Static Lists of note fields for permanent, long term storage and usage.

Flex Fields: We allow four flex fields available for use by all Marketo users. These are called Flex Field 1, Flex Field 2, Flex Field 3 and Flex Field 4. These can be used to store any type of information temporarily – for a half hour – which is enough time to move the data to either a Note field or a Static List for segmentation purposes.

Types of Data: there are 3 main types of questions and data:

  • Finite Choices: These are questions that have a limited number of answers such as Yes / No or Multiple Choice
  • Scalar Data: These are data that use a number to represent a scale. Examples would be number of users, ratings, etc. If this information can be grouped – such as “From 1 to 100” or “Less than 100” or “More than 9000” it can be stored in lists for those ranges: 1-100, 101-500, 501-1000, etc. These ranges can be unique for each form and product line.
  • Free Form Text: This is data that the form respondent supplies by typing in an answer. This data is not standardized, may contain typing errors and cannot be used for reporting.

Fields vs. Lists:

Fields are points of data that exist on every record in every region in the database. Fields tend to be limited to data points applicable to every lead in every region or at least enough leads to compromise a significant amount, or are used by numerous product lines and regions.  While information like Name, Email Address, Phone Number, Company Name and the like are applicable to every lead some information like Number of Operating Theaters, Number of Births, Number of Doctors, Number of Beds in ICU are not used by all product lines but are used by several product lines in several regions.

Static Lists are lists to which we can add records to indicate having a response to a custom question. If we have a question that cannot be answered by an existing field, and that answer is multiple choice or Yes / No, we can write that record to a list to show interest.

Most of the existing roles can create static lists and add or remove members from the list. In combination with the Flex Fields, thousands of points of data can be collected without adding a custom field.

If the data is more free-form in nature, that is to say it’s information that the customer provides that is not their personal contact information, or it’s information that then needs to be passed to SalesForce, that information can be written to a note field.

Presentation on Forms:

When creating a form, add the custom field and then change the label to have the field appear like a custom question. Below is an example of a form used by Healthcare Digital for collecting specific product interest by using a Flex Field and assigning a couple of choices the form respondent can pick from.

pastedImage_0.png

Both of these questions below tie to Flex Fields (Values shown for example)

pastedImage_1.png

Storing in Static Lists:

Within Marketo, there is a Smart Campaign flow step called “Add to List.” Within the flow step, there is a button on the Top Right called “Add Choice” which acts as IF/THEN type of logic. Putting in multiple choices in a single step means that Marketo will look for the first time the conditions are true and then abandon the rest of the choices. Alternatively, you can add multiple flow steps.  The example below is from the Product Preference project in place for Europe, where information that maps to a single, temporary Marketo fields needs to be stored permanently. The information is delivered with multiple values at once, which then need to be separated into lists.  It’s worth noting that almost 100 static lists are used to track interest in products for Core Imaging alone.

pastedImage_5.png

Here is an example of using a Flex Field to drive a product interest into a Static List:

pastedImage_6.png

Storing in Notes:

When the information needs to be passed to SalesForce, the only mechanism is to either put in a pre-existing field or into a note field. This also works for data that cannot be used for segmentation, such as self-entered information.  An example I like to use is the number of says someone could type in “Saint Joseph’s Hospital”: St. Joseph, St Joseph’s, St. Joe’s, SJHC, etc. All of these appear as completely discrete values to a computer which would have no way to group them. This self-entered information is best put into a note field.

Example: In this Smart Campaign, we build a Form Lead Note.  We’ve standardized how we use the Flex Fields on these forms.

Since the entire new value isn’t visible, here are the New Values:

Flex Field 1 is always used for Product Interest on certain Healthcare Digital forms:

Lead Notes: {{lead.contact question}}; Facility Type: {{lead.Facility Type}}; Product interest: {{lead.Flex Field 1}}

Flex Field 4 is always used for miscellaneous purposes on Healthcare Digital forms:

Lead Notes: {{lead.contact question}}; Facility Type: {{lead.Facility Type}}; Product interest: {{lead.Flex Field 1}}; {{lead.Flex Field 4}}

pastedImage_7.png

These values then carry over to SFDC where they appear in the Notes field.

Flex Fields used in scoring campaigns

Sometimes for product specific scoring, we want to ask questions that are very unique to an individual product. An example here is for our Cardiology product line, which asks which kinds of labs the hospital has. We store this information in Flex Field 3 from the form. When the form is filled out, this scoring workflow is triggered:

pastedImage_8.png

This scoring program will only start if the person indicates they have one of the above mentioned Cardiology Lab environments, which is asked on the form.

Using Static Lists for Segmentation

Now that you’ve seen examples how to translate form questions to Static Lists, the next step is to show you how those lists can be used like custom fields.

A Best Practice for segmentation lists is to create a program in an Operational Programs folder. This is an Operational Type program used only to house your lists.  These can also be stored in the Lead Database, but to make things simple we’ll show how these create what we in Healthcare Digital call “Core Lists.”

pastedImage_10.png

Here you’ll see the Core Lists we use for Cardiology. In here are two types of lists: Static Lists and Smart Lists. A static list is basically a list of names that you decide who is on or off.  A Smart List is more like a query: you use logic to determine if someone should be a member or not.

Static Lists give you ultimate control over who is on the lists but need to have Smart Campaigns – workflows – to add or remove names.

Smart Lists constantly update themselves as records qualify or disqualify. Smart lists can use static lists as inclusion or exclusion criteria.

In this example, we have created lists for our various Cardiology products. Lists marked as “IB” are current customers. CCE is one version of our product, CCI, CCW and DMS are other versions. Because this product line is very volatile – new products are constantly added, product names are subject to change and product lines can be retired – having these products as unique fields is not scalable. While the product is called DMS today it could be called ABC or something completely different tomorrow. Once a field is created in Marketo and is populated with data you cannot change the field name. It is also very labor intensive to remove fields.

Building a Segment

In this next graphic, you’ll see how a Smart List uses several Static Lists for segmentation. The purpose of this list is to dynamically build a list of people at hospitals who own our product, who would be interested in hearing about our Perinatal product line.

We know that hospitals have numerous specialties that are of no interest to each other. If we were to generate a list of all hospitals who owned a product and then sent to all records working at that hospital our Unsubscribe numbers would quickly elevate. Why would a neurosurgeon or a proctologist care about Perinatal software? Likewise, as a software company, we know that executives, business analysts and IT personel will sign up for information on this product line while they are evaluating or installing the product but they have no interest in communications meant for the end users.

This Smart List combines all people we know who are listed as being tied to one of the 12 Perinatal products and then filters them by job title. Here were are looking for 107 values that we feel are unique to people who work in women’s health: maternity, labor and delivery, infant care, obstetrics or gynecology, etc.

pastedImage_13.png

If these product interests were separate fields we would have to create a very complex list saying (1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13) and 14 and 15 and 16 and 17 as each product would be a separate line. From experience, the concept of “OR” and parenthesis is hard to explain to an average Marketo user while just putting all of the lists into a single box is much easier. Every time we add in a product line we would have to add it to this list and then manually rewrite the Advanced Filters as points 14-17 would need to be incremented by the number of new fields added in parentheses.

Living From Experience:

Prior to 2015, Healthcare Digital had our own instance of Marketo which had been in place since 2011. When it started, the thought was that unique fields would be used as needed and every product interest could have a unique field to show interest.

Over 5 years, that ended up created 121 fields. Many of these are for products that never ended up being real – we prepared for them but they were canceled. Many of these product ended up being retired. Others had their names changed and we were stuck referring to them by an old name. We also made fields to show is someone owned the product or was merely interested in it. This has been difficult to work with.

REST API Name

AreyouanexisitingCPNcustomer

CBS-BusinessETM

CBS-PracticeManagement

SS-Periop-AnesthesiaInteractionChecking

SS_PACSIW_Services

areyouanexistingC360customer

CBS-BusinessEWS

CBS-PracticeManagement-VAR

SS-Periop-BedManager

SS_PACS_Services

AreyouanexistingCBcustomer

CBS-BusinessHPA

CBS-PracticeSolution

SS-Periop-BusinessContinuity

SS_Periop_Analytics

AreyouanexistingCCAcustomer

CBS-BusinessInformatics

CBS-PracticeSolution-VAR

SS-Periop-BusinessObjectsReporting

SS_Periop_ePREOP

areyouanexistingCentricitycustomer

CBS-BusinessMCA

CBSCEServices

SS-Periop-ClinicalInformationViewer

SS_Periop_Services

AreyouanexistingCPACScustomer

CBS-BusinessPatientProtocolManager

CBSPopHealthManagement

SS-Periop-CPAandCPM

SS_RISIC_Services

AreyouanexistingCPScustomer

CBS-BusinessPOL

CBSVARServices

SS-Periop-NursingDocumentation

AreyouanexistingCVITcustomer

CBS-EDIServices

CBS_Business_FRM

SS-Periop-PACUDocumentation

areyouanexistingDPcustomer

CBS-EMR

CBS_Business_Services

SS-Periop-PATClinic

areyouanexistingEMRcustomer

CBS-EMR-VAR

CBS_GroupManagement_Services

SS-Periop-Perioperative

AreyouanexistingGEPACSIWcustomer

CBS-EMRClinicalContent

CBS_Practice_Services

SS-Periop-PerioperativeAnesthesia

AreyouanexistingGERIScustomer

CBS-EMRDocumentManagement

SS-Analytics

SS-Periop-PerioperativeManager

AreyouanexistingGMcustomer

CBS-EMReRX

SS-CentricityClinicalArchive

SS-Periop-PerioperativeTracker

AreyouanexistingPeriopcustomer

CBS-EMRPatientPortal

SS-CPN-Analytics

SS-Periop-Scanning

areyouanexistingPopHealthcustomer

CBS-EMRSecureMessaging

SS-CPN-Connect

SS-Periop-SterileProcessingManager

AreyouanexistingUniversalViewercustomer

CBS-GroupManagement

SS-CPN-IB

SS-Periop-SupplyChain

AreyouanexistingVARcustomer

CBS-GroupManagementAnalyzer

SS-CPN-Interoperability

SS-Periop-Weblink

Are_you_an_existing_CE_customer

CBS-GroupManagementDocumentMgmt

SS-CPN-LandD

SS-RIS-IC

CBS-Business

CBS-GroupManagementEligibility

SS-CPN-NICU

SS-UniversalViewer-CPACS

CBS-BusinessAdvancedWeb

CBS-GroupManagementHostedClaimsManager

SS-CPN-Services

SS-UniversalViewer-IW

CBS-BusinessAnesthesia_Radiation

CBS-GroupManagementPatientPortal

SS-CPN-WebAccess

SS-ZFP

CBS-BusinessASP

CBS-Hardware

SS-CVIS

SSC360CaseArchive

CBS-BusinessBAR

CBS-HealthCheck

SS-CVPACS

SSC360CaseExchange

CBS-BusinessBundledCareManager

CBS-Lab-Laboratory

SS-Integration

SSCCAVAR

CBS-BusinessCBO

CBS-Mobile

SS-MobileAccess

SSCVITUV

CBS-BusinessClinicalConnectivity

CBS-Pharm-Administration

SS-Omnyx

SSGEHealthCloud

CBS-BusinessConsulting

CBS-Pharm-Pharmacy

SS-OneView

SS_AW

CBS-BusinessEDIServices

CBS-PracticeAnalytics

SS-PACS

SS_CCA_Services

CBS-PracticeDocumentManagement

SS-PACSIW

SS_CVIT_Services

When we migrated to the Global instance we made a decision to never make this mistake again. This wasn’t scalable, was hard to support and very hard to train when product names change or retire. All of these issues go away by using lists.

Conclusion

It is my experience that there are very few reasons for adding in new fields, especially product specific or region specific fields. Having given this much thought, testing and experience and conferring with other Marketo Experts, Champions and Consultants, the consensus is that static lists work very well for preserving information that is Boolean in nature, such as a product interest.

As a Marketo admin, my goal isn’t to reject all requests but to work with the regional users and show them how results can be achieved without disrupting the database by adding fields. Along with the other admins we carefully consider and discuss proposals for new fields and judge it on the following criteria:

  • Can the same results be achieved in any other way?
  • Would the requested field serve any other region or product line?
  • Is the field name going to persist for years?
  • Is the field relevant to a large portion of the database? (At least 15%)

If we can achieve results without adding a new field and if data is only relevant to one product line or region then we are not likely to add in the new field.

The reason for not adding new fields is simply scale. If we do it for one product in one region we have to offer the same opportunity for every other product in every region. This is not scalable and would be very difficult to administer. There are too many products in too many regions and in the end only a very small number of records would utilize the field.

6808
13
13 Comments
Grégoire_Miche2
Level 10

Nice one Rob. A must read for all field-creation-happy admins.

-Greg

David_Gaible
Level 7

Great piece Robb Barrett PRD​. We use a similar process to capture lots of the one-off preferences for the dozens of live events our marketing hosts each year. I love keeping the number of fields down! (Now if I just convince my Salesforce admin to do the same I'll be set).

Geoff_Krajeski1
Level 10 - Champion Alumni

Interesting Robb Barrett PRD​.

My only concern here is for teams that actually use the fields in BI systems like Tableau, etc. 
Wondering what effect there may be in a situation like that?

Robb_Barrett
Marketo Employee

When a region admin recently asked me to add in 140 new fields, many of which were specific to a single event - I told him it wasn't scalable in Marketo. He then went to Marketo support who told him that we could add as many fields as we wanted with no database performance hit. That's not an answer that I'll support as I have 160+ users who all feel that "Just a couple more new fields" won't cause a problem.

I've managed to build hundreds of programs for dozens of products without adding in new fields.

Smaller company admins might not think this applies to them, but scalability is always a factor as is long-term vision. Your company will grow. Your marketing will grow. Your product line will change. What you're doing today is different than tomorrow. You have to think about how to make this manageable. You won't have that role one day and your replacement will inherit a mess.

Robb_Barrett
Marketo Employee

Good question.

First off, analytic tools like Tableau shouldn't necessarily be run directly off of a production database. If you have the money for Tableau (or any other BI tool) you have the money to use an MS-SQL, MySQL, Oracle, etc to build a datamart for reporting. In this case, you wouldn't want to have product interest attributes as part of your lead object, you'd want to create a product level fact table that would link to your lead record fact table. This not only normalizes your data but provides performance boosts, ability to integrate slowly changing dimension tacking and sets you up for inclusion in a more fully functioning data warehouse.

Full Marketing Analytics consists of several pieces: a CRM, a Marketing Automation platform, a BI tool and a datamart.  While each of these components show capability to act as the others, no one product is scoped to natively do all of these well, which is why there's so much competition in each of their marketplaces. Yes, they provide BI on their transactional data. Yes, they store data and provide mechanisms to change or update data. Yes, they provide some workflow management.  However, if you really want to do Marketing Operations correct, you need to have all of these pieces.

I happen to be a big fan of Richard Kimball and the work he's done on describing how to build a data warehouse, how to use it and why. Tableau is just a semantic layer on top of a well structured database designed to visually represent the aggregated data.

To answer your question, Geoff Krajeski​, you would look at pushing your data to your staging server at the time of form-fill, before the flex fields reset. A Staging Server is a temporary storage place where you store transactions row by row. For example, let's say you filled out 5 forms today. Every time you fill one out I would sent a complete snapshot of your lead record to the staging server along with the information you filled out in the forms.

At the end of the day, the staging server runs a process that will normalize the data and prepare it to be pushed into the datamart / warehouse. First thing it would do is look at all of your personal information and compare it to what it has on file. Did your email, phone or name change? Maybe you put Geoffrey in one form and Geoff in another. I would then compare that to other data points that can act as keys - lead ID, email, phone for example, and if I think it's the same person I'll store your name in tblFirstName in two rows: Geoff and Geoffrey.

Now, let's skip to the temporary fields.  In one form, I used a temp field to show that you were interested in Module 1 of Product A.  I now create a link to show that you have this product interest. Getting into the specifics of data modeling is outside the scope of this answer but I love talking Data Modeling and would be happy to do so at some other point, perhaps over a beer with a placemat and a pencil.

When all is said and done, I now have a Marketo instance that uses static lists to show you're interested in Product A and moduleA_Product1. I also have well structured data in my datamart that will allow me to see your product interests currently, your product interests over time, which products and modules you've ever been interested in and all sorts of other statistical magic I can infer based on when and how I captured your information and others who had similar paths.

So, TL;DR

If you're going to use a BI tool, go all in and use it on a datamart, not a transactional data source. Otherwise, you're only getting a small fraction of what you're paying for with Tableau.

Geoff_Krajeski1
Level 10 - Champion Alumni

Thanks.  I fully understand the need for a datamart (intermediary repository of data) as we're working with this methodology!

Your "flex" fields are the same as my "proxy" fields that I employ.

My question was moreso towards "non-product" type implementations, but I can see examples of how this may extend to my use cases.

Refining the database to require fewer fields is a wonderful thing for sure! 

It also will, in fact, speed up any queries (reports, smart lists, etc.) for any user, much to the contrary of the point this someone made to your regional admin. I've experienced this myself and will attest.

Gerard_Donnell4
Level 10

Great post Robb. 

James_Chung
Level 2

Robb Barrett PRD wrote:

Flex Fields: We allow four flex fields available for use by all Marketo users. These are called Flex Field 1, Flex Field 2, Flex Field 3 and Flex Field 4. These can be used to store any type of information temporarily – for a half hour – which is enough time to move the data to either a Note field or a Static List for segmentation purposes.

Great post, thanks for sharing. One quick question-

How are you clearing flex fields in your current setup?  I'm guessing at the very final step of the smart campaign you reset the field?  My concern is that there may be previous values from another user if the value is not properly reset.

Trinity_Levens3
Level 3

Hey Robb!   People on my team welcome this method, I share your posts with them....question, what's the best way to bring that data over to the notes field?

Robb_Barrett
Marketo Employee

We have an automated operational campaign that looks for them to change and wipes them after 30 minutes