Implementing a New Lead Scoring Model

[Richard_Wasylyn]

I'm in the process of remodeling our lead scoring. We have defined all the scoring adjustments and I'm in the process of implementing the new campaigns. We've decided to remove all previous scoring and run batch scores to re-calibrate all leads in the system.

For any behavioral activity the campaigns will work based on the triggers. However, on the demographic side, we have a third party service that cleans our data, so I'm concerned that we will double score leads based on demographic info. Here is the scenario:

- Lead is created - demographic information is scored (eg. industry value = telecom)

- 24hrs later our vendor cleans the data - demographic (vendor changes industry value from telecom to technology)

The only way I can think this through so we aren't double counting is to zero out the scores and re-calibrate the demographic score on a regular basis.

This could happen through sales activity updating records as well, if a rep changed someone's title multiple times in one week would it just keep adding based on the data value changes?

[Casey_Grimes]

Hi Richard,

Have you considered using a wait step for allowing the vendor time to clear the data? That is, you have triggers set up for either the lead being created or the data value changing, put a 25-hour buffer, and then calculate the score based on the final demographic value that's there after the wait step. That way, your scoring will run a day behind on demographics, but won't require reworking. This also skirts around mis-entered information--I personally like to put a 15 minute wait step on ANY hand-created lead for human error fixes.

[Edward_Unthank_]

Hey Richard, I've gone through this before, and done this exact whiteboarding exercise.¹

I solved it by having a few different triggers to recalculate demographic scoring. Here's a screenshot:²

A - First Time: This is the original trigger through lead processing, aka through a "Lead is Created" kind of trigger. This just adds up from Null.

B - Gating ReRoll: This is like you said, to make sure that either data value change from data enrichment or through an XDR doesn't screw up the behavioral scoring. Upon data value change of any of the scoring inputs (steps 110-220 in the screenshot), it waits 15 minutes (in case someone is manually changing values, giving them time to finish updating), resets demographic score to =0, then rescores from 0. Leads can only run through once per hour to make sure it doesn't run lots of time unnecessarily.

Here's a look at the B - Gating ReRoll smart campaign:

(The 2. Gating folder is how we latch in lead enrichment to make sure we're only using webhook enrichment calls when they're necessary—if a prospect doesn't pass the demographic score threshold, we enrich with BuiltWith and InsideView and others. That changes the data values of the inputs, which in turn triggers B - Gating ReRoll to recalculate from 0 for a better picture of demographic score.)

Cheers,

Edward Unthank | Founder, Etumos

¹ Demographic Gating: Stronger Scoring for Lead Qualification

² I talked through this at my Marketo Summit 2015 presentation called Architecting a Robust and Scalable Marketo Instance (the time to check is 34:50-37:00).

[Josh_Hill13]

I agree with Ed, but focus on that last screenshot with the following campaigns:

• Demo Scoring Trigger > any demo change causes rescore
  • call various scoring campaigns for each Demographic Value

[Justin_Norris1]

Richard Wasylynchuk​,

Every way I have seen this solved uses a variant of what Edward Unthank​ has described -- zero out demographic, (if using a combined lead score, also set lead score = behavioural), then re-run all demographic flows based on current data.

One issue we have found in practice is that there can be race conditions if a lead has multiple value changes in succession, and these can lead to unpredictable results.

Edward has solved for this with the 15 minute wait step, another method we've used is to have demographic scoring controlled by scheduled batches instead of trigger campaigns.

This works by having a single smart list that identifies leads who have had any demographic value change in the past x hours, and then have recurring scheduled batch campaigns set up to run every x hours that will zero out the demographic score and request the demo score flows to re-score any members of that smart list.

You need a few extra scheduled batch campaigns to make this work, and it's definitely less real-time, but you could argue it's a bit more bulletproof.

For instance, using Edward's set-up (which is awesome), I could nonetheless envision a scenario where value changes could get missed. E.g.,

- a rep makes changes to a lead record.

- re-score is triggered and runs 15 minutes later

- another 5 minutes after that, the rep makes a few more changes

- re-score is triggered but the lead doesn't qualify because of the once-per-hour qualification rules.

I think those demographic changes would never get re-scored until (and if) there were further demo changes more than an hour later to kick things off again.

This is an edge case but something to keep in mind. (And Edward please correct me if I'm missing something that solves for this.)