AnsweredAssumed Answered

The Chicken or the egg? - What happens first: Added to smart lists or "person created" trigger?

Question asked by Victor Herrero on Nov 22, 2018
Latest reply on Dec 10, 2018 by Victor Herrero

Like • Show 0 Likes0
Comment • 19

Hi,

This is almost a philosophical question on the origin of things...

I have a Smart Campaign that looks something like this:

Smart list

Trigger: Person is created

Filter: Country is xxxx

Filter: Person is NOT member of "Smart List yyyy"

Flow

Send email zzzz

My question would be: when a person is created, does it get added to my "Smart List yyyy" fast enough to NOT qualify?

My intended behavior is that they don't qualify. I want to use this method to handle exceptions but I need to know if it works and if so, it it is reliable.

I could always let them qualify and remove them from the flow if they belong to the list, but I have a feeling its more convenient, better for MKTO performance and perhaps more maintainable to do this in the smart list rather than inside the flowsteps.

Thanks!

No one else has this question

Outcomes

19 Replies

Ronen Wasserman Nov 22, 2018 5:36 AM
Mark Correct
Correct Answer
HI Victor,

If the person doesn't exist already in the list I don't think it will work however what you can do is:
1. Smart campaign:
Trigger: Person is created
Flow: Wait a few minutes.
Request campaign.

2. Smart campaign:
Trigger: Campaign requested
Filter: Country is xxxx
Filter: Person is NOT a member of "Smart List yyyy"

Flow: Send Email

1 person found this helpful
Like • Show 1 Like1
Actions
- Sanford Whiteman @ Ronen Wasserman on Nov 23, 2018 12:08 AM
  Mark Correct
  Correct Answer
  If the person doesn't exist already in the list I don't think it will work
  It's a Smart List, though. So "membership" in the list is dynamic; there's no separate step to add to the list.
  
  The better way to phrase the question is "does the person match the filter parameters of the Smart List?" And that actually depends on what those parameters are. For example, if you trigger a webhook when somebody comes in, then the fields updated by the webhook may be updated a second or so later. If in the meantime you see if they're a member of a Smart List that filters on those field values, they may or be not be at that time (thus the outcome is unreliable). But a filter on data that already exists will find them.
  1 person found this helpful
  Like • Show 1 Like1
  Actions
  - Floyd Alvares @ Sanford Whiteman on Nov 22, 2018 6:52 PM
    Mark Correct
    Correct Answer
    Just adding my 2 cents:
    
    "My question would be: when a person is created, does it get added to my "Smart List yyyy" fast enough to NOT qualify?"
    
    When a member of a smart list filter (A) is referenced in a smart list within a smart campaign (B) or within another smart list (C). Then when B or C are executed, they will filter of each of the fields/criteria within the other smart list(A). So it is not entirely accurate to check if they would qualify fast enough in the smart list.
    
    However, the chicken and egg or in my terms we refer to them as the race condition will be needed to be taken into consideration.
    
    Example: a record may fill out a form on the website where the record populates certain field information (example country). Now if you run a smart campaign with the trigger "Fills out Form + Country value = YYYY". There will be a race condition between the trigger (Fills out Form) which is based on an activity in Marketo versus the update condition that Marketo is updating a record in Marketo (which is not based on the activity). However both these actions will happen at the same time.
    
    The same logic applies for filters like Data Value Changes and Person is created.
    
    The solution to this would be to either run batch campaigns for actions that do not need immediate effect OR leverage the solution recommend by Ronen Wasserman
    
    I hope this made sense
    Thanks
    Floyd
    Like • Show 0 Likes0
    Actions
    - Sanford Whiteman @ Floyd Alvares on Nov 23, 2018 12:57 AM
      Mark Correct
      Correct Answer
      To be clear, in this particular case there's not a race condition.
      
      If the Smart List qualification filters are based on data values that exist on the lead at the point of entry into the system (i.e. form fields in the same form post that is their Web Form Fillout source on a New Person activity) they will be members of the SL at the time the Person Created trigger fires.
      1 person found this helpful
      Like • Show 1 Like1
      Actions
      - Victor Herrero @ Sanford Whiteman on Dec 3, 2018 3:52 AM
        Mark Correct
        Correct Answer
        I see what you mean.
        As long as the source form includes the filters that you are using on the filter smart list, the trigger will fire correctly, because the necessary information will exist on the record on creation.
        You seem to be 100% sure about the speed of dynamic Marketo lists, but are all records constantly being added to and removed from smart lists on creation / update instantaneously?
        Is marketo THAT fast? Is it constantly re-evaluating membership for all records on each update?
        
        I didn't want to reveal / bore with the details of the campaign since I also wanted an answer that will help me in the future as well, but in this particular case, upon creation, all information needed for the Trigger and the Smart List is there.
        So the setup should be reliable.
        Like • Show 0 Likes0
        Actions
        Sanford Whiteman @ Victor Herrero on Dec 3, 2018 2:17 PM
        Mark Correct
        Correct Answer
        but are all records constantly being added to and removed from smart lists on creation / update instantaneously?
        It's not about "adding" or "removing" in any tangible sense. It's about whether queued-up database updates have been committed (you can think of "committed" as meaning "hardened" or "finished" or "made available to other processes") before the same database is searched by another process.
        
        And in this particular case they have. In many other cases, the order of updates and searches is unreliable. This simply isn't one of them.
        
        P.S. There's a fascinating world out there of newfangled database locking and concurrent access methods. These are basic building blocks of systems design studied since the 1960s, but people have gone back to the drawing board recently with the explosion of internet traffic and the need for better and better multi-user performance. Locking is made much more complex w/distributed systems (i.e. more than one server, as any large organization has, and more than one client), and distributed systems may accept being loosely consistent, meaning every server does not have the same data at every point in time. But again: not to worry in this case.
        1 person found this helpful
        Like • Show 1 Like1
        Actions
        Victor Herrero @ Sanford Whiteman on Dec 4, 2018 1:29 AM
        Mark Correct
        Correct Answer
        Ok, so after every update / creation, first smart list qualification is determined / fixed and then all other search processes take place.
        
        When it comes to smart list memberships, I find it fascinating that there is no actual field in Marketo's own database (or each of our instances) where for each record there is a "true/false" value somewhere.
        How does it really work?
        Like • Show 0 Likes0
        Actions
        Sanford Whiteman @ Victor Herrero on Dec 5, 2018 11:21 PM
        Mark Correct
        Correct Answer
        I find it fascinating that there is no actual field in Marketo's own database (or each of our instances) where for each record there is a "true/false" value somewhere. How does it really work?
        It works like any database search query.
        
        For a greatly simplified example, in SQL
        
        select id from users where filter1 = 'something' and filter2 = 'something else'
        
        returns a result set with the matching user IDs at that point in time. It doesn't write anything back to the users table. (The database user's permissions need not even allow it to update anything at all, just select.) The same query issued again can have a different result set.
        
        Yes, caching heuristics and intermediate tiers in many systems may store and reuse the entire result set for a certain period: "until the underlying table/s are known to have been updated" is the lossless way, while "until a fixed number of seconds have gone by" is the lossy but sometimes necessary way. (Try as you might to avoid the latter by tweaking indexes, analyzing query plans, etc., you'll end up caching something somewhere in high-volume and/or legacy environments.)
        
        But regardless of whether caching is employed, it would never be feasible to store on the lead level whether each person matched each query of an effectively infinite number of queries. Some architectures flatten (denormalize) known-interesting data back to the individual record level. Indeed, that's a form of storing somebody's match-iness back on the record, and that's how Marketo segments work. But you could never do this for all possible queries.
        1 person found this helpful
        Like • Show 1 Like1
        Actions
        Victor Herrero @ Sanford Whiteman on Dec 7, 2018 6:35 AM
        Mark Correct
        Correct Answer
        So you are saying that each smart list is basically a query result that gets updated every time the trigger is triggered, and until all query results have been delivered (i.e. first name is changed, so all smart lists and processes that are using that as a trigger are re-queried and the result returned) all other processes are on hold?
        
        returns a result set with the matching user IDs at that point in time. It doesn't write anything back to the users table. (The database user's permissions need not even allow it to update anything at all, just select.) The same query issued again can have a different result set.
        you mean since its a query you don't need to write, you are just reading?
        
        Yes, caching heuristics and intermediate tiers in many systems may store and reuse the entire result set for a certain period: "until the underlying table/s are known to have been updated" is the lossless way
        I take it Marketo operates like this most of the time with its smart lists right?
        
        But regardless of whether caching is employed, it would never be feasible to store on the lead level whether each person matched each query of an effectively infinite number of queries. Some architectures flatten (denormalize) known-interesting data back to the individual record level. Indeed, that's a form of storing somebody's match-iness back on the record, and that's how Marketo segments work. But you could never do this for all possible queries.
        So marketo segments DO generate a field on the records that stores membership data... this paragraph I don't understand too well.
        Like • Show 0 Likes0
        Actions
        Sanford Whiteman @ Victor Herrero on Dec 7, 2018 7:05 PM
        Mark Correct
        Correct Answer
        So you are saying that each smart list is basically a query result that gets updated every time the trigger is triggered, and until all query results have been delivered (i.e. first name is changed, so all smart lists and processes that are using that as a trigger are re-queried and the result returned) all other processes are on hold?
        I'm sort of saying that, but I don't want you to get the idea that triggers block on all prospective or pending updates. Some things you might think of as an update from a user-facing perspective (like from a form post or list upload) aren't committed to the database for some time. Only after they're fully committed can there then be a question of whether a filter will see them or not.
        
        Yes, caching heuristics and intermediate tiers in many systems may store and reuse the entire result set for a certain period: "until the underlying table/s are known to have been updated" is the lossless way
        I take it Marketo operates like this most of the time with its smart lists right?
        It depends.
        
        Let's just say that for your particular case a caching layer will never get in the way.
        
        So marketo segments DO generate a field on the records that stores membership data... this paragraph I don't understand too well.
        Correct. This flattened field is accessible in Velocity, for example.
        
        But you only have 20 Segmentations in your whole instance. That's very different from the infinite number of Smart Lists that can be created -- and modified -- throughout your database.
        1 person found this helpful
        Like • Show 1 Like1
        Actions
        Victor Herrero @ Sanford Whiteman on Dec 10, 2018 12:43 AM
        Mark Correct
        Correct Answer
        Sanford, thank you very much for your lessons
        I feel like these discussions are the ones that generate a deeper understanding of the tool.
        I appreciate it.
        Like • Show 0 Likes0
        Actions
  - Victor Herrero @ Sanford Whiteman on Nov 23, 2018 2:57 AM
    Mark Correct
    Correct Answer
    Ok, so my takeaway from your answer is that its unreliable, same as Ronan, you are saying that if the data is already there when the trigger happens - it's relaible.
    Otherwise, you can't be sure if it's going to work a 100% of the time.
    Like • Show 0 Likes0
    Actions
    - Sanford Whiteman @ Victor Herrero on Nov 23, 2018 10:22 AM
      Mark Correct
      Correct Answer
      You haven't said where you're getting the values from.
      
      Again, if I post a form with a Last Name and the lead is created via that form, yes you *can* nest a Smart List that filters on Last Name. It will always work and isn't unreliable.
      1 person found this helpful
      Like • Show 1 Like1
      Actions
  - Victor Herrero @ Sanford Whiteman on Nov 23, 2018 3:07 AM
    Mark Correct
    Correct Answer
    Here is another suggestion...
    
    Never use "person created" as a trigger in your instance's smart campaigns. Always use instead "added to list: "Person created" ".
    Create the list "Person created" via a smart campaign with the trigger "Person is created" (only this once, throughout your whole instance).
    Perhaps this would add enough delay and guarantee that the record has been created with at least all of the values it first came with, so by the time you check for "added to list" in your campaign all of the information should be there already.
    
    My concern here would be whether the instance could become sluggish, or whether there is an actual limit to the people that can be in a list.
    
    What do you think?
    Like • Show 0 Likes0
    Actions
    - Sanford Whiteman @ Victor Herrero on Nov 23, 2018 10:29 AM
      Mark Correct
      Correct Answer
      As above: depends on the source of the data value. If you do Change Data Value and then Add to List, then the trigger on Added to List can be filtered on the new data value with 100% reliability.
      
      If you do Call Webhook and then Add to List, then try to trigger on Added to List and filter on the data value, you have a race condition and your setup is not reliable.
      
      If you Call Webhook and trigger on Data Value Changes, that's 100% reliable.
      
      I never use Wait steps to anticipate data value changes. Not only is Wait clunky and itself unreliable, there's always another way.
      1 person found this helpful
      Like • Show 1 Like1
      Actions
- Victor Herrero @ Ronen Wasserman on Nov 23, 2018 2:54 AM
  Mark Correct
  Correct Answer
  Hi Ronen,
  
  Thanks for you answer!
  So you suggest I break that into two campaigns that happen in sequence.
  But doing that implicates duplicating the numbers of campaigns... perhaps in terms of maintainability not the best idea
  Like • Show 0 Likes0
  Actions
  - Grégoire Michel @ Victor Herrero on Nov 25, 2018 10:51 AM
    Mark Correct
    Correct Answer
    Hi Victor,
    
    The other way to look at it is to remove the filters from the smart list, then add a 2 minutes wait to the flow start, and use choices in your flow steps: if member of smart list then do nothing, default, send email.
    
    Just keep in mind that, whatever the duration of the wait, in theory, there are always situation where some of your leads will miss the filter. These are situations in which the database update is not quick enough to guarantee all the data is posted to the underlying database. This means that if you are working on a heavily loaded database, you might want to extend the delay.
    
    -Greg
    1 person found this helpful
    Like • Show 0 Likes0
    Actions
    - Sanford Whiteman @ Grégoire Michel on Nov 25, 2018 11:35 AM
      Mark Correct
      Correct Answer
      But that isn't true of this example. There's no race condition here and no need for an arbitrary wait.
      1 person found this helpful
      Like • Show 0 Likes0
      Actions
    - Victor Herrero @ Grégoire Michel on Dec 3, 2018 3:38 AM
      Mark Correct
      Correct Answer
      Thanks for your suggestion. Yes, working within the flow steps seems like a 100% reliable method. I will definitely bear that in mind in general.
      
      I am inclined to agree with Sanford though, wait steps are clunky and if you add enough in different places, your whole system will become vulnerable and hard to maintain.
      Potentially the wait steps will interfere with each other and you will have a hard time figuring out why this or that is not working as expected.
      Like • Show 0 Likes0
      Actions

The Chicken or the egg? - What happens first: Added to smart lists or "person created" trigger?

Outcomes

Related Content

Recommended Content