Leads failing to qualify for triggers

[Liliana_Cheng1]

Hey community, our instance has been experiencing major delays (two to four hours) in our triggers. Some leads who fill out a form do not qualify for the triggers they're supposed to, sometimes for hours, sometimes for over a day. This issue is not isolated to one particular form or trigger, it's across our instance and it's inconsistently happening because some of my test leads have qualified for the trigger without an issue.

I've spoken to two Marketo support reps who confirmed that our triggers are set up correctly to capture, and the affected leads SHOULD be qualifying for the triggers. They also confirmed that we do not have a heavy volume of leads in the trigger queue. I am still waiting for a manager to update me but this is troubling.

What confuses me is when leads just sit in Marketo for two+ hours without any further activity will suddenly qualify for their trigger. No further actions were taken from the lead and I didn't manually request the campaign but they qualified - why is this happening? So most of the leads who failed to trigger immediately yesterday are okay after four hours... Is anyone else experiencing this massive delay in triggered behaviors?

If it makes a difference, I noticed this yesterday 3/6/2017 around 1:50pm Eastern and leads kept failing to qualify until 3pm. Another round of trigger failures happened between 7pm and 9pm. Most of them were caught up this morning but the most recent round of failures started around 11:30am today.

Thanks.

[Dan_Stevens_]

We encountered something similar a few months ago (For those of you on Corona (part of Project Orion), are you experiencing issues explained here? ), and occasionally experience this behavior.  I wonder if this has anything to do with the Orion migration.

[Liliana_Cheng1]

Thanks for the link, Dan Stevens​. I added it to the case that I opened and asked if it was related to the Orion migration. I see that Ajay Awatramani​ replied to your discussion saying this bug was patched in November, but you mentioned that you still experience this behavior occasionally. How does your team handle these occasional delays? Does Marketo know this is still happening to your instance even after their patch?

Right now I'm just refreshing a smart list that catches leads who filled out any form, but do not have a SFDC created date, and then manually reconciling them using the "request campaign" action for the triggers they were supposed to qualify for. I'm also a little frustrated by the lack of communication from the support team so far - the ticket was pushed up to a manager over 24hrs ago and I haven't heard anything from them even though the reps I spoke to this morning said there are new internal notes on my case...

[Josh_Hill13]

Pretty typical. Look in Campaign Inspector for a list of triggered flows and see if you can shut some down. The slowdowns occur on the Trigger Eval Queue and often the cause is Visits Web Page, Lead is Created, or DVC.

[Liliana_Cheng1]

Hi Josh Hill​, do you mean trigger delays are pretty typical? I watch this smart list closely and have never experienced these extensive delays before. I'll take your advice on shutting down some triggered flows, but multiple Marketo support reps confirmed our trigger queue is negligible...

[Josh_Hill13]

Typical of Support.

There are periodic issues and i've found the following occurs:

• Unknown reason
• Engineering finds an issue after pressing Support
• Support tells you there is a trigger issue somewhere or a loop.
• Support tells you little.

Read more Load Balancing in Marketo and Marketing Automation - Marketing Rockstar Guides

[Dan_Stevens_]

Liliana, what I was saying is that while we no longer experience the issue back from November, that occasionally we will experience significant delays with trigger campaigns.  And like Josh mentioned, the first place you check is the Campaign Queue to see if anything is holding it up.

In terms of Support follow-up, I do know that they are experiencing a large backlog of items to address and engineering is only working on P1 and P2 escalation issues.

[SanfordWhiteman]

Liliana, in addition to the great tips from experience from Dan & Josh, bear in mind that trigger backlogs -- though always unwanted -- can theoretically happen regardless of the load on your instance or pod, by virtue of the very fact that there is a queue.

That is, when a system has multiple services handing off data to each other -- as any scaleable system must -- instead of a direct throughline from the client to all the back-end servers, that means more moving parts. And it also means temporary failure of any of the parts that enqueue and dequeue data can result in a later "replay" of delayed actions. That replay may seem mysterious, even impossible, but actually is due to the (relative) resilience of the architecture. Of course we don't want any failures, but when they happen it's reassuring that the queue pipeline continues where it left off instead of losing data.

[Liliana_Cheng1]

Thanks for your insight, Sanford Whiteman​. The issue isn't just the significant delay in triggers, but there are a few test leads who are still just sitting there without qualifying for the trigger that they should have. There might have been more leads that didn't qualify for their triggers, but I manually reconciled the legit leads so our sales team could work them.

[Molly_Cousins]

I'm experiencing this issue as well - and on the Filled Out Form trigger, to boot! Nothing amiss in Campaign Queue for us...

Has this issue resolved for you Liliana Cheng