Leads failing to qualify for triggers

[Liliana_Cheng1]

Molly Cousins​, Marketo acknowledged that this is "an issue with their infrastructure causing performance delays" and said it could take months to roll out upgrades to their infrastructure. Incredibly frustrating for us and we are considering other options...

As of last week, the frequency and length of delays had slowed down for us - we noticed it happening once a week or so, and the leads would start moving through the triggers within two hours or so. For better or for worse, they had told us that these trigger delays seemed to be limited to just a handful of instances so it was not a widespread issue.

But today is a totally different story. Today has been significantly worse - my test leads from 10am still have ZERO activity on them apart from the original filled out form activity and the list of leads failing to qualify has grown. The senior support rep on our case advised that there was a server outage today that is affecting performance across a lot more instances. They had a P1 alert banner in the support portal that talked about performance issues all morning and most of this afternoon, but it's gone now.

I suggest that you open a ticket right away so they know your account is affected as well. They are planning to "replay the triggers" today but they have not confirmed what time this will happen.

Edit: never mind, I still see the P1 alert banner in the support portal. And my list of leads who are failing to qualify for their triggers keeps growing. They haven't fixed anything yet.

[Molly_Cousins]

Liliana Cheng Wow. I opened a P2 ticket 3+ hrs ago and hadn't heard anything back so I guess that explains their slow response. Thanks for the info! This is nuts.

[Dan_Stevens_]

Right now, I'm noticing a pretty significant delay in trigger campaigns being run.  And nothing in my campaign queue.

[Liliana_Cheng1]

Dan Stevens​, our Senior Support Rep told us as of 8:30am Eastern today, they still have not replayed the triggers so this issue is not resolved yet. If you haven't already done so, I suggest opening a ticket with support and making sure they know how big of a deal this is for your instance.

Since we depend heavily on the Fills Out Form and Request Campaign triggers for syncing and routing, I have been manually syncing leads and routing since last night. I'm still seeing new leads from this morning who fail to qualify for any triggers, which makes sense because Marketo hasn't pushed a patch or resolved the root cause, whatever it is. So frustrating.

[Sam_Chou1]

Are you guys still having issues this week? I thought it would resolve over the weekend, but my campaigns are still inconsistently firing or taking 3+ hours to trigger. Had to re-open my case.

[Liliana_Cheng1]

Sam Chou​, yes this is still ongoing for us. I asked for an update today but haven't gotten anything useful. We have a call scheduled with one of the support managers later this week. I suggest consistent follow ups on your ticket with examples of the leads and the triggers they should be qualifying for.

I've been updating my ticket every few hours with new examples. Admittedly overkill, but I think it helps to drives the point home how much of a negative impact this has on our company. We can't have SQL leads sitting in Marketo without qualifying for their triggers for hours at a time, especially not during the end of the month, and especially not since we've raised this issue since March 6th without resolution. I have been manually reconciling and routing leads all day and it's an inefficient use of time.

[Josh_Hill13]

This appears isolated to some pods. Several pods went down and up over the past few weeks.

I've had a lot of similar issues with Marketo and did some of the same things you have all tried. Their answer has been "Move to Orion" if you aren't already on it.

Triggers typically are not "replayed" if the leads were not already in the Trigger Eval Queue. So you'd need a separate batch to fix things; adding a Campaign is Requested to triggers downstream of the batch might help you call the items you wanted to kick off. Use carefully.

Could be wrong about this, however, that's been my experience.

I'm not seeing the P1 notice right now. Did they resolve Liliana Cheng​ ?

[Liliana_Cheng1]

Hey Josh Hill​, thanks for your insight. A Senior Support Manager confirmed last week that "most" of our instance has already been moved to Orion, most notably the Spark Indexer component and some other components that would most affect trigger actions.

Additionally, our support rep has been working with engineering to get our triggers replayed regularly because of the large amounts of leads who just sit in our instance without action for 8+ hours at a time.

The latest from yesterday was "We cleared out our lag and replayed events this morning and we will do the same again later this evening as the backlog builds up. We are still working on a solution. We will keep you updated on a fix." so nope, they haven't resolved anything.

The volume of leads who fail to qualify for triggers, combined with the extended delays has gotten significantly worse in the past week. Back in March and early April, it used to affect a handful of leads per day that I could manage to manually reconcile without major interruption to my day. Since last week, the volume has grown to 70+ leads per day who don't qualify for one trigger or another and it has become obnoxiously disruptive.

I also want to note that the triggers that are failing are limited to the "Fills Out Form" trigger and the "Request Campaign" trigger, so those two are totally unreliable and cannot be used when I reconcile.

[Anonymous]

We noticed the delay as well, or rather that we have leads not going through the proper trigger campaigns. Any delay on 'Lead is Created' has caused certain triggers not to fire or become delayed. We've been running ad hoc programs (carefully designed and implemented) so we continue to run relatively smoothly. It sounds like you've had to do something similar,

I understand the frustration though! I spent a good chunk of time on Monday trying to figure out why these leads were not getting assigned a country and thus not syncing to sfdc.

Thanks for updating this thread as you hear from support and engineering.

[Bruce_Jeffers]

We also have been experiencing this quite a bit, off and on, over the past couple weeks. Most of the failures/delays are in campaigns with 'Filled Out Form' or 'Lead is Created' triggers. Support's response was just to convert as much as possible to Batch -- but that of course can't work for certain business-critical flows and processes. Looking forward to having this resolved.