Our ticket has been escalated to engineering as well. Still seeing 8+ hours of delays. Bruce Jeffers definitely recommend opening a ticket if you haven't done so already -- my "Filled Out Form" and "Lead is Created" are the only triggers I've seen affected as well. Total pain!
I can confirm our instance is also affected.
I spoke to a key person at Marketo yesterday. They are working on it.
We have been affected too. Unfortunately, this has had a huge impact on our business, as our trial forms are processed through Marketo. No response from our Account Manager and still no known cause from Support...
I agree that the Communication on this was less than optimal. No one posted that Update until Dan saw it and the update was from April 24.
Did anyone receive an email or support call about this?
Here's an official statement by Marketo Support: Marketo Service Interruption Update
Support says it was fixed at 1028am. Sounds like there's still work to do though.
Open a ticket if you aren't sure the leads were processed.
We never got any official communication around this. In fact, the link I shared here didn't even mention the specific pods that were affected (or if it was system-wide).
Thanks for the link Dan Stevens, I didn't see that post and I don't understand why Marketo support didn't specifically reach out to those of us who have active tickets open for this issue. Especially since I've been contacting support once a week since March 6th and twice per day since last week about this issue.
Yesterday, we did not have leads get stuck in the "Fills out form" trigger which is an improvement, however, we did have a significant amount of leads who were stuck in various "Request campaign" triggers. This is still true today. We currently have 50+ leads stuck in a request campaign trigger.
Maybe they were secretly hoping we wouldn't have noticed last week, being at Summit and all....
5/1/17 UPDATE - this issue looks like it's been resolved (for now). We had a call with our account exec and a support escalation manager this past Friday, and they said the engineers planned to push a patch over the weekend. Since then, I've been monitoring and testing and our triggers seem to be working as they're built to be.
They haven't given us an in-depth analysis of what happened, but they said it was related to "the high speed database used to capture activity and how it was tied to trigger processing". The patch that was pushed this weekend was to remove the dependency. I'm not sure what their plan is after this (whether or not they'll tie these two problematic things together again).
If you are still seeing issues related to delayed triggers, you should contact support with some examples so they're aware this is still an issue for you.