That's exactly how we identified it. The data is there - it's just not being seen in any list/smart list; and most importantly, our smart campaigns that execute based on this data, are not triggering. Above, I mentioned that this was fixed by reindexing our DB. Unfortunately, the issues still remain - just not as severe. We continue to find smart campaigns that aren't processing leads, even though the data is there to support it. And Engineering informed me that they aren't working on any sort of patch - and instead, considered the re-indexing the proper fix.
We apologize for the lag issues your subscription has witnessed. Ultimately, we needed to put out a patch to fix a bug as well as "re-index" your subscription. Corona has been around for a few years now, though was recently upgraded for some subscriptions. It relates to our smartlist technology that vastly improves the speed of retrieval of smartlist results. The way this works is that lead and related data gets indexed into a specialized database to speed up queries, thereby returning smartlist results faster. In this case, the lag was the result of a bug, when we take the latest updates and move (index) them into Corona. This has since been patched. We have also reviewed our lag monitors and operating procedures to stay close to this. Once again, apologies for any inconvenience or challenges this has caused.
Just to let folks know, it seems that this bug is still haunting some instances. It is very frustrating and very scary to not be able to trust that smart lists/triggers can pull the right records and do the right things. In this case, Marketo Support has escalated to engineering. They supposedly "patched" the instance fixed it a few weeks ago but it's still happening. Re-escalated it again last week but still waiting to hear back and the problems are continuing.
I would caution folks to check your data to make sure the right things are happening with smart list/triggers/filters.
The lag is expected. "Corona" is just a name they use for the Solr indexes they use to index your data (Note: you don't need to be on Orion to have Corona). The lag is where your data is cached or changing and needs to be replicating. If you have an ever changing DB i.e. Change Data Value etc then you will experience a small lag on load.
I wish it was just a lag problem. Unfortunately, it isn't. This is a top priority issue with engineering right now.
It could be poor setup on Marketo for sure on your pod then. IMHO they made a terrible mistake with not going fully managed/hosted (hopefully something they will fix with the move to GCP).
If you have an ever changing DB i.e. Change Data Value etc then you will experience a small lag on load.
Isn't this basically every Marketo customer? CDV is probably the most common flow step that's used in everyone's environment. And for us, it's not just a small lag - it's pretty significant.
Agreed, this can't be the cause of significant latency.
This is scary stuff Dan Stevens!
How do you know if you are on Corona?
Has anyone on Corona been told of these known issues and the effects it could be having? (Is there a known list of issues I can check my instance against?)
I am now witnessing similar issues. New leads are entering our system via API. In their profile view the fields are updating with data but when I look at the Activity Log the Change Data Value actions are not logged.
We have smart campaigns based on Change Data Value triggers which are now not firing, meaning new leads are not receiving critical emails.