Delays/unable to search for new people immediately after they've been entered (manually or via list import)

[Dan_Stevens_]

I'm interested to know if any of you are experiencing this: after we enter new people into Marketo (either manually or via a list import), it can take upwards of 15-20 minutes before we can retrieve them when doing a search (or seeing them in the lead grid).  Until then, we get a "No people were found matching xxx@domain.com".  And sometimes we'll even get some odd error that the list was not imported correctly - even though it was - and then a member of my team will try to import again right away (which, as you may or may not know, this creates duplicates as Marketo does not de-dupe against existing leads until they've been in the database for at least 5 minutes or so).

When raising this issue with Support, we received the following reply:

You do have blackcat enabled which is faster. The thing is that blackcat even though its faster when creating a new record it uses mySQL to create the lead. and then if in blackcat we might see delays for it to update through the UI if blackcat is enabled and we really do not have a way of getting this to work better at the moment.

The solution here is to have your All Leads (smartlist) in mySQL so when you add someone it finds them in that original request at the moment.

I have tested this and can confirm if I switch running format from mySQL and blackcat when creating new leads and having blackcat rendering smartlists it will take some time but it switched back to mySQL its always there.

Also the blackcat render delay of adding new leads is intermittent not always a delay.

I've never even heard of "blackcat".  Is this something that's part of Orion?  I even had a call with the support rep and left the call confused.

[SanfordWhiteman]

Blackcat predates Orion IIRC and it's the Smart List caching layer.

[Dory_Viscoglio]

Is it like Corona? That was something else I got us onto at eMarketer.

Regardless, that's insane and very frustrating. We've had similar issues but only with form fillouts where records aren't created immediately (can take 15-20 minutes), and one time where we received a list upload error and it still actually uploaded -- creating duplicates as you suggest.

Very frustrating, and support has been little to no help on the issue.

[Josh_Hill13]

This has been true for years now. It will take 5 to 20 minutes to "see" visibly a lead in Marketo. The system usually processes the lead though.

I have flagged it before but nothing has changed.

[Steven_Vanderb3]

Most customers with large databases are provisioned onto our Smart List indexer which uses Blackcat/Corona technology (and in process of fully transitioning towards our Project Orion infrastructure).  The reason it can take time to search for a brand new lead in the database is because it can take a few minutes for that data to replicate into the indexer.  Your Smart Lists (including the plain Lead Database) uses the indexer to return results much faster than the SQL based backbone, so you may not see a lead return there immediately after creation.  The data is definitely there immediately after a form is submitted, you create a new lead from Lead Database, you upload a list, etc - it can even trigger campaigns (because campaign trigger isn't relying on the indexer).  But it is possible for there to be a small delay in searching for that lead in an indexer powered Smart List immediately after creation.

The upside to the indexer is that it makes your Smart Lists return many times faster than if you didn't have it, but this is a quirk that comes along with the technology.

For the import errors, that I'm not familiar with so I think that specifically needs to be looked at by Support. You shouldn't be receiving an error for a successful import.

[Dan_Stevens_]

Thanks Steven - I suspected that was the reasoning behind all of this, but the other reason I opened the support ticket is because of the inconsistent behavior we experience.  We can manually create a lead in our default lead partition and it won't be appear in the search results for several minutes.  Yet, it's immediately available when searching in other partitions that have access to the default partition. 

BTW, we have a relatively small database - about 230k people.

[Steven_Vanderb3]

There's a few other moving cogs like caching based off your session that could cause that.  I'll agree that it can be hard to predict at times close to the time of lead creation.

Your database sounds small but there are some circumstances where you'll still be moved onto the indexer depending on how many activities overall you have in your database or if you have one of the Project Orion components such as Munchkin V2 or our ABM add-on that use the indexer.