List import of members taking significantly longer for them to be available to smart campaigns

[Dan_Stevens_]

We have a common process for those programs that include offline engagement - like attending an event.  When the event has concluded, the marketer provides the list of attendees - contained in our standard "campaign response" Excel template - for a member of my team to upload to the appropriate program (added to a static list within that program).  Once imported, we have a batch campaign that is run against this list to place these members into the appropriate program status (attended, promoted to MQL, etc.).

Lately, we're finding that it sometimes takes over an hour before the imported leads - primarily NEW leads - are available to run through the batch campaign.  So even though they appear in the static list, they are not reflected within the SCHEDULE tab ("Smart list status: Set to affect xx existing people").  Eventually they are represented in the number of qualified people, but the delay we're experiencing seems awfully excessive (hours/days) and is causing a lot of frustration on my team (since they can't complete the process until they're available; which affects other marketing activities they're working on).

Is anyone else experiencing this sort of delay?

BTW, I started a similar discussion a few weeks back: Delays/unable to search for new people immediately after they've been entered (manually or via list ....  But now, the delays are really becomng excessive.

[Anonymous]

Was anyone able to find a resolution here? I loaded a list (25 members) to a static list and all the members are showing in the list, but only 4 qualifying for both the existing trigger campaign and a newly created batch campaign with a Member of List filter.

Is this consistent with what you guys were seeing as well? And if so, has anyone found something that will resolve it?

[Dan_Stevens_]

Hi Kayla - yes, this is very consistent with what we've been experiencing in our instance.  And no, there is no resolution.  When I spoke to one of the Support leads last week, they informed me this has been on ongoing known issue and they are able to resolve them on a one-off basis.  When I say "resolve", I mean the specific issue (e.g., a specific smart campaign properly qualifying the correct number of people).  They issue a patch into the program - but that's it.  It's not an instance-wide patch/resolution.  So it will probably occur again - until they figure the root-cause of the issue.

[Anonymous]

Have you found that it will resolve itself over time or would you recommend reaching out to Marketo?

[Dan_Stevens_]

In our experience, it does not resolve over time.  Although the workaround we were provided by Support was to manually enter all of the email addresses into a smart list of a smart campaign, vs. referencing the static list.

I would definitely reach out to Marketo so that they can get a better grip on the impact this is having on its customers.

[Dan_Stevens_]

This happened to us again today.  Members of my team imported lists into the event programs to process and anyone that was new, never showed up in the SCHEDULE tab of the smart campaign to process the attendees/MQLs.  The response we received from Support is similar to what we were told in the past:

This is due to a database issue that you've experienced before. I can escalate it to engineering to get it fixed for this list but that can take some time. Since it's urgent the quickest way would be to resolve - instead of referencing “member of list” in your smart list - is to copy/paste the specific email addresses for those leads into the smart list of the smart campaign. I have overridden the Smart List processor so the campaign shows the members correctly. I am going to create a case with Engineering to get this resolved.

The workaround they mention can be very risky if you have duplicates that exist in the same or other partitions (based on how Marketo determines the ACTIVE lead - see Re: If duplicate leads are an issue, be aware of how Marketo determines which lead is active ).  This ongoing issue continues to hamper my team’s ability to use Marketo effectively across our global organization.  The move to the Google Cloud can't happen soon enough.

[Rachel_Noble]

We've haven't personally but a few of our customers have in the past few weeks.

Today I watched someone upload a list of 1 person and it took about 8 minutes.

[Dan_Stevens_]

This past week, during some testing, we manually added a new person to the US partition.  While the person appeared in the default partition (after about 10 minutes), it took over an hour for this person to appear (and be used in a smart campaign) in the US partition - even though we specifically defined the partition on the record as "US".

[Dan_Stevens_]

We opened up a case yesterday with Support around a similar issue.  Their response was very surprising (and concerning).  And btw, our database is relatively small (< 200,000) compared to many customers.

Following initial creation, it can take a few hours for the indexing for the Large Database processor to finish the index job; Once that's done the record will be searchable/usable by normal means.

[Anonymous]

Dan Stevens   If that is the case, then Marketo should send an email alert or system notification letting you know when the indexing is done. IMO the import isn't done until it is usable.

[Anonymous]

Dan Stevens​

Hi Dan,

The only time our team noticed this type of issue it turned out to be an indexing backlog. Here is how it presented. The list upload always seemed to run within the expected time, but if you searched against the static list, the records would not show up.  Bottom line - ingesting data was fine however indexing the date which makes it available for programs was delayed.

I hope that gives you path to explore.

Eric

[Dan_Stevens_]

Hi Eric - yeah, that's consistent with what we were told by support - as documented in this thread: Delays/unable to search for new people immediately after they've been entered (manually or via list ...

[Kyle_McCormick]

I have noticed this quite a bit when cloning programs or batch campaigns lately. I typically delete the filters in the smart list and re-add them and the leads will usually show up as qualified leads.  Not sure if you're cloning or not, but that is usually what is happening to me. Hope that helps.

-Kyle

[Dan_Stevens_]

Hi Kyle - the majority of our program originate from a core set of program templates contained in our CoE workspace.  As our programs are quite complex, it's necessary for my team to start with the proper framework vs. start from scratch each and every time.  Not sure how/why that would impact the list import delays that we're experiencing.

[Kyle_McCormick · Accepted solution]

Our process is very similar to yours.  It could very well be something else that's causing the delay, however I have found that deleting triggers and filters in the smart lists seemed to fix it. I've been wanting to ask support, but I can never seem to replicate the issue when I want it to occur.

[Dan_Stevens_]

Hi Kyle - members of my team were running into this issue again today.  So I figured I'd give your workaround a shot.  Low and behold, it worked!  Simply delete the smart list filter that references the static list and then re-added it - and it worked.  All members of that static list now appear within the schedule tab and will qualify when the batch campaign is run.

Amber Hall​

Kelly Heywood​

[Grégoire_Miche2]

This is really weird...

-Greg

[Dan_Stevens_]

Tell me about it!  We had waited for over 90 minutes  - with no one qualifying in the "schedule" tab of the batch campaign that referenced the static list in the smart list - until we just started to try to remedy this with some workarounds.  And then I remember Kyle's suggestion here and gave it a shot. 

[Grégoire_Miche2]

Looks like a serious bug. Have you logged a support case?

-Greg

[Dan_Stevens_]

Yes I did.  It's been with Engineering since November.  I've copied some of their responses here in this thread (and this thread: Delays/unable to search for new people immediately after they've been entered (manually or via list ... .  Here's a summary:

At the moment either support or engineering would need to fix this particular type of issue. However, part of the reason we are making the move to the Google Cloud Platform is to help prevent this type of issue and increase the scalability. Since this issue isn't particularly unique to your company, we are actively engaged in a universal fix. I understand its frustrating but we are devoted to getting a permanent solution.

At the moment it requires a case-by-case fix, so once it's noticed we have to get engineering to resolve it permanently but we can take steps to prevent a block in your production but this also needs to be done case-by-case, as I did when the issue was first raised, and then get engineering to do their thing.

This issue won’t happen with most imports, but our engineering team is aware of the issues that do happen and are working on a wider overall resolution.

Engineering is actively engaged in resolving the wider issue and we will keep you updated on the status of that resolution.

You do have blackcat enabled which is faster. The thing is that blackcat even though its faster when creating a new record it uses mySQL to create the lead. and then if in blackcat we might see delays for it to update through the UI if blackcat is enabled and we really do not have a way of getting this to work better at the moment. The solution here is to have your All Leads (smartlist) in mySQL so when you add someone it finds them in that original request at the moment. I have tested this and can confirm if I switch running format from mySQL and blackcat when creating new leads and having blackcat rendering smartlists it will take some time but it switched back to mySQL its always there.  Also the blackcat render delay of adding new leads is intermittent not always a delay.

Following initial creation, it can take a few hours for the indexing for the Large Database processor to finish the index job; Once that's done the record will be searchable/usable by normal means.

[Grégoire_Miche2]

. I can't say more. Let's hope that the Google Cloud migration will solve it.

-Greg