List import of members taking significantly longer for them to be available to smart campaigns

[Dan_Stevens_]

We have a common process for those programs that include offline engagement - like attending an event.  When the event has concluded, the marketer provides the list of attendees - contained in our standard "campaign response" Excel template - for a member of my team to upload to the appropriate program (added to a static list within that program).  Once imported, we have a batch campaign that is run against this list to place these members into the appropriate program status (attended, promoted to MQL, etc.).

Lately, we're finding that it sometimes takes over an hour before the imported leads - primarily NEW leads - are available to run through the batch campaign.  So even though they appear in the static list, they are not reflected within the SCHEDULE tab ("Smart list status: Set to affect xx existing people").  Eventually they are represented in the number of qualified people, but the delay we're experiencing seems awfully excessive (hours/days) and is causing a lot of frustration on my team (since they can't complete the process until they're available; which affects other marketing activities they're working on).

Is anyone else experiencing this sort of delay?

BTW, I started a similar discussion a few weeks back: Delays/unable to search for new people immediately after they've been entered (manually or via list ....  But now, the delays are really becomng excessive.

[Kyle_McCormick]

I have noticed this quite a bit when cloning programs or batch campaigns lately. I typically delete the filters in the smart list and re-add them and the leads will usually show up as qualified leads.  Not sure if you're cloning or not, but that is usually what is happening to me. Hope that helps.

-Kyle

[Dan_Stevens_]

Hi Kyle - the majority of our program originate from a core set of program templates contained in our CoE workspace.  As our programs are quite complex, it's necessary for my team to start with the proper framework vs. start from scratch each and every time.  Not sure how/why that would impact the list import delays that we're experiencing.

[Kyle_McCormick · Accepted solution]

Our process is very similar to yours.  It could very well be something else that's causing the delay, however I have found that deleting triggers and filters in the smart lists seemed to fix it. I've been wanting to ask support, but I can never seem to replicate the issue when I want it to occur.

[Dan_Stevens_]

Hi Kyle - members of my team were running into this issue again today.  So I figured I'd give your workaround a shot.  Low and behold, it worked!  Simply delete the smart list filter that references the static list and then re-added it - and it worked.  All members of that static list now appear within the schedule tab and will qualify when the batch campaign is run.

Amber Hall​

Kelly Heywood​

[Grégoire_Miche2]

This is really weird...

-Greg

[Dan_Stevens_]

Tell me about it!  We had waited for over 90 minutes  - with no one qualifying in the "schedule" tab of the batch campaign that referenced the static list in the smart list - until we just started to try to remedy this with some workarounds.  And then I remember Kyle's suggestion here and gave it a shot. 

[Grégoire_Miche2]

Looks like a serious bug. Have you logged a support case?

-Greg

[Dan_Stevens_]

Yes I did.  It's been with Engineering since November.  I've copied some of their responses here in this thread (and this thread: Delays/unable to search for new people immediately after they've been entered (manually or via list ... .  Here's a summary:

At the moment either support or engineering would need to fix this particular type of issue. However, part of the reason we are making the move to the Google Cloud Platform is to help prevent this type of issue and increase the scalability. Since this issue isn't particularly unique to your company, we are actively engaged in a universal fix. I understand its frustrating but we are devoted to getting a permanent solution.

At the moment it requires a case-by-case fix, so once it's noticed we have to get engineering to resolve it permanently but we can take steps to prevent a block in your production but this also needs to be done case-by-case, as I did when the issue was first raised, and then get engineering to do their thing.

This issue won’t happen with most imports, but our engineering team is aware of the issues that do happen and are working on a wider overall resolution.

Engineering is actively engaged in resolving the wider issue and we will keep you updated on the status of that resolution.

You do have blackcat enabled which is faster. The thing is that blackcat even though its faster when creating a new record it uses mySQL to create the lead. and then if in blackcat we might see delays for it to update through the UI if blackcat is enabled and we really do not have a way of getting this to work better at the moment. The solution here is to have your All Leads (smartlist) in mySQL so when you add someone it finds them in that original request at the moment. I have tested this and can confirm if I switch running format from mySQL and blackcat when creating new leads and having blackcat rendering smartlists it will take some time but it switched back to mySQL its always there.  Also the blackcat render delay of adding new leads is intermittent not always a delay.

Following initial creation, it can take a few hours for the indexing for the Large Database processor to finish the index job; Once that's done the record will be searchable/usable by normal means.

[Grégoire_Miche2]

. I can't say more. Let's hope that the Google Cloud migration will solve it.

-Greg