Maximum Number of Contacts per Campaign Run

[Anonymous]

To keep our lead data clean in Marketo, I've created a smart list called "Stale Leads" that contains any lead that hasn't been synced with our CRM within the past 30 days.  This could be for various reasons but those are out of scope.

Let's say we have 100,000 total leads in Marketo and the smart list above returns 70,000 leads on Monday.  Is there a way to create one campaign that uses this list to perform a set of tasks up to 20,000 leads?  The end result I would want is that on Monday, 20,000 out  of 70,000 leads are processed.  On Tuesday, I would want the next 20,000, and etc.  Keep in mind that additional leads might become stale so the progression could be something as follows:

Monday: 70,000 leads in list and 20,000 are processed.
Tuesday: 51,000 leads in list and 20,000 are processed. (20,000 gone from Monday to 1,000 new)
Wednesday: 34,000 leads in list and 20,000 are processed. (20,000 gone from Tuesday to 3,000 new)
Thursday: 14,000 leads in list and 14,000 are processed (less than 20 left).

Initially I used the random sample in my campaign.  Then I realized that it would never do all the contacts ... just x% of the total list.  What I really need is a filter "Max number" that lets me set an upper threshold such as 20,000 that will process up to that number per campaign.

Since such a filter does not exist, has anyone been able to accomplish something similiar to this without using static lists?  On a weekly basis, I could have a campaign that divides the 70,000 into four/five static lists and then use a different list each day during the week but that sounds like overkill to me.

Thank you.

[Anonymous]

Hey Jeff, your suggestion using static lists is probably the way I would do it off the top of my head, although it's still very sub-optimal. 

You could have your 5 static lists that get over-written every week if need be. I would use a weekly scheduled campaign to distribute the ENTIRE LIST of leads to one of the 5 lists. I am guessing new leads would need to be added to a list. 

Then your campaigns that run the tasks could use 

member of list [static list 1] 
member of smart list [not stale leads] 

In the criteria. 

If you balance your static lists correctly you could maybe get close to 20K each time and would act only upon leads that are not stale at the time of sending. 

Still this whole thing is a huge pain in the rear-end. Perhaps there is a better way. 

[Josh_Hill13]

So why do you only want to process 20,000 leads per day?

[Anonymous]

Thanks, Justin.  I have another idea that similiar to the static lists idea but would only require one list.  I need to bake it.  If it works, I'll share it here.

Josh, the main reason is that I do not want a campaign to run for hours because it runs into a large list and I do not want to risk the chance of using all my API calls to SFDC.  We only get 89,000 total calls per 24 hours.  Limiting my list to 20,000 will ensure I only use up to 20,000 calls for this particular campaign per 24 hour period.

[Anonymous]

Jeff, you may want to review this KB article about how Marketo uses the Salesforce API and then do some testing: 

https://community.marketo.com/MarketoArticle?id=kA050000000Kyr6CAC

Your updates will be batched in groups of 200, so even if you ran 100,000 all at once, you would only use 500 API calls in theory. 

We have far fewer API calls than you (only about 30K) and I semi-frequently run huge batches like a rescore that operate multiple updates across 40,000+ leads, and I have not run into problems. 

[Anonymous]

Thanks for the article, Justin.  I believe the following from that article pertains to my situation:

"If you use a Sync Lead to SFDC or Add to Campaign flow step, each one uses one API call per lead; the API call happens when the lead hits that flow step (instead of waiting for the next sync)."

For each stale contact, I'm running the "Sync Lead to SFDC" flow step which is a single API call.  If I have 60,000 stale contacts in my list, then I'll be making 60,000 API calls.  While this large number should be rare in the long-term, there is nothing to prevent it from occuring so it would be nice to have some way to cap it.

After further thought, I decided to go with the following approach:

(1) Created a "stale contact" smart list that grabs all contacts that haven't been updated in last 30 days.
(2) Created five empty static lists named Monday, Tuesday, etc.
(3) Create a campaign that will run weekly Monday at midnight.  It uses the "stale contact" smart list and breaks it up between the five empty static lists using the random sample flow.  Each static list will be 20% of the contacts in the "stale contact" smart list.
(4) Created five campaigns that each use a different static list and run weekly on a given day (Monday, Tuesday, etc) to do what I need to do.  For each contact in the static list for that day, it will sync it with SFDC and remove it from the static list.  If the contact is still in my stale contact smart list after the sync, then I remove it from Marketo.
(5) By the end of the week, each of the static lists will be empty and I start the process over again the next week.

Thanks for all the feedback.