programmatically get smart list total people count

[dtavangar]

Hello all, I understand there are no API endpoints available to get the total people to count for Smart-lists. I was wondering if anyone has found a workaround for this. I am more looking for a programmatic approach. Would it be possible to create a batch campaign to add the Smart-list members to a list (list name would be predefined) and then query the list by name to get the total member count? any other approaches? 

Thank you In advanced. 

[SanfordWhiteman · Accepted solution]

You could have a nightly batch that passes all members of the Smart List through a webhook-compatible service. The service would merely increment a counter variable for every call. (And of course you’d want to zero out the counter each day.) Then you’d query the service instead of Marketo. It’s pretty easy to set up but obviously is outside of Marketo proper.

Then again, depending on the typical size of your Smart List, you could just enqueue the SL each day using the Bulk Export API + smartListId filter and poll the status. The status.json endpoint returns a numberOfRecords value. The issue with this is large SLs take a long time to export. Since you don’t ever need the file itself, that’s a lot of wasted time/resources, especially if you’re running other export jobs.

[NagaprabhuK]

Why Marketo always recommends hard way of doing things instead of bringing in a direct way?  We are spending a lot of money on your subscriptions and yet there isn't any improvements in terms of tool's UI and API supports. And the crazy part is whenever we search for something the questions were asked in Marketo Nation like long time ago, yet the enhancements are not done. There is no visibility on Enhancement requests that are placed through TAMs and there isn't even a roadmap of when that request will be fulfilled. 

Also, I'm not sure why these threads are marked as "Answered" or "Correct Answer"

[SanfordWhiteman]

---

Also, I'm not sure why these threads are marked as "Answered" or "Correct Answer"

---

Because a correct answer was provided! You can get the count using the REST API (that’s what programmatic approach means). There’s another method mentioned which is also correct. There’s no “more correct” answer than these two.

In other cases, the correct answer to Is <thing> currently possible? is a firm No. That’ll also be marked correct.

As a technical note, Marketo Smart Lists are dynamic SQL queries across a virtually unlimited number of tables (not just the main person table but custom objects, accounts, and the time series of activities). There’s no database in the world that lets you fetch the row count as of <now> without running the query now. Of course it’s not always necessary to return all fields if you’re only checking the count. But it’s absolutely necessary to run the query — not sure why anyone would imagine there’s a way around that.

[NagaprabhuK]

"There’s no database in the world that lets you fetch the row count as of <now> without running the query now."

One of your competitor company was providing it through a direct REST API and we were using it (through a programmatic approach). As because your systems couldn't provide, it doesn't mean that the entire world couldn't. 

"Because a correct answer was provided! You can get the count using the REST API (that’s what programmatic approach means). There’s another method mentioned which is also correct. There’s no “more correct” answer than these two."

The answer marked as "correct" is not clear it is mentioned to use "bulk export API + smartListId filter" what does this mean? There are 4 bulk export API in your documentation. Can you give us more details on how it works.

[Jo_Pitts1]

@NagaprabhuK ,

Firstly, you need to be careful about the use of the word 'your'.  These questions are not being answered by Adobe people, but by fellow community members who are giving up their time, knowledge, and energy freely to help you.  If you want to talk improving feature sets (and trust me, we all do), then it is best to take it up with your Adobe account rep.

Secondly, YES - It would be nice if there was a 'get smart list count' API call.  If such a thing existed (and as you've noted, some other competing platforms offer this) it would, to @SanfordWhiteman's point, be running a select count(*) on the underlying dataset.  A count(*) operation still runs the entire query, it just doesn't worry about returning the records.

You can achieve the same thing by enqueuing a bulk export job with a smart list filter.  @SanfordWhiteman has provided you a link to the API call.  It's not as elegant as a direct call, but will get you the same result.  

I understand your frustration, but you have been given a couple of solid answers that works within the constraints of the platform.  I think the next steps for you are to work out how to apply one or both to your situation.  If you get stuck at that point and ask targeted questions, people will be happy to help.

Cheers

Jo

[NagaprabhuK]

@Jo_Pitts1 
I do understand that the questions are replied by some random person. But the place I'm posting these questions is in "Adobe's" authorized forum. So I don't think it matters who is replying. So, the "your" refers to the authorized forum's tool in which customers like us invest a huge money.

Talking about the support requests, we had a lot of support requests placed and some are upgraded as an Enhancement request as well. The requests remain as is and the worst part is there isn't even a ETA on when it will be added to Adobe's roadmap atleast. 

A lot of people asked similar questions in this community (a longtime ago) and we have placed a support request as well to see if such requests are considered atleast now to fix. Shockingly, we received this same reply constantly "this is an expected behavior", "currently it's not available" blah blah blah, but not a solution! 

[SanfordWhiteman]

---

One of your competitor company was providing it through a direct REST API and we were using it (through a programmatic approach). As because your systems couldn't provide, it doesn't mean that the entire world couldn't. 

---

There’s no app that doesn’t run a query to get the real-time count of a Smart List-equivalent dynamic list. Point to the competitor’s endpoint and I’ll explain how it‘s running the query when you hit a so-called “direct” REST API endpoint. Marketo’s API runs the query asynchronously using a job queue, while other apps attempt to run the query synchronously (i.e. during the course of a long-lived HTTP connection). But they’re all running the query.

"Because a correct answer was provided! You can get the count using the REST API (that’s what programmatic approach means). There’s another method mentioned which is also correct. There’s no “more correct” answer than these two."

The answer marked as "correct" is not clear it is mentioned to use "bulk export API + smartListId filter" what does this mean? There are 4 bulk export API in your documentation. Can you give us more details on how it works.

---

Create Export Lead Job. Enqueue the job and poll its Status. You do not need to download because the record count is clear in the Status.

[NagaprabhuK]

Thanks for providing the exact BULK API. I'm not sure if the count is going to be real-time.

I don't think I need to know how it process the REST APIs, all I was trying to tell is one of "Adobe's" competitor is able to do it and why not such huge companies like Adobe can't find a fix or solution for the past 2years. Here is the screenshot of the API documentation from Adobe's competitor company which provides a direct endpoint and final audience count.

[SanfordWhiteman]

You accidentally cropped the part of of their docs that shows you've made a category error. See red arrow below:

They’re returning stale data and lastCalculatedAt shows you how stale it is. That’s not a real-time reflection of <competitor> database at the point you hit the API endpoint, because it’s not recalculating the segment membership.

You see, Segments in <competitor> aren’t the same as Smart Lists in Marketo (nor are they the same as Segments in Marketo):

• Marketo Smart Lists are automatically updated when they’re used, based on the real-time contents of the database.
• In contrast, <competitor> Segments are updated lazily, using a background timer.
• Marketo Segments are also updated lazily using a background timer, but they operate only on flat properties, not on activities.

This means a <competitor> Segment can be stale at the time it’s used to create an audience. It can contain contacts who, if you refreshed the Segment now, would no longer be in the Segment — or vice versa, it can be missing contacts who would be in the Segment based on their current state.

This can’t happen with a Marketo Smart List. When you use the Smart List in a Program, Smart Campaign or Report, or nest it another Smart List, membership is always fresh.

Thanks for providing the exact BULK API. I'm not sure if the count is going to be real-time.

---

Of course it’s real-time as of when the job completes, because that’s what fetching Smart List members means in Marketo. Calculating members is not done lazily as it is in <competitor>.

When you submit a Bulk Export job, it’s getting you everybody who qualifies at the time the job is processed, based on the real-time contents of the database. Just like when you run a Marketo Smart Campaign, the membership is as of the time of the run, not as of some arbitrary “calculation” point in the past.