API questions: Programs and Lead Activity

Dan_Stevens_
Level 10 - Champion Alumni

API questions: Programs and Lead Activity

We're building out a data warehouse of lead activity that will be used for our dashboards/reports in Power BI.

QUESTION 1 (PROGRAMS): During our call today, we learned that the developers are having to make daily calls to pull all PROGRAMS into our DB.  I asked them why then aren't just grabbing new programs and appending them to a master Program table.  Then state that it's not possible.  While the GET programs call does return a "CreatedAt" date:

pastedImage_0.png

There's no way to make subsequent calls to only return those programs created since the last call (since there's no filter type to do so):

pastedImage_1.png

That makes no sense.  Are our developers looking at this the wrong way - and instead should be using another approach to get just new programs (rather than the entire list each day)?

QUESTION 2 (LEAD ACTIVITY): Ideally, we need to capture specific activity based on PROGRAM STATUS (and also attribute it to the appropriate PROGRAM).  For example, one of our metrics is to measure the number of RESPONSES/ENGAGEMENT (based on PROGRAM STATUS, eg., "fills out form", "engaged - downloaded", etc.) for each reporting period (daily, monthly, quarterly).  So we need to filter just that activity that contains any of the defined PROGRAM STATUSES.  What is the appropriate call(s) in order to grab something that ultimately looks like this (especially when ChannelID and/or ProgramID aren't included attributes)?:

ActivityID, LeadID, ProgramID, ProgramStatusID

The "GET Lead Activities" doesn't seem to contain all of these attributes - specifically the key attribute for us, PROGRAM STATUS.

Hopefully Sanford Whiteman​ (in addition to other API experts in the community) will provide some insight here.

Tags (2)
25 REPLIES 25
SanfordWhiteman
Level 10 - Community Moderator

Re: API questions: Programs and Lead Activity

Hey Dan,

IIRC, the Get Lead Activities stream contains the event Change Status in Progression, which is the old-school term for a change in Program Status. This includes the Program ID, so while it doesn't let you backfill programs that don't have any members, it will expose the ID of a program as soon as someone is added to it, at which the Asset API can be used to get the Program metadata.

Dan_Stevens_
Level 10 - Champion Alumni

Re: API questions: Programs and Lead Activity

Thanks for chiming in, Sandy.  The GET LEAD ACTIVITIES stream only returns the following:

pastedImage_0.png

Unless I'm looking at this wrong, I don't see anywhere information around Program/Progression Status.

Also, with regards to my first question on building an active PROGRAMS table in our data warehouse - and considering we have over 2,500 programs today - is it true that it's required to download this massive number each day (wasting precious API calls) vs. simply updating the DB with just those programs that have changed (or that have been added/deleted)?

SanfordWhiteman
Level 10 - Community Moderator

Re: API questions: Programs and Lead Activity

Unless I'm looking at this wrong, I don't see anywhere information around Program/Progression Status.

One of the activity types (activityTypeId 104 in my primary instance, though this is not guaranteed to be the same in all instances) is Change Status in Progression, with primary attribute Program ID and auxiliary attributes:

       [

        {

          "name": "Acquired By",

          "dataType": "boolean"

        },

        {

          "name": "New Status ID",

          "dataType": "integer"

        },

        {

          "name": "Old Status ID",

          "dataType": "integer"

        },

        {

          "name": "Reason",

          "dataType": "string"

        },

        {

          "name": "Success",

          "dataType": "boolean"

        },

        {

          "name": "New Status",

          "dataType": "string"

        },

        {

          "name": "Old Status",

          "dataType": "string"

        }

      ]

These activities are a substitute for notification that a new Program ID exists. So you don't have to download all the Program IDs every day to find new Programs, as long as you limit yourself to Programs that have had at least one member.

Detecting changes to the Program-level config, though, or if a Program has been deleted, I think requires download-and-compare via the Asset API like you're doing now.  You could decide to redownload the program when triggered by a Change Status activity (that is, the first time you see a Change Status for a Program on a given day, update the corresponding Program metadata, regardless of whether you have existing metadata).  Then maybe check for deletes once a week.

Dan_Stevens_
Level 10 - Champion Alumni

Re: API questions: Programs and Lead Activity

Thanks again, Sandy, but I'm afraid I'm still not following.  I'm going to loop in our developer, Grant Stephens​, to see if he can make better sense of this.

Grégoire Michel, have you any experience with what we're trying to do here?

Grégoire_Miche2
Level 10

Re: API questions: Programs and Lead Activity

Hi Dan,

Sorry for answering late, I was off with the flu.

May be would it be worth starting with the activity type endpoint http://developers.marketo.com/rest-api/endpoint-reference/lead-database-endpoint-reference/#!/Activi...  that will provide you with the complete list of codes and labels for activities in your instance.

-Greg

Anonymous
Not applicable

Re: API questions: Programs and Lead Activity

So the elements map as below for activity 104

primaryAttributeValueId: = programID

primaryAttributeValue = programName

SanfordWhiteman
Level 10 - Community Moderator

Re: API questions: Programs and Lead Activity

Right.

ETA: ... though this isn't really apparent from the endpoint documentation.  In any case you should only use the ID as the name may change.

Dan_Stevens_
Level 10 - Champion Alumni

Re: API questions: Programs and Lead Activity

It seems that we're still not able to pull in the appropriate data properly.  Here's a an example of what we're aiming for (this is from RCE):

pastedImage_0.png

Can this or can this not be accomplished use a combination of API calls?

Dan_Stevens_
Level 10 - Champion Alumni

Re: API questions: Programs and Lead Activity

Just to further clarify, when pulling lead activities, the desire is to include the "programName", "programID", and "Program Status".  I think it's that last attribute that's still unclear.  Where is "Program Status" found within the Lead Activity?  I'd have to think this would be key data as part of the data returned with Lead Activity, but it's not apparent by looking at the documentation.