Script for extracting all Lead Activities via Rest API?

[Anonymous]

Looking for some help in getting a script that will extract all out Lead Activities. I've been successful in manually retrieving the data​ as a test, however our Lead Database has 120k customers. I'm trying to get Send Email, Open Email, Click Email plus more for at least 8 different sends, this could easily be 1.2M+ rows in total data. Obviously this is not feasible to do manually given the 300 record limit per API call. I'm also aware of the 10k API calls a day limit.

At the moment, I'm doing:

https://<endpoint>.mktorest.com/rest/v1/activities.json?access_token=<access_token>&nextPageToken=<n...

I'm not a developer but do have basic to intermediate programming skills. Is there any sort of script out there (no script language preference) that I can modify and use for my purposes to extract all the Lead Activities?

[Anonymous]

Actually Python is now the preferred scripting language as I've had that installed!

[SanfordWhiteman]

Question should be in Products, not Community ("Community" here refers to the portal app itself, not the user community. Yes, it is confusing.)

300 * 10,000 = 3 million.  So in fact you can download those 1.2MM rows per day by iterating over the batches of 300, updating the token (which amounts to a database cursor) every turn of the loop -- provided that is the only thing you're using the API for.  The problem with the 10,000-call daily limit is when you need more than 3MM rows per day from a single integration, or you have a range of unmetered integrations, one or a combination of which could suck up all 10,000, representing a DoS against all the others.

You could also examine the offerings from CData, such as their Excel connector for Marketo.  I'm not sure what you mean by having intermediate programming skills but not being a developer. 9 out of 10 professional developers have intermediate programming skills at best, so this seems contradictory.

[Chris_Johnston]

Hi Carrie, we are working on an app that does something similar that could be adapted to do this. Because it isn't launching until Summit, if you would like to e-mail me, we can keep our app under wraps but might be able to help solve your problem before then.

chris @ salytics.com

[Anonymous]

Hi Carrie Flexmore,

You can try Data Loader for Marketo (DLM) which allows you to automatically retrieve data from Marketo without any coding. You can get activity data for a given time range and activity types (say those 8 email related activities) including standard and custom types via friendly UI. Besides, you can also schedule daily automatic job to update activity data, that would help you not hit the daily API call limitation.

Here are the links to the articles which help you to know how to achieve it.

Get Marketo Lead Activities via DLM now!

Automatize the Activity extraction from Marketo!​

Feel free to contact us for further information. Thanks!

Vivian

[Kurt_Koller]

Hi Carrie,

There is a Bulk Extract API now for activities. You can use a filter of up to 30 days at a time, and you can specify the activity types.

http://developers.marketo.com/rest-api/bulk-extract/bulk-activity-extract/

The design of the Bulk Extract API has some issues, but seems perfectly suited for your use case.