BIRST Marketo data connector extract failing repeatedly

[Anonymous]

I have been trying to set up Marketo application connector and pull in sample of various objects into birst. I was able to connect and also bring in basic objects like Program and Program tags, but I am not able to get Lead object (which is close to 2M in all). I have tried multiple options like (reduced # of columns, lesser number of rows based on timestamp limitation etc). 

The process runs for day or more and even show that its pulling 150k+ leads and then suddenly fails. Since its failing at extraction, I dont see that staged data available for data processing either. Can anyone help please?

Are there any best practices that you guys use if you can provide as an outline? Any help is appreciated.

Best

-Prasad

[Anonymous]

I would recommend try a shorter timerange first (a day or week).

[Anonymous]

Thanks for quick reply Murtza,

I am wondering if "LastModifiedDateTime" the only way to limit these rows? When I try to look for all columns on "Lead" object, this modified-date-time is not one of the columns. Do you what other columns that are better for limiting row sets and also increase performance?

On a good news front, when I pulled in limited set of columns for just 1 day range, I got about 220K+ leads; however within the data flow I am not able to enable that data source nor am I able to set primary key etc.

[Anonymous]

Also Murtza,

Can you please tell me how Lead object is linked to various Activity_ objects? Is the activity_id and lead_id same across all tables? I dont know if I am asking details that are not usually disclosed, but I am essentially looking for Marketo datamodel.

[Edward_Masson]

This should help you (unless you have it already-sorry)
http://developers.marketo.com/documentation/soap/

[Anonymous]

Hello Edward,

Please dont apologize and you were definitely helping. That link provided gives valuable information for developers who are using SOAP API and writing custom code. We are using a DW SAAS solution called Birst and they got a connector built to access Marketo via SOAP.

When I try to pull leads using "LastModifiedDateTime" for a single day span, its pulling so many leads, that its puzzling me. So I am trying to research how the leads-> activity objects are linked.