How do I page leads in API results?

[Anonymous]

I've gone through the API example code, but have not seen a way to page through many results when using getMultipleLeads().  Presumably this is handled with streamPosition, but the WSDL defines that field as being of type "string".  What have I missed?

I see a startPosition field for the getLeadActivity() and getLeadChanges() methods, but surely getMultipleLeads() also has this capability.

I'm using C#, so examples in .NET are ideal, but I'll take what I can get.  Thank you!

[Anonymous]

Related post: https://community.marketo.com/MarketoArticle?id=kA050000000L6LH&src=

[Anonymous]

When you make the API call you'll get the streamPosition back and then you need to pass that as a token back into the subsequent call.

[Anonymous]

There seems to be more to it; capturing the previous loop's result.newStartPosition value and including it in the following call's parameters, the result.remainingCount value is not reduced.  Can you see what I'm missing?

public IEnumerable<Activity>  GetActivities(int leadId)

{

ParamsGetLeadActivity  parameters = new ParamsGetLeadActivity();

parameters.leadKey = CreateLeadKey(leadId.ToString());

while (1==1)

{

SuccessGetLeadActivity  response = _client.getLeadActivity(CreateHeader(), parameters);

LeadActivityList  result = response.leadActivityList;

foreach (var activityRecord in result.activityRecordList)

yield return new Activity(activityRecord);

// Are there any additional results we'll need to page?

if (result.remainingCount == 0)  yield break;  // Yes, drop out

// No, advance our start position and get more results

parameters.startPosition = result.newStartPosition;

}

}

If there's actual API documentation, beyond the WSDL and the examples, that would be great, please point me to it.
 

[Anonymous]

Another related thread:
https://community.marketo.com/MarketoDiscussionDetail?id=90650000000PFutAAG

This seems to describe the same problem, but it is unsolved.  I would have reopened that thread if I'd found it earlier, my bad.

[Anonymous]

You have the right link for the documentations- there is a PDF attached to that article that details all calls and provides additional information.

Can you post the XML for your requests and responses? It tends to be easier than trying to troubleshoot the code itself.

[Anonymous]

Aha, thank you.  I went through the block of links but didn't see the attachment at the end.  Unfortunately, it gives no additional information about paging beyond your reply on Friday (though the typos are amusing).

I found and removed a complication: I was reading tasks' activities in nested loops.  The tasks are actually paging OK, but not activities.  I've swapped the process to spool leads in their entirety before looping through them again to read activities - and that's where the real problem is.

Checking the IDs returned, it looks like paging is working for activities, but imperfectly: the second page has the same results as the first, and the third page returned has the second page of actual results.  This is the case even if I manually specify startPosition.offset: I get the same results for task #2041892 if offset is 0 or 100.  The pattern holds if I use a batch size of 50 or 20: the second page repeats the first, and subsequent pages proceed from there.

That's easily fixed, I simply track which page number I'm on and don't return the second.  getMultipleLeads does not seem to have this problem, just getLeadActivity so far.

Unfortunately, the .Net layer wrapping a WSDL is opaque; it's really quite complex to get to the raw XML.  Here's a typical request for getMultipleLeads:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">

<s:Header>

<h:AuthenticationHeader xmlns:h="http://www.marketo.com/mktows/" xmlns="http://www.marketo.com/mktows/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<mktowsUserId xmlns="">volusion1_604057994F982D94426080</mktowsUserId>

<requestSignature xmlns="">df9b34f783a186cf2661dc33849c2e329c74ec3f</requestSignature>

<requestTimestamp xmlns="">2013-01-14T21:36:33.069Z</requestTimestamp>

</h:AuthenticationHeader>

</s:Header>

<s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<paramsGetMultipleLeads xmlns="http://www.marketo.com/mktows/">

<lastUpdatedAt xmlns="">2013-01-14T00:00:00Z</lastUpdatedAt>

<streamPositionxsi:nil="true"xmlns=""/>

</paramsGetMultipleLeads>

</s:Body>

</s:Envelope>

Getting to the response is harder, not least because of the Gzipping.

[Anonymous]

Hmm, that makes troubleshooting this hard. Plus, if your Gzipping we probably can't use Wireshark or something to capture the XML.

I was hoping to look at how you were using the streamPosition. I know that this works, in general, but we're probably losing something in the various levels of wrappers around the API.

Have you tried a dead simple version of getMultipleLeads where you call it three times, each time passing the stream position to the next one (without editing it)? Does it not page in that case?

[Anonymous]

Sorry, leads are actually a red herring here: once I peeled out leads and accounts, I found the problem was with the latter.

Indeed, I'm just copying the previous call's result.newStreamPosition or result.newStartPosition, as appropriate, into the next call's parameters.streamPosition or parameters.startPosition.

My final working code is much the same as above, but it tracks the page number and skips returning the second:

public IEnumerable<Activity>  GetActivities(int leadId)

{

StreamPosition  pos = null;

int  pageNum = 1;

while (1==1)

{

ParamsGetLeadActivity  parameters = new ParamsGetLeadActivity();

parameters.leadKey = CreateLeadKey(leadId.ToString());

parameters.startPosition = pos;

SuccessGetLeadActivity  response = _client.getLeadActivity(CreateHeader(), parameters);

LeadActivityList  result = response.leadActivityList;

if (pageNum != 2)  // The second page of results returned repeats the first

foreach (var activityRecord in result.activityRecordList)

yield return new Activity(activityRecord);

// Are there any additional results we'll need to page?

if (result.remainingCount == 0)  yield break;  // No, drop out

// Else

pos = result.newStartPosition;

++pageNum;

}

}