SOAP API getMultipleLeads problem newStreamPosition is empty, remainingCount is positive

[Anonymous]

I'm using a 2_3 SOAP endpoint with a python client powered by suds. The response to my initial getMultipleLeads call looks like:

In [35]: type(response)

Out[35]: suds.sudsobject.ResultGetMultipleLeads

In [36]: response.remainingCount

Out[36]: 8

In [37]: response.newStreamPosition is None

Out[37]: True

I found an earlier disscussion with no answer on the same topic:
https://community.marketo.com/MarketoDiscussionDetail?id=90650000000Pp6gAAC

What's the right thing to do here?

[Anonymous]

Did this first call return leads, or just the remaining count? 

[Anonymous]

It returned leads. 

response is the initial response, and I used a batchSize of 100. I got 100 lead records back.

In [39]: len(response.leadRecordList.leadRecord)

Out[39]: 100

In [40]: response.returnCount

Out[40]: 100

[Anonymous]

You would need to iterate through the rest of leads available. Use the token returned in this call to get the next 100.

[Anonymous]

You should see something in your first response xml:

<newStreamPosition>Somevaluehere</newStreamPosition>

Then pass this value as <streamPosition>Somevaluehere</streamPosition> in the second API request. 

[Anonymous]

The problem is that my response does not include a token. This might be a marketo bug?

You could also take a look at https://community.marketo.com/MarketoDiscussionDetail?id=90650000000Pp6gAAC which looks like the same problem and has XML snippets. Please let me know if it would help for me to provide a more detailed code example.

In [43]: lead_record_length, return_count, remaining_count, new_stream_position = len(response.leadRecordList.leadRecord), response.returnCount, response.remainingCount, response.newStreamPosition

In [44]: (lead_record_length, return_count, remaining_count, new_stream_position)

Out[44]: (100, 100, 8, None)

[Rick_Bergen]

I've found that if the returnCount is less than the batchSize I can conclude that there are no more records to get despite the remainingCount being greater than zero.  The remainingCount is notoriously unreliable with the excuse being it is an "estimate".

This is the hack I put in my code to avoid the somewhat infinate loops I was getting:

                    if (returnCount < batchSize)
                    {
                        remainingCount = 0;
                    }


A better solution may be to look for a blank newStreamPosition as suggested in the referenced article as I suppose having exactly a batch size of records is possible.