Getting all Leads via PHP API Client

[Anonymous]

I'm using the PHP API Client to connect to Marketo and pull all my leads. It's connecting ok, and it does pull data, but I only get about 240 leads. I should get 24000 or so. I placed the code I used below, and I have killed the process at various points, var_dump'ing both $success and $result, but it always shows a returnCount of ~239. I poste the beginning the the var_dump below the code.

What am I doing wrong that I can't pull all the leads?

Thanks for any help!

************************

Here is the function I wrote into the API Client for getMultipleLeads:
public function getMultipleLeads()

 {

  $retLead = null;

    $params = new paramsGetMultipleLeads();

    $params->batchSize = 1000; //Max 1000, default 1000

    $options = array();

    $authHdr = $this->_getAuthenticationHeader();

  $result = array();

  $streamPosition = 0;

  do

  {

    $params->streamPosition = $streamPosition;

    $success = $this->soapClient->__soapCall('getMultipleLeads', array($params), $options, $authHdr);  

    $result[] = $success;

    $streamPosition = $success->result->newStreamPosition;

  }

  while($success->result->returnCount > 0);  //I also used remainingCount, but got similar results
...it returns the $result array

**********************

var_dump results:

object(stdClass)#18 (1) {
  ["result"]=>
  object(stdClass)#17 (4) {
    ["returnCount"]=>
    int(237)
    ["remainingCount"]=>
    int(0)
    ["newStreamPosition"]=>
    string(34) "id:1302116:ts:1378308540:os:0:rc:0"
    ["leadRecordList"]=>
    object(stdClass)#21 (1) {
      ["leadRecord"]=>
      array(237) {  //After this is all the lead data

[Anonymous]

Please log a support ticket

[Anonymous]

Actually, I think I resolved it. I had to pass the lastUpdatedAt. WIthout it, it seems to pull only 'today'. By specifying the lastUpdatedAt, I now get 'all' the leads, and I'm now wrangling a:

Gateway Time-out

The gateway did not receive a timely response from the upstream server or application.

Thanks!

[Anonymous]

Create a campaign to place your leads into a Static List and then use the getMultipleLeads by Static List option.  This option is better suited/optimized to get a large number of leads back from Marketo.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:mkt="http://www.marketo.com/mktows/">

   <soapenv:Header>

      <mkt:AuthenticationHeader>

         <mktowsUserId>XXXXXXXXXXXXXXXXXXXX</mktowsUserId>

         <requestSignature>XXXXXXXXXXXXXXXXXXXX</requestSignature>

         <requestTimestamp>XXXXXXXXXXXXXXXXXXXX</requestTimestamp>

      </mkt:AuthenticationHeader>

   </soapenv:Header>

   <soapenv:Body>

      <mkt:paramsGetMultipleLeads>

         <leadSelector xsi:type="mkt:StaticListSelector" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

           <!-- <staticListName>?</staticListName> -->

           <staticListId>1309</staticListId>

         </leadSelector>

         <batchSize>100</batchSize>

      </mkt:paramsGetMultipleLeads>

   </soapenv:Body>

</soapenv:Envelope>

[Anonymous]

We're trying to avoid a static list, so our MArketing team doesn't have to manually update it constantly. I got the PHP client to run multiple records fine, but I get the Gateway timeout after a few passes. My best so far is 9522 records returned. I'm currently running in 1000 record batches, to keep the number of API calls down. This morning, I only got 2000. I inserted a sleep(10) statement to causing the calls to stagger a bit, too.

My current logic calls the getMultipleLeads, for 1000 record batchsize, then inserts or updates my local database, then calls for the next 1000, using the streamPosition. I had hoped that would cause a delay that might improve the API response. I'm also trying to limit the number of fields I return, but our long term goal might be 10-15 fields.

[Anonymous]

Marketing can create operational Smart Campaign to move leads to the Static List therefore not having to manually update anything.

Static List option is recommended for larger lead extracts to avoid timeouts.  

NOTE:  The following may not be applicable for this use case since you may want to keep the entire static list intact but a populare lead extraction technique is to use the listOperation API call to remove leads retrieved from each GetMultipleLeads from the Static List API Call... this is a popular method to ensure that all leads are retrieved from a static list.