How to speed it up when getting multiple Leads by SOAP API?

[Anonymous]

Hi all,

Now getMultipleLeads() costs me nearly 4 minutes per 1000 records. And I have more than 500k records updated per a week. I need to get them out in 24 hours at most.
Is there any way to speed it up?

Best Regards,
Biao

[Anonymous]

What is the environment running API calls? I am used to have faster execution times for getMultipleLeads.

[Anonymous]

VS2010 with C#

[Anonymous]

I am not in position to compare any results since I am Solaris, Linux and Mac OS X user therefore far away from any Microsoft products. However I can comfortably state there is noticeable performance penalty using Eclipse IDE (Helios and Kepler), in some ways similar to Visual Studio. I would suspect the slowness would be a mix of network bandwitdh and Visual Studio itself.

I may be totally wrong and will gladly accept any comments from someone familair with Microsoft environments. 

[Anonymous · Accepted solution]

Biao,
Here are our recommendations -

1. If you know which lead attributes you need to execute your business logic, you can specifying only those attributes you wish to be returned with each lead.  This helps in two very big ways.  The performance is much faster and you will also reduce the response payload.  This will give you the biggest performance boost with the least implementation change.
 
                Sample Input XML:
                                <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
       <ns2:paramsGetMultipleLeads xmlns:ns2="http://www.marketo.com/mktows/">
           <leadSelector xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:LastUpdateAtSelector">
              <latestUpdatedAt>2013-09-08T16:13:52.977-07:00</latestUpdatedAt>
               <oldestUpdatedAt>2013-09-05T16:13:52.978-07:00</oldestUpdatedAt>
           </leadSelector>
           <batchSize>200</batchSize>
           <includeAttributes>
               <stringItem>FirstName</stringItem>
              <stringItem>AnonymousIP</stringItem>
               <stringItem>Company</stringItem>
           </includeAttributes>
       </ns2:paramsGetMultipleLeads>
 
 
2. Use the <leadSelector> field to specify dates instead of <lastUpdatedAt>.  (This alone won't help performance – however <lastUpdatedAt> is deprecated)
                                Sample Input XML:
                                <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
       <ns2:paramsGetMultipleLeads xmlns:ns2="http://www.marketo.com/mktows/">
           <leadSelector xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:LastUpdateAtSelector">
              <latestUpdatedAt>2013-09-08T16:13:52.977-07:00</latestUpdatedAt>
               <oldestUpdatedAt>2013-09-05T16:13:52.978-07:00</oldestUpdatedAt> <!-- The timestamp value you were specifying in lastUpdatedAt -->
           </leadSelector>
           <batchSize>200</batchSize>
       </ns2:paramsGetMultipleLeads>
 
3. If you want to really speed things up now that you are using leadSelector is to specify a date range.  <oldestUpdatedAt/> is the since time stamp and <latestUpdatedAt/> is the until time stamp.  With this approach you could thread multiple calls for different time ranges.  If you choose this approach I highly recommend you implement your SOAP client using persistent SOAP connections.  This is a bit more complex, but the rewards can be significant for processing larger data sets.
 

[Anonymous]

Thanks, bgomes.

And hi Raj,
I've tried your first way, and when taking 1/10 of the attributes, about half time would be saved. So I guess something else also costs large time. Thank you very much.

Best Regards,
Biao