Best practices for managing API calls to minimize "web services" errors

[Dan_Stevens_]

As we integrate other platforms with Marketo, we're finding it very difficult to configure the individual systems to minimize the various web services errors (as a result of exceeding concurrent, daily and other limits placed on the calls made to Marketo) - especially given the lack of a comprehensive API "dashboard"/management console.  And just today, we learned that MSI contributes to those API calls as well.  Can anyone share some best practices on how you manage this effectively?

[Dan_Stevens_]

With regard to API call limits, when making a single "lead activities" call that returns 300 results, that will count as a single call against our 10,000 (not 300), correct?

[Grégoire_Miche2]

HI Dan,

Yes this count for 1, even if you fetch 300 records. Attention, though, not all endpoint can retrieve multiple leads or activities per call. the developers have to read the doc carefully on this.

-Greg

[Nicholas_Manojl]

A good practice would be to use a seperate API only user for each of your individual systems. This will help you troubleshoot which of your systems are firing off and consuming your quota. They don't have to be a real email address.

Failing that, support was able to tell me which API endpoint was consuming lots of data for me when I made a similar request once.

[Grégoire_Miche2]

Agreed, this is a basic rule to be enforced.

-Greg

[Dan_Stevens_]

Yep - we do that today for each service that connects to Marketo.

[Steven_Vanderb3]

MSI calls should be marked as internal in our system and not count towards your API limits.

[Dan_Stevens_]

Even though it's included in the totals here?

Marketo Support just informed us that the highlighted row is the MSI API.

[Devraj_Grewal]

Dan, where is this dashboard viewable? Is this within Marketo?

[Dan_Stevens_]

It's not very evident - you need to click on the "number of requests made in the last 7 days":

[Nicholas_Manojl]

It's in the Admin section, under Web Services.

[Steven_Vanderb3]

Yeah but I don't think that's supposed to be counting towards your limits.  I see CJ is working with engineering to investigate the issue for you.

[Ben_Griffith1]

We are having similar issues with a client.  Currently we are looking at increasing the concurrent limit from 10 to 30.  Daily calls have already been increased to 300K.

Additionally, we are planning to move calls that pull data directly from Marketo to instead pull from a data warehouse that is already pulling data from Marketo.

For custom API calls that are driven from webhooks initially we are planning to queue and process to prevent excessive concurrent connections.

[Grégoire_Miche2]

Agreed on this: it's easier to optimize interactions between Marketo and a datamart than to manage ad-hoc queries ran directly towards Marketo from various clients.

-Greg

[Dan_Stevens_]

Wow, 300K!  That probably costs as much as the actual Marketo license!  Do you know if there's a standard pricing structure for increasing the calls?  I've already reached out to our Account Exec - just waiting to hear back.

Also, does Marketo offer an initial database dump from a specified date?  Our challenge now is to backfill data from September 1, 2016.  I feel we're never going to get caught up.

[Ben_Griffith1]

I'm not sure about the pricing, but I think increasing to 100K is relatively reasonable.

I have not seen a database dump supplied, but you can process quite a bit with more API calls when you are pulling in batches of hundreds depending on the API calls being used.