Asynchronous Data Import

[Jay00031987]

In Marketo, 10 API call can be processed simultaneously (2 parallel and 8 queue) but business need is to do bulk import (more than 10 API). Does Marketo support team allows Asynchronous process to archive this function.  The current Synchronous process is taking 2 to 3 min for 10 MB file but business requirement is to speed up the process and looking for suggestion. Asynchronous process can be helpful but any other suggestion will be welcome.

[SanfordWhiteman · Accepted solution]

You're using "asynchronous" in a sort of confusing way, because bulk import jobs are already asynchronous.

So "the current Synchronous process" doesn't exist.

That's why you can have a queue at all. Jobs are pulled off the queue asynchronously — otherwise, you wouldn't even get an HTTP response from a submission until each entire file is processed. The fact that you submit, then the back end does the jobs later is kind of the definition of asynchronous!

Now, if what you're asking is whether there can be some sort of other hierarchical queue, like a second-level queue, the answer is: Not within Marketo itself. You could certainly build your own queue that accepts an unlimited number of simultaneous jobs, and then pulls jobs off the queue only when Marketo signals that it has an open slot. As long as you submit to this second queue, it would appear to the client app as if there were no queue limit.