Rest API - Bulk Import Status

Highlighted

Rest API - Bulk Import Status

I'm having an issue tracking the status of bulk imports and I'm curious if anyone else has had the same issue. 

When I create a bulk import and add to a static list, it appears that the batch id that is assigned to the action is the same as the list id.  I think that the bulk import has a queue of up to 10 jobs and can run 2 simultaneously.  However, since all the jobs get the same batchid, when I try to check the progress, I get odd results.  I can generally work around this for large imports that require multiple jobs.  It does create an issue when I try to check the status and it gives me the status of the job from yesterday.

/bulk/v1/leads.json?access_token=123&format=csv&listId=22038&lookupField=email

7 REPLIES 7
Highlighted
Marketo Employee

Re: Rest API - Bulk Import Status

Do you have any examples demonstrating this?

Highlighted

Re: Rest API - Bulk Import Status

I'm not exactly sure what you're looking for.  When you upload a CSV to the bulk import using the Rest API link shown above, the result is a JSON object that contains the requestId and three elements of the result:  batchId, importId, and status.  When uploading to a static list, the batchId and the importId are always the same number as the static list id.  For example, above I'm uploading to static list 22038.  When I create upload jobs for this list, both the batchId and the importId are always returned as 22038. 

This causes the status to be imperfect, since I don't get a unique job number for each upload.  There appear to be no parameters in the API to force a unique batchId for this type of upload.

Highlighted
Marketo Employee

Re: Rest API - Bulk Import Status

Sorry if I was unclear, but I'd like to see some of the example requests and responses where you're observing this behavior, rather than just a description.

Re: Rest API - Bulk Import Status

Here is a set of actions via R...attempt to upload file "01", then attempt to upload file "02"...appears to fail indicating that queue is not work...then a successful upload of file 02, which results in teh same batchId (equal to the listid).

This is the url string I'm using:

https://947-IOX-305.mktorest.com/bulk/v1/leads.json?access_token=ABCDEF&format=csv&listId=22038&look...

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "f0b5#15fb6ed37a8"

$result

$result[[1]]

$result[[1]]$batchId

[1] 22038

$result[[1]]$importId

[1] "22038"

$result[[1]]$status

[1] "Queued"

$success

[1] TRUE

> filename<-paste0(filepath,"discovery_integration_02.csv")

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "7784#15fb6edeb75"

$success

[1] FALSE

$errors

$errors[[1]]

$errors[[1]]$code

[1] "1019"

$errors[[1]]$message

[1] "Import is in progress"

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "82e#15fb6ef9ffd"

$result

$result[[1]]

$result[[1]]$batchId

[1] 22038

$result[[1]]$importId

[1] "22038"

$result[[1]]$status

[1] "Queued"

$success

[1] TRUE

Highlighted
Marketo Employee

Re: Rest API - Bulk Import Status

Do you have the real requests and responses?  I have trust issues with variable dumps.

Highlighted

Re: Rest API - Bulk Import Status

I'm having trouble getting the response in a non-variable format, but it's just a JSON object.  Here is the request, in part, with the 1000 lead CSV data omitted from the middle. 

-> POST /bulk/v1/leads.json?access_token=123&format=csv&listId=22038&lookupField=email HTTP/1.1

-> Host: 947-IOX-305.mktorest.com

-> User-Agent: libcurl/7.54.1 r-curl/2.8.1 httr/1.3.1

-> Accept-Encoding: gzip, deflate

-> Accept: application/json, text/xml, application/xml, */*

-> Content-Length: 396883

-> Content-Type: multipart/form-data; boundary=------------------------d16b91702e24668e

->

>> --------------------------d16b91702e24668e

>> Content-Disposition: form-data; name="file"; filename="discovery_integration_01.csv"

>> Content-Type: text/csv

>>

>> "accountnumber","company",

...

>>

>> --------------------------d16b91702e24668e--

<- HTTP/1.1 200 OK

<- Server: nginx

<- Date: Tue, 14 Nov 2017 15:23:13 GMT

<- Content-Type: application/json;charset=UTF-8

<- Content-Length: 113

<- Connection: keep-alive

<-

Highlighted

Re: Rest API - Bulk Import Status

Just wanted to see if anyone else has any thoughts about this issue.  Fundamentally, the batch id that is returned by the Bulk Lead Import API cannot be the static list id because it makes it difficult to track status of the job.