Rest API - Bulk Import Status

[Jason_Ferrell]

I'm having an issue tracking the status of bulk imports and I'm curious if anyone else has had the same issue. 

When I create a bulk import and add to a static list, it appears that the batch id that is assigned to the action is the same as the list id.  I think that the bulk import has a queue of up to 10 jobs and can run 2 simultaneously.  However, since all the jobs get the same batchid, when I try to check the progress, I get odd results.  I can generally work around this for large imports that require multiple jobs.  It does create an issue when I try to check the status and it gives me the status of the job from yesterday.

/bulk/v1/leads.json?access_token=123&format=csv&listId=22038&lookupField=email

[Kenny_Elkington]

Do you have any examples demonstrating this?

[Jason_Ferrell]

I'm not exactly sure what you're looking for.  When you upload a CSV to the bulk import using the Rest API link shown above, the result is a JSON object that contains the requestId and three elements of the result:  batchId, importId, and status.  When uploading to a static list, the batchId and the importId are always the same number as the static list id.  For example, above I'm uploading to static list 22038.  When I create upload jobs for this list, both the batchId and the importId are always returned as 22038. 

This causes the status to be imperfect, since I don't get a unique job number for each upload.  There appear to be no parameters in the API to force a unique batchId for this type of upload.

[Kenny_Elkington]

Sorry if I was unclear, but I'd like to see some of the example requests and responses where you're observing this behavior, rather than just a description.

[Jason_Ferrell]

Here is a set of actions via R...attempt to upload file "01", then attempt to upload file "02"...appears to fail indicating that queue is not work...then a successful upload of file 02, which results in teh same batchId (equal to the listid).

This is the url string I'm using:

https://947-IOX-305.mktorest.com/bulk/v1/leads.json?access_token=ABCDEF&format=csv&listId=22038&look...

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "f0b5#15fb6ed37a8"

$result

$result[[1]]

$result[[1]]$batchId

[1] 22038

$result[[1]]$importId

[1] "22038"

$result[[1]]$status

[1] "Queued"

$success

[1] TRUE

> filename<-paste0(filepath,"discovery_integration_02.csv")

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "7784#15fb6edeb75"

$success

[1] FALSE

$errors

$errors[[1]]

$errors[[1]]$code

[1] "1019"

$errors[[1]]$message

[1] "Import is in progress"

> postresult<-POST(url,body=list(file=upload_file(filename)))

> batchId<-content(postresult)$result[[1]]$batchId

> content(postresult)

$requestId

[1] "82e#15fb6ef9ffd"

$result

$result[[1]]

$result[[1]]$batchId

[1] 22038

$result[[1]]$importId

[1] "22038"

$result[[1]]$status

[1] "Queued"

$success

[1] TRUE

[Kenny_Elkington]

Do you have the real requests and responses?  I have trust issues with variable dumps.

[Jason_Ferrell]

I'm having trouble getting the response in a non-variable format, but it's just a JSON object.  Here is the request, in part, with the 1000 lead CSV data omitted from the middle. 

-> POST /bulk/v1/leads.json?access_token=123&format=csv&listId=22038&lookupField=email HTTP/1.1

-> Host: 947-IOX-305.mktorest.com

-> User-Agent: libcurl/7.54.1 r-curl/2.8.1 httr/1.3.1

-> Accept-Encoding: gzip, deflate

-> Accept: application/json, text/xml, application/xml, */*

-> Content-Length: 396883

-> Content-Type: multipart/form-data; boundary=------------------------d16b91702e24668e

->

>> --------------------------d16b91702e24668e

>> Content-Disposition: form-data; name="file"; filename="discovery_integration_01.csv"

>> Content-Type: text/csv

>>

>> "accountnumber","company",

...

>>

>> --------------------------d16b91702e24668e--

<- HTTP/1.1 200 OK

<- Server: nginx

<- Date: Tue, 14 Nov 2017 15:23:13 GMT

<- Content-Type: application/json;charset=UTF-8

<- Content-Length: 113

<- Connection: keep-alive

<-

[Jason_Ferrell]

Just wanted to see if anyone else has any thoughts about this issue.  Fundamentally, the batch id that is returned by the Bulk Lead Import API cannot be the static list id because it makes it difficult to track status of the job.