Conflicting responses to BULK API calls: Polling Job Status v.s. Enqueue Job

[JohnG]

When building a BULK API extraction job, I run into conflicting responses to the status of the job. 

When I first built the job, the enqueued it I followed up with a simple check status and got this response. 
{"requestId":"c0f3#173de0fc479","result":[{"exportId":"f49eb72e-6554-42a6-aba9-260c27890309","format":"CSV","status":"Created","createdAt":"2020-08-11T14:38:27Z"}],"success":true}

Parsed out to the relevant part: "status": "Created"

I was confused, thinking I had already Enqueued this job, so I tried to enqueue it again and I got this (Failed) result. 

{"requestId":"17a7c#173de129713","success":false,"errors":[{"code":"1029","message":"Export 'f49eb72e-6554-42a6-aba9-260c27890309' is in Processing status"}]}

Again the Cut down portion to:   is in Processing status"}]}

Okay, Strange, I polled the job status again just to see and I got the same response as the first. Telling me the job was in "created" status. 

I called for the job to enqueue again one last time, this time I told me "is in Completed status" 

Which is great, I'm going to try to pull the information from it, but why is the job polling telling me the result was successful, but then not telling me the updated job status? 

[Amit_Jain]

So the steps you followed are:

1. enqueued the job and got following response:
   {"requestId":"c0f3#173de0fc479","result":[{"exportId":"f49eb72e-6554-42a6-aba9-260c27890309","format":"CSV","status":"Created","createdAt":"2020-08-11T14:38:27Z"}],"success":true}
   
2.  You again tried to enqueue job and got following response:
   {"requestId":"17a7c#173de129713","success":false,"errors":[{"code":"1029","message":"Export 'f49eb72e-6554-42a6-aba9-260c27890309' is in Processing status"}]}
3. You tried to get the status again and it said "Created" 
4. You again tried to enqueue it again and got "is in Completed status" 

So when you create the job or try to enqueue the job, you will get "status: created" in response. But once you have enqueued the job, the status might change Queued, Processing, Cancelled, Completed, or Failed from "Created". 

The error you got when you tried to enqueue it again, it was because the job with that exportId was already in progress. You should be hitting check status endpoint, again and again to check the status until it's "Completed" or "Failed". But should not enqueue it again. Please try it again. 

[JohnG]

Thank you very much for the response, but I want to clarify. I was using the polling job status API call, and it was providing wrong, or outdated information. 

@Amit_Jain wrote:

So the steps you followed are:

1. enqueued the job and got following response:
   {"requestId":"c0f3#173de0fc479","result":[{"exportId":"f49eb72e-6554-42a6-aba9-260c27890309","format":"CSV","status":"Created","createdAt":"2020-08-11T14:38:27Z"}],"success":true}
   
2.  You again tried to enqueue job and got following response:
   {"requestId":"17a7c#173de129713","success":false,"errors":[{"code":"1029","message":"Export 'f49eb72e-6554-42a6-aba9-260c27890309' is in Processing status"}]}
3. You tried to get the status again and it said "Created" 
4. You again tried to enqueue it again and got "is in Completed status" 

I was polling "Job Status" in between each Queue call. 
1.) Enqueue Job and got a successful response. 

2.) A moment later check "status via 

 /bulk/v1/leads/export/{exportId}/status.json​

This Check status showed the Job to be in "created status" 
Since I was worried that my Enqueue job API call did not go through, I tried to enqueue again. 
When I would try to queue the job a 2nd time I would get an "error" but I would also get accurate status updates
It seems like for some reason Job Status is updating is being lagged or delayed. 

TLDR: When a job is enqueued, polling the status of the Job provides inaccurate information. Trying to enqueue the job again will provide an error, but will give accurate status along with the error. 

[Tony_Mayse · Accepted solution]

Export Job statuses returned by the status endpoint are not instantaneous results. The results returned are a snapshot in time of a recent status. The intent of the status endpoint is to monitor for completion. 

You can trust the status returned by the enqueue endpoint. The job can go from processing to enqueued again, so forward progression isn't strictly expected. The transition from created to queued, however, should be one-way.