Mihai Dobrescu

Large File upload via REST API

Discussion created by Mihai Dobrescu on Jun 24, 2019
Latest reply on Jun 26, 2019 by Jep Castelein

Hello,

 

I'm trying to upload a file to Marketo via REST API(nodejs app) and I've managed to do it for small size files with npm package "request-promise". 

 

const options = {

method : 'POST',
uri: uri,
formData:{
name:filename,
folder:folder,
file: {
value: fs.createReadStream(path, {highWaterMark: 256 * 1024}),
options: {
filename: filename,
contentType: mimeType
}
}
}
};

 

The issue comes when I'm trying to upload a larger file and I get "413 Request Entity Too Large" errors. It looks like the stream of file is too big for Marketo to handle. The file has 11mb so it's not over Marketo's limit.

 

So now I'm trying to use "https" npm module so I can write the request myself but I'm getting a 611 Marketo Error(unhandled error)

 The code for the request looks like this:

let mimeType = mime.lookup(filename);

var file = {
value: fs.createReadStream(pathFile + "/data/process/scanned/" + filename),
options: {
filename: filename,
contentType: mimeType
}
};

var options = {
"method": "POST",
"hostname": clientConfig.host2,
"path": "/rest/asset/v1/files.json?access_token=" + result.access_token,
"headers": {
"content-type": "multipart/form-data; boundary=----WebKitFormBoundary8MA4YWxkTrZu0gW"
}
};


var strFile = '------WebKitFormBoundary8MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name = "file"; filename = "' + filename + '"\r\n' + 'Content-Type:' + mimeType + '\r\n' + JSON.stringify(file);
var strFolder = '------WebKitFormBoundary8MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name = "folder" \r\n' + JSON.stringify(folder);
str2 = strFile + "\r\n" + strFolder + "\r\n" + "------WebKitFormBoundary8MA4YWxkTrZu0gW-";

req.write(str2);
req.end();

 

On runtime i get this request:

 

------WebKitFormBoundary8MA4YWxkTrZu0gW
Content-Disposition: form-data; name = "file"; filename = "fi.zip"
Content-Type:application/zip
{"value":{"_readableState":{"objectMode":false,"highWaterMark":65536,"buffer":{"head":null,"tail":null,"length":0},"length":0,"pipes":null,"pipesCount":0,"flowing":null,"ended":false,"endEmitted":false,"reading":false,"sync":true,"needReadable":false,"emittedReadable":false,"readableListening":false,"resumeScheduled":false,"emitClose":false,"destroyed":false,"defaultEncoding":"utf8","awaitDrain":0,"readingMore":false,"decoder":null,"encoding":null},"readable":true,"_events":{},"_eventsCount":1,"path":"C:\\LEADFABRIC\\kanthal\\fileUpload/data/process/scanned/fi.zip","fd":null,"flags":"r","mode":438,"end":null,"autoClose":true,"bytesRead":0,"closed":false},"options":{"filename":"fi.zip","contentType":"application/zip"}}
------WebKitFormBoundary8MA4YWxkTrZu0gW
Content-Disposition: form-data; name = "folder"
{"id":"1400","type":"folder"}
------WebKitFormBoundary8MA4YWxkTrZu0gW-

 

 

I've tried to make it so it's similar to Marketo documentation on file upload(https://developers.marketo.com/rest-api/assets/files/#create_and_update ) but can't seem to make it work.

 

Any ideas how to solve this issue?

 

Thank you

Outcomes