For Bulk Activity Export - exporting a file it is both stored as plain text and transmitted without gzip compression.
This data is repetitive and compresses very well. In my analysis of files with all activity types, ~90%+ with gzip -6 settings. It's a little better if you're asking for the same activity type.
If we ask for 205M worth of activities, and if the data were stored compressed, it would be 20.5M versus 205M. Even if just gzip were turned on in nginx, at least that would save 90% of transmission time. Ideally this data would be stored compressed to not count toward the quota.
At a minimum can compression be turned on for the file download so that part is quicker? The server currently doesn't seem to send gzipped data even when asked.
Encoding means the text in the payload is compressed and automatically decompressed by your client. It's not a gzip file (otherwise requesting gzip for JS files, like browsers do, could never work since ungzip isn't a native JS capability).
No, HTTP compression doesn't need to have any relationship to the size of the data exported from a database. They're 2 different areas.
(Though if the data in the db were pre-gzipped, it wouldn't do any good -- only bad, due to resource overhead -- to compress it again before putting it on the wire.)
requesting to compress the request only results in a faster transfer, no other benefits...
Right, that would be the only guarantee (and while it is a definite improvement, downloading uncompressed 500MB doesn't take long anyway, considering you can only do it once per day).
Having the quota apply after compressing data may or may not happen in the future, I guess. Depends on what the 500 MB daily limit was really meant to address, e.g. saving Marketo's external network utilization or (implicitly) saving database server/client resources or both.
HTTP compression (gzip) is supported by bulk extract leads and activities. When calling either Get Export Lead File, or Get Export Activity File endpoints, simply add the HTTP header "Accept-Ecoding: gzip" and the response body will be gzipped.
I tried adding HTTP header "Accept-Ecoding: gzip" but in response header we are not receiving Content-Encoding: gzip and response is coming in plain text format.
.png)
Adchoices