AnsweredAssumed Answered

Compression on Bulk Export APIs?

Question asked by Kurt Koller on Jun 22, 2017
Latest reply on Feb 14, 2018 by fa688a5b34d230fd9fa57feca9a5fb87e6ecb967

For Bulk Activity Export - exporting a file it is both stored as plain text and transmitted without gzip compression.


This data is repetitive and compresses very well. In my analysis of files with all activity types, ~90%+ with gzip -6 settings. It's a little better if you're asking for the same activity type.


If we ask for 205M worth of activities, and if the data were stored compressed, it would be 20.5M versus 205M. Even if just gzip were turned on in nginx, at least that would save 90% of transmission time. Ideally this data would be stored compressed to not count toward the quota.


At a minimum can compression be turned on for the file download so that part is quicker? The server currently doesn't seem to send gzipped data even when asked.