Software developers and service operators need log files to identify issues, detect anomalies and trace execution behaviour. The amount of generated log data is increasing, and often log files need to be kept for longer periods of time due to regulations. To preserve logs in a cost-efficient manner, they are typically compressed, at certain cost for running the compression, and then stored in long-term archives, again at certain cost per size-duration products. The goal is decrease both cost components, but there are certain trade-offs, for instance a highly efficient compression that consumes a lot of CPU but leads to better compression ratios, consuming less storage capacity as a result. The decision which compression tools and parameters to use is usually hardcoded. We present a smart knowledge-based advisor service to query goal-based adaptive compression commands to maximise savings.

Continue reading