Significantly Improving Lossy Compression Quality Based on an Optimized Hybrid Prediction Model

Authors: Xin Liang (University of California, Riverside), Sheng Di (Argonne National Laboratory), Sihuan Li (University of California, Riverside), Dingwen Tao (University of Alabama), Bogdan Nicolae (Argonne National Laboratory), Zizhong Chen (University of California, Riverside), Franck Cappello (Argonne National Laboratory)

Abstract: With ever-increasing volumes of data produced by large-scale scientific simulations, error-bounded lossy compression has come into a critical place. In this paper, we design a strategy to improve the compression quality significantly based on an optimized, hybrid prediction framework. The contribution is four-fold. (1) We propose a novel, transform-based predictor and optimize its compression quality. (2) We improve the coefficient-encoding efficiency for the data-fitting predictor. (3) We propose an adaptive framework that can select the bestfit predictor accurately for different datasets. (4) We perform the evaluation by running real-world applications on a supercomputer with 8192 cores. Experiments show that our adaptive compressor can improve the compression ratio by 112~165% over the second-best state-of-the-art lossy compressor. The parallel I/O performance is improved by about 100% because of significantly reduced data size. The total I/O time is reduced by up to 60x with our compressor compared with the original I/O time.


Presentation: file


Back to Technical Papers Archive Listing