Parallel and GPU-based compression of 3D floating-point scientific data
Loading...
Date
2025-11-21
Authors
Σκόνδρας, Γεώργιος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
We live in an era where digital systems generate, store, and transmit enormous amounts of data at any given moment. High-performance applications, such as simulations, have now scaled to unprecedented levels, both in size and resolution. Their performance depends largely on the efficiency of storing and transferring large quantities of data. The application of data compression techniques can be vital in addressing these challenges, effectively reducing data size and bandwidth requirements. Equally important is the use of parallel computing, leveraging widely available parallel architectures, such as multi-core CPUs and GPUs, to accelerate compression and input/output (I/O) operations, enabling their effective integration into scientific computing pipelines. This thesis explores both the field of data compression and parallel processing, focusing on integrating GPU acceleration into an existing framework for scientific data compression and I/O that handles floating-point data. The work focuses on two major technologies: the ZFP library, which is widely used for both lossy and lossless compression of numerical data, and the CubismZ framework, a high-performance software platform for parallel compression and I/O for 3-dimensional floating-point scientific data. In the C++ implemented CubismZ software, the original dataset is divided into a 3-dimensional grid of blocks. Parts of this grid can be assigned to different MPI processes, with compression being achieved on the Central Processing Unit (CPU) using OpenMP threads. Data compression is performed in two stages, one stage using lossy floating-point compression algorithms (e.g. ZFP, SZ, FPZIP) and an optional stage using general-purpose lossless compression algorithms (e.g., Zlib, LZ4) for binary data. The GPUbismZ extension developed as part of this thesis retains all the functionality of the existing infrastructure, adding the ability to utilize multiple GPUs for efficient compression. An experimental evaluation of both the ZFP library and the GPUbismZ extension implemented in this work is performed, with different parameters and under various configurations, in order to determine the performance, scaling, and limitations of the proposed approach on real scientific data.
Description
Keywords
Data compression, Floating-point data compression, Parallel programming, GPUs