Efficient data restructuring and aggregation for I/O acceleration in PIDX

Update item information
Publication Type pre-print
School or College Scientific Computing and Imaging Institute
Department Computing, School of
Creator Pascucci, Valerio
Other Author Kumar, Sidharth; Vishwanath, Venkatram; Carns, Philip; Levine, Joshua A.; Latham, Robert; Scorzelli, Giorgio; Kolla, Hemanth; Grout, Ray; Ross, Robert; Papka, Michael E.; Chen, Jacqueline
Title Efficient data restructuring and aggregation for I/O acceleration in PIDX
Date 2012-01-01
Description Hierarchical, multiresolution data representations enable interactive analysis and visualization of large-scale simulations. One promising application of these techniques is to store high performance computing simulation output in a hierarchical Z (HZ) ordering that translates data from a Cartesian coordinate scheme to a one-dimensional array ordered by locality at different resolution levels. However, when the dimensions of the simulation data are not an even power of 2, parallel HZ ordering produces sparse memory and network access patterns that inhibit I/O performance. This work presents a new technique for parallel HZ ordering of simulation datasets that restructures simulation data into large (power of 2) blocks to facilitate efficient I/O aggregation. We perform both weak and strong scaling experiments using the S3D combustion application on both Cray-XE6 (65,536 cores) and IBM Blue Gene/P (131,072 cores) platforms. We demonstrate that data can be written in hierarchical, multiresolution format with performance competitive to that of native data-ordering methods.
Type Text
Publisher Institute of Electrical and Electronics Engineers (IEEE)
Language eng
Bibliographic Citation Kumar, S., Vishwanath, V., Carns, P., Levine, J. A., Latham, R., Scorzelli, G., Kolla, H., Grout, R., Ross, R., Papka, M. E., Chen, J., & Pascucci, V. (2012). Efficient data restructuring and aggregation for I/O acceleration in PIDX. International Conference for High Performance Computing, Networking, no. 6468529.
Rights Management (c) 2012 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Format Medium application/pdf
Format Extent 1,439,976 bytes
Identifier uspace,18291
ARK ark:/87278/s65t449j
Setname ir_uspace
Date Created 2013-05-31
Date Modified 2021-05-06
ID 708832
Reference URL https://collections.lib.utah.edu/ark:/87278/s65t449j
Back to Search Results