Characterization and modeling of PIDX parallel I/O for performance optimization

Update item information
Publication Type pre-print
School or College <blank>
Department Computing, School of
Creator Pascucci, Valerio
Other Author Kumar, Sidharth; Saha, Avishek; Vishwanath, Venkatram; Carns, Philip; Schmidt, John A.; Scorzelli, Giorgio; Kolla, Hemanth; Grout, Ray; Latham, Robert; Ross, Robert; Papka, Michael E.;Chen, Jacqueline
Title Characterization and modeling of PIDX parallel I/O for performance optimization
Date 2013-01-01
Description Parallel I/O library performance can vary greatly in re- sponse to user-tunable parameter values such as aggregator count, file count, and aggregation strategy. Unfortunately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some characteristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tunable parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space exploration and performance prediction.
Type Text
Publisher Institute of Electrical and Electronics Engineers (IEEE)
Issue 67
Language eng
Bibliographic Citation Kumar, S., Saha, A., Vishwanath, V., Carns, P., Schmidt, J. A., Scorzelli, G., Kolla, H., Grout, R., Latham, R., Ross, R., Papka, M. E., Chen, J., & Pascucci, V. (2013). Characterization and modeling of PIDX parallel I/O for performance optimization. International Conference for High Performance Computing, Networking, Storage and Analysis, SC, no. 67
Rights Management (c) 2013 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Format Medium application/pdf
Format Extent 3,122,229 bytes
Identifier uspace,18705
ARK ark:/87278/s6k10dc1
Setname ir_uspace
Date Created 2014-06-11
Date Modified 2021-05-06
ID 712581
Reference URL
Back to Search Results