Heterogeneity-Aware Collective I/O for Parallel I/O Systems with Hybrid HDD/SSD Servers
Authors: S. He, Y. Wang, X.-H. Sun, C. Huang, C. Xu
Date: June, 2017
Venue: IEEE Transactions on Computers (TC), vol. 66, no. 6, pp. 1091-1098
Type: Journal
Abstract
Collective I/O is a widely used middleware technique that exploits I/O access correlation among multiple processes to improve I/O system performance. However, most existing implementations of collective I/O strategies are designed and optimized for homogeneous I/O systems. In practice, the homogeneity assumptions do not hold in heterogeneous parallel I/O systems, which consist of multiple HDD and SSD-based servers and become increasingly promising. In this paper, we propose a heterogeneity-aware collective-I/O (HACIO) strategy to enhance the performance of conventional collective I/O operations. HACIO reorganizes the order of I/O requests for each aggregator with awareness of the storage performance of heterogeneous servers, so that the hardware of the systems can be better utilized. We have implemented HACIO in ROMIO, a widely used MPI-IO library. Experimental results show that HACIO can significantly increase the I/O throughputs of heterogeneous I/O systems.