pMEMCPY: a simple, lightweight, and portable I/O library for storing data in persistent memory
Authors: L. Logan, J. Lofstead, S. Levy, P. Widener, X.-H. Sun, A. Kougkas
Date: September, 2021
Venue: The 1st Workshop on Re-envisioning Extreme-Scale I/O for Emerging Hybrid HPC Workloads (REX-IO'21), in conjunction with the 2021 IEEE International Conference on Cluster Computing (CLUSTER'21), September 7-10, 2021
Type: Workshop
Abstract
Persistent memory (PMEM) devices can achieve comparable performance to DRAM while providing significantly more capacity. This has made the technology compelling as an expansion to main memory. Rethinking PMEM as storage devices can offer a high performance buffering layer for HPC applications to temporarily, but safely store data. However, modern parallel I/O libraries, such as HDF5 and pNetCDF, are complicated and introduce significant software and metadata overheads when persisting data to these storage devices, wasting much of their potential. In this work, we explore the potential of PMEM as storage through pMEMCPY: a simple, lightweight, and portable I/O library for storing data in persistent memory. We demonstrate that our approach is up to 2x faster than other popular parallel I/O libraries under real workloads.