Skip to main content

WorkshopNovember, 2025Published

Quantifying AWS S3 I/O Performance Boundaries Using the Roofline Model

Authors
M. Tang, Z. Zhu, L. Guo, J. G. Bandy, T. Carlson, S. Neuwirth, A. Kougkas, X.-H. Sun, N. R. Tallent
Venue
10th International Parallel Data Systems Workshop (PDSW 2025), Proceedings of the SC '25 Workshops
Date
November, 2025
Type
Workshop

Abstract

High-performance computing facilities increasingly form hybrid environments that integrate cloud services. To avoid cumbersome network transfers when sharing data, a new class of storage gateways map a subset of facility storage to a cloud counterpart and automatically manage data mirroring. However, the performance characteristics of accessing AWS's S3 from HPC systems using different methods and patterns remains poorly understood. This paper presents a roofline-based analysis of three S3 integration approaches: NFS-mounted AWS Storage Gateway, data migration through Storage Gateway, and direct S3 API transfers. We extend I/O roofline modeling to characterize operational intensity and bandwidth ceilings across varying data sizes and access patterns. Our experimental evaluation demonstrates significant performance differences between access methods, with POSIX I/O on NFS Storage Gateway achieving up to 6.4× higher bandwidth than other approaches for large transfers. The roofline analysis reveals distinct characteristics for each method, enabling informed selection of S3 integration strategies.