Skip to main content

MegaMmap at SC24: tiered distributed shared memory beyond DRAM

Storage · · 6 min read

Luke Logan presented MegaMmap at SC24, the International Conference for High Performance Computing, Networking, Storage, and Analysis, held in Atlanta from November 17 to 22, 2024. The paper, by Logan, Anthony Kougkas and Xian-He Sun, describes a software distributed shared memory that extends memory into storage, so that applications can work on datasets larger than the DRAM of their nodes without writing out-of-core code.

Luke Logan at the SC24 lectern presenting the MegaMmap paper
Luke Logan presenting MegaMmap at SC24 in Atlanta.

Datasets larger than memory​

Scientific simulations and data analysis increasingly work on datasets that do not fit in the DRAM of the nodes they run on. Buying more DRAM is expensive, and the usual workaround, out-of-core programming, makes every application partition its data, schedule its own I/O and synchronize the pieces. When a run outgrows memory anyway, Linux terminates it, and the science stops at the size of main memory.

MegaMmap takes the other route: it enlarges the effective memory capacity of a cluster by tiering DRAM with the NVMe, SSD and HDD storage that nodes already have, and manages the data movement itself.

How MegaMmap works​

MegaMmap presents a shared, byte-addressable address space that spans every process and every storage tier. Each process has a private DRAM cache for low-latency access. Behind it, a shared cache spreads data across all processes and all tiers, and a data organizer decides which tier each page belongs on based on how it is used and how fast each device is.

MegaMmap architecture: per-process private caches over a shared, tiered cache spanning DRAM, NVMe, SSD and HDD
MegaMmap architecture. Figure from Logan, Kougkas and Sun, SC24.

Traditional distributed shared memory systems have to guess what an application will do next. MegaMmap instead lets the application say so. Code accesses data inside transactions that declare a range and an intent, such as sequential read-only access. The declared pattern drives the prefetcher, which stages data before it is needed, and lets MegaMmap pick a lighter coherence protocol for common HPC patterns such as read-only analytics or write-only simulation. Operations on the same page are ordered, so a read always sees the preceding write. Data movement runs asynchronously, overlapping with computation, and datasets can be staged to and from persistent files.

MegaMmap is built on Hermes, GRC's hierarchical buffering platform, which supplies the data movement and metadata infrastructure. For MegaMmap the team extended Hermes with low-latency NVMe access through SPDK, memory-mapped access for CXL memory devices, and data staging for HDF5 and Parquet.

Programming with MegaMmap​

The interface is a C++ vector. A program opens a dataset as a shared vector, bounds the DRAM it may use, partitions it across processes and iterates over it inside a transaction. This KMeans fragment, from the paper, reads a Parquet file with a 1 MB DRAM budget:

// Create a shared vector from a Parquet file
mm::Vector<Point3D> pts("/points.parquet");
pts.BoundMemory(MEGABYTES(1)); // Limit to 1MB DRAM
pts.Pgas(rank, nprocs); // Partition across processes

// Begin read-only transaction
auto tx = pts.SeqTxBegin(pts.local_off(), pts.local_size(), MM_READ_ONLY);

float distance = 0;
for (Point3D p : tx) {
distance += pow(NearestCentroid(p, ks), 2);
}

Only the data structures that are too large for memory need to change, along with the loops that access them. In the paper's applications, the MegaMmap versions were 45% to 2 times smaller than their out-of-core equivalents, because all explicit I/O code goes away.

Lines of code for MegaMmap versus out-of-core implementations of the evaluated applications
Code size comparison. Figure from Logan, Kougkas and Sun, SC24.

Results​

The evaluation ran on a 32-node research cluster in which each node has 48 GB of DRAM, a 128 GB NVMe drive, a 256 GB SSD and a 1 TB HDD. The workloads were KMeans and DBSCAN clustering on cosmology data produced by the Gadget-4 simulation code, a random forest classifier, and the Gray-Scott reaction-diffusion simulation.

  • Less DRAM. With 1,536 processes on 32 nodes, KMeans ran with 2.6 times less DRAM, and DBSCAN and random forest with 2 times less, while staying within 10% of their performance with full DRAM.
  • Against MPI and Spark. In scaling runs up to 768 processes, MegaMmap performed close to the MPI implementations and up to 2 times faster than Apache Spark 3.4.1, which also used 3 to 4 times as much DRAM.
  • Past the memory limit. The MPI version of Gray-Scott crashes once the grid grows beyond L = 2688. MegaMmap ran it at L = 3456, a 1.5 TB grid with over twice the data, by spilling to node-local NVMe. Up to L = 2688 it was at least 20% faster than the other tiered I/O versions, because it places data during the first compute phase instead of waiting for it to finish.
  • Choosing tiers. In a 768-process Gray-Scott run that wrote 8 TB, adding NVMe to the hierarchy improved performance by up to 1.8 times, and the study shows how performance tracks the cost of each tier mix.
Scalability results for MegaMmap and the compared systems
Scalability results. Figure from Logan, Kougkas and Sun, SC24.
Cost-performance results for MegaMmap across storage tier configurations
Cost-performance results. Figure from Logan, Kougkas and Sun, SC24.

Software and support​

MegaMmap is open source on GitHub (opens in a new tab). The work was supported by the National Science Foundation under grants CSSI-2104013 and Core-2313154 and by the U.S. Department of Energy Office of Science under contract DE-SC0024593. Development used the Chameleon Cloud (opens in a new tab) testbed.

Learn more​