LABIOS

LABIOS represents I/O as a Label that carries an operation, a data pointer, and routing metadata. The runtime can route, schedule, transform, and execute that work independently of the storage backend. The HPDC'19 paper reports up to 17 times higher I/O performance than synchronous I/O and up to 6 times higher I/O performance when it manages burst buffers and storage servers together. It received the Karsten Schwan Best Paper Award and later U.S. Patent 11,630,834 B2.

The public LABIOS 2.0 prototype (opens in a new tab) applies the abstraction to I/O from applications, AI frameworks, and agents. Its open-source implementation retains the label pipeline while adding an agent-facing interface.

LABIOS 2.0: Agent I/O Runtime​

• Clients create labels through C++, C, Python, or LD_PRELOAD interception.
• The dispatcher aggregates writes, tracks dependencies, forms supertasks, schedules work, and runs completion actions.
• Databot workers execute individual I/O operations. Pipeline workers execute storage-operation graphs.
• Storage adapters currently cover files, SQLite, and an optional key-value backend.

NATS JetStream carries labels, while DragonflyDB stages data. Docker Compose deploys the prototype, and a packaged example checks the complete source-to-pipeline-to-destination path. Agentic workers and S3, vector, and graph backends remain planned. Workspaces are currently process-local.

Research Contributions​

The Label Abstraction (HPDC'19, Karsten Schwan Best Paper Award)​

The Label decouples an I/O operation from its storage backend by carrying the operation, data pointer, and metadata about origin, destination, and state. The HPDC'19 paper reports:

• Asynchronous I/O with scheduling. Applications issue labels and continue computation without blocking. The paper reports up to 17 times higher I/O performance than synchronous I/O.
• Storage elasticity. LABIOS provisions and reclaims resources across DRAM, NVMe, SSD, HDD, and parallel storage without application changes.
• Computational storage. Operations embedded in labels can process data in transit instead of using a separate read, process, and write cycle.
• Storage bridging. Labels map to POSIX file semantics and key-value or object semantics, allowing HPC and Big Data workloads to share one storage system.

Read the paper.

Storage Semantics via Data Labels (ACM TOS'20)​

The journal extension expanded the LABIOS design with computational storage and software-defined storage capabilities. The paper shows how the label abstraction supports hierarchical storage compositions where new hardware (NVMe, persistent memory) can be integrated without modifying application code. Labels carry enough semantic information for data placement, tiered caching, and in-situ analytics across the deep memory and storage hierarchy.

Read the paper.

MegaMmap: Memory-Storage Convergence (SC'24)​

MegaMmap applies the same separation of semantics from hardware placement to out-of-core memory. It manages DRAM and storage as a tiered software distributed shared memory. The SC'24 paper reports up to 2.6x lower peak DRAM use at competitive performance.

Read the paper.

Performance Results​

Results reported in the HPDC'19 paper:

Storage bridging: Hadoop K-means on LABIOS compared with native Hadoop, on disk and in memory (HPDC'19).

Resource heterogeneity: HACC checkpointing through burst buffers and storage servers, with up to 6x higher I/O performance (HPDC'19).

Extending the Label Model​

Current work applies labels to three data paths. MegaMmap manages out-of-core data across DRAM and storage. HStream separates data and compute planes for hierarchical streams. DeepIO studies model weights and KV caches across GPU memory, host memory, and storage.

Current Results​

• GPU memory. MegaMmap combines GPU-resident model loading with lossless compression in llama.cpp. The project's 2025 to 2026 annual report records a 15 GB coding model mapped into 12 GB of GPU memory, with about 5 tokens/s versus 2.4 tokens/s for the out-of-core baseline.
• Dataflow characterization. Peer-reviewed work characterized six HPC workflows totaling 3,500 task instances, 1,387 unique files, and 1.5 TB of I/O at IPDPS 2026, and three AWS S3 integration paths at PDSW'25.
• Agent-facing storage. Acropolis and AgentStage were accepted at eScience'26. See all LABIOS publications.

LABIOS is protected under U.S. Patent 11,630,834 B2.