Skip to main content

NPC 2024 keynote: Hermes, Coeus and ChronoLog

Talks & Insights · · 3 min read

Xian-He Sun gave the first keynote of the IFIP International Conference on Network and Parallel Computing (NPC 2024) in Haikou, China, on December 7, 2024. Titled "Hermes, Coeus, and ChronoLog: Some big data I/O systems for network, parallel and distributed computing", the talk walked through three GRC systems that each attack a different part of the data access bottleneck.

The data bottleneck in AI​

Sun started from deep learning. Its success depends on processing vast amounts of data, and computing systems are struggling to supply data fast enough, which turns I/O into the bottleneck. He argued that data access has become the main performance limit for network, parallel and distributed computing (NPC) when it serves AI, and that AI can in turn help relieve that limit. The talk looked at both directions, NPC for AI and AI for NPC, from the point of view of the I/O system.

Three systems​

Hermes is a multi-tiered I/O buffering system. It optimizes how data moves through a deep memory and storage hierarchy, from memory through local flash and burst buffers to the parallel file system. The abstract describes it as serving both directions, NPC for AI and AI for NPC.

Coeus enriches the metadata that scientific data systems keep, and uses active storage, storage that can run computation, to calculate derived quantities and answer complex queries over scientific data. The goal is to let scientists ask questions of their data without first moving all of it to compute nodes. Coeus is a collaborative project funded by the DOE Office of Advanced Scientific Computing Research and led by Sandia National Laboratories, with GRC as a partner.

ChronoLog is a distributed, tiered log store for log data in cloud computing environments. It orders events by time across many nodes, so that logs from large distributed systems can be written and read at scale. ChronoLog is funded by an NSF CSSI framework grant, and its first public beta, version 1.0.0, was released in June 2024.

Sun closed with the open questions these systems raise and with ideas for computer system designs built for AI and big data. The talk extended the keynote he gave at IEEE/ACM CCGrid 2024 in May, which also covered Hermes, ChronoLog and Coeus.

Learn more​