DFTracer 1.0.0 Released: I/O Tracing for AI-Driven Workflows
· One min read
On June 26, 2024, DFTracer v1.0.0 (opens in a new tab) was released. DFTracer is a data flow tracer for deep learning and AI-driven workflows, developed at Lawrence Livermore National Laboratory by a team led by SCS Lab alumnus Dr. Hariharan Devarajan, with GRC co-authors on its SC'24 paper.
What it does
DFTracer captures trace data across several layers of the software stack, from the application and its data loaders down to POSIX I/O, so that data loading stalls in training pipelines can be attributed to their source. The SC'24 paper reports lower runtime overhead and smaller traces than Score-P, Recorder, and Darshan on the workloads it studies, and tracing of dynamically spawned processes in AI workflows.