SC19 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Multi-Level Memory and Storage for HPC, Data Analytics, and AI

Authors: Hans-Christian Hoppe (Intel Corporation), Michèle Weiland (Edinburgh Parallel Computing Centre), Kathryn Mohror (Lawrence Livermore National Laboratory)

Abstract: This BoF investigates the opportunities arising from the progress in storage class memory (SCM) technology and the parallel rapid emergence of data-intensive application in the HPC context, which increasingly combine simulations with data analytics, AI, or graph analytics techniques. Topics include the benefit of SCM for applications, integration with the system architecture, SW interfaces, and, of course, results from proof of concept projects.

The session brings together technology providers, application and system SW developers, and system operators to engage in a discussion with the audience.

Long Description: The field of storage-class memory (SCM) has seen significant progress in the last six months, including Intel’s non-volatile RAM product, and novel NVMe-based solutions (like the ones from Excelero). At the same time, emerging workloads mixing HPC simulations with big data processing or AI do create a need and perfect opportunity for innovative HPC systems embracing SCM. These will substantially improve delivered performance, scalability and energy efficiency for data-oriented HPC codes as well as “mixed” applications, and can play a large role in removing the “I/O wall” when moving to Exascale class systems.

Memory and storage devices introduce ever-increasing capacities and performance, many of them non-volatile by nature, and systems with multiple memory and/or storage levels are becoming state-of-the-art. This presents a perfect opportunity to address the traditional Exascale “I/O gap” and at the same time provide the emerging breed of combined HPC and AI applications with solid and highly performing memory/storage platforms. Of particular importance are the support for efficient access over high-performance network fabrics, the integration with HPC-class systems, and evaluation of SW interfaces that provide alternatives to Posix I/O.

This BoF continues the series of successful similar sessions at ISC and SC; it bring together technology providers, application and system SW developers, and system operators to discuss use cases and requirements for next-generation multi-level storage/memory systems, present proof of concept prototype results, and examine the system SW and tools development. Special focus will be on non-traditional use cases, covering AI, data analytics and graph processing in an HPC context.

A number of important constituencies will profit from participating at the session: developers of data-intensive HPC or combined HPC and AI applications will learn how system SW and hardware can better support them, and they can start co-design discussions. Developers and architects of system SW (like I/O and RMS) will learn about new use cases and technologies, and can receive feedback to their ideas, and finally technology providers (in particular for storage and memory media, interconnects and accelerators) will receive valuable guidance on how their technology can be used in the HPC, HPDA, AI and graph analytics communities.

Back to Birds of a Feather Archive Listing