SC19 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Containers in HPC

Authors: Andrew Younge (Sandia National Laboratories), Shane Canon (Lawrence Berkeley National Laboratory), CJ Newburn (Nvidia Corporation)

Abstract: Container computing has revolutionized how many industries and enterprises develop and deploy software and services. Recently, this model has gained traction in the HPC community through enabling technologies like Charliecloud, Docker, Kubernetes, Podman, Shifter, and Singularity. While containers look to provide greater software flexibility, reliability, ease of deployment, and portability for users, there are still open questions that need to be addressed as we race toward adoption in the Exascale era. In this BoF, we will provide an opportunity for the HPC community to engage with the leaders in the field who can provide real-world experiences to containerization in HPC.

Long Description: Container computing has revolutionized the way groups are developing, sharing, and running software and services. This has initially been led by growth with Docker, which has provided an ecosystem of tools to enable container based computing. This paradigm shift has since made in-roads in the HPC community enabled by container runtimes like Podman, Singularity, Shifter, and Charliecloud, which allow end-users to run containers in environments where standard Docker tools would not be feasible. Orchestration systems such as Kubernetes also allow for quick, scalable service deployments in conjunction with HPC. While this adoption is growing, the larger HPC community still has many questions around this new model. Furthermore, there still exist several undefined research questions related to containerization in HPC that the community must address for containers to be a successful technology for HPC applications.

This BOF will bring together key experts in the field of containerization and HPC to describe and share their experiences, tools, and knowledge and engage in interactive discussions with the audience on this topic. Given the recent increase in interest surrounding containers and large turnout the last two years, we expect this BOF to be a very popular topic at SC18 with particular applicability towards the greater HPC community. The goals of this BOF are to: introduce the community to the various solutions, share experiences and some cutting-edge efforts underway, promote the use of containers in HPC, and build a community to help promote this new model and facilitate further adoption.

First, we will initiate the session by introducing several prominent members in the field who will give short lightning talks focusing on the state-of-the-art use cases and experiences of deploying containers in various HPC settings. Next, these members will combine with leaders from major HPC container solutions including Docker, Charliecloud, Podman, Shifter, and Singularity to form a full panel. This panel will be steered by a moderator who will drive an interactive discussion on containers. We envision the bulk of the BOF session to be devoted to lively, constructive, and useful discussion between the panel and audience. The moderator will initially seed discussion topics, but then transition to use online and show-of-hand survey questions and other tools to encourage the audience to participate in the discussion.

Specifically, some key questions the panel will discuss include: What are some best practices that users and facilities can invoke to ensure consistent usage and performance portability with HPC containers? What are the key security considerations when using containers in production HPC facilities? Should HPC facilities embrase or support service-based container computing (eg: Kubernetes, OpenShift, etc)? What challenges exist to integrating container services with traditional batch mechanisms? What are some of the biggest changes or advances still to come with containers at Exascale?

In addition to building a network of new proponents for containers in HPC, other outcomes from this BOF will include responses to survey questions to help guide the research community and identifying mechanisms for community engagement, such as an email list or web-based collaboration sites.

Back to Birds of a Feather Archive Listing