SC19 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Inter-Job Scheduling for High-Throughput Material Screening Applications

Workshop: Inter-Job Scheduling for High-Throughput Material Screening Applications

Abstract: Material screening entails a large number of electronic structure simulations that solve the Kohn-Sham (KS) equations based on the high-throughput density functional theory. The computational efficiency for solving the KS equations, however, is largely dependent on the initial displacement of the particles. Traditionally, simulation runs are treated separately as solving independent KS equations with their own initial conditions. As such, one cannot take advantage of the potential correlation between different simulations. In this paper, we propose to explore the job inter-dependencies and formulate material screening as an inter-job scheduling problem. More specifically, We schedule the material screening jobs according to the potential correlation between them and run the simulations by reusing the particle distribution from previous runs as the initial condition. We propose a heuristic inter-job scheduling algorithm and compare that with an optimal case where the pair-wise correlation between different simulations is known a priori. Experiments on the Sunway TaihuLight supercomputer show that the total time to run the large number of jobs in one practical material screening example is significantly reduced, by as much as 89%.

Back to The 14th Workshop on Workflows in Support of Large-Scale Science (WORKS19) Archive Listing

Back to Full Workshop Archive Listing