Presentation
Poster 38: High-Performance Backpropagation in Scientific Computing
Advisors
Event Type
Doctoral Showcase
Posters
TP
Student Program
TimeThursday, 21 November 20192:30pm - 2:50pm
Location210-212
DescriptionDevito is a domain-specific language for the automatic generation of high-performance solvers for finite difference equations provided in a high-level symbolic representation. The primary use of Devito is to enable Full-waveform inversion, which is an adjoint-based optimization problem that uses the wave equation as part of its objective function. This doctoral project consists of three elements of Devito that are key to solving adjoint-based optimization problems (i.e. the back propagation). The first is automatic differentiation of stencil loops. This allows the automatic generation of high-performance code implementing the derivative of any provided function. This is essential to enable the use of a wider range of physical equations - in order to use better physical approximations. An essential feature of the generated derivatives is that they show scaling and performance behaviour that is very similar to the forward function. A common issue in solving adjoint-based optimization problems is the prohibitively high memory requirement. The second contribution is a runtime for automatic execution of checkpoint-recompute schedules (called pyRevolve) to alleviate this memory requirement by trading it off for re-computations. These schedules may involve storing some intermediate states on disk, some in memory, and recomputing others. The third contribution is the use of lossy compression (ZFP/SZ), which is a second means of trading off memory and compute, automatically as part of checkpointing in pyRevolve. This is the first time it has been shown that lossy compression combined with checkpoint-recomputation can provide a much better error-memory tradeoff than using checkpoint-recomputation or lossy compression alone.
Archive