Distributed Halide
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Adobe, Cambridge, MA (United States)
Many image processing tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement image processing algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing when and where to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, however, Halide has been restricted to parallel shared memory execution, limiting its performance for memory-bandwidth-bound pipelines or large-scale image processing tasks. We present an extension to Halide to support distributed-memory parallel execution of complex stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, our extensions give up to a 1.4× speedup on a single node over regular multithreaded execution with the same number of cores, by mitigating the effects of non-uniform memory access. The distributed benchmarks achieve up to 18× speedup on a 16 node testing machine and up to 57× speedup on 64 nodes of the NERSC Cori supercomputer.
- Research Organization:
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- Grant/Contract Number:
- SC0005288
- OSTI ID:
- 1557579
- Journal Information:
- SIGPLAN, Vol. 51, Issue 8; ISSN 0362-1340
- Publisher:
- ACMCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Web of Science
Supporting Very Large Models using Automatic Dataflow Graph Partitioning
|
conference | January 2019 |
| Supporting Very Large Models using Automatic Dataflow Graph Partitioning | text | January 2018 |
| Loop Tiling in Large-Scale Stencil Codes at Run-time with OPS | text | January 2017 |
Similar Records
PaKman: A Scalable Algorithm for Generating Genomic Contigs on Distributed Memory Machines
Roofline Analysis in the Intel® Advisor to Deliver Optimized Performance for applications on Intel® Xeon Phi™ Processor