Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

Verified lifting of stencil computations

Journal Article · · ACM SIGPLAN Notices
 [1];  [2];  [3];  [3]
  1. Adobe, Cambridge, MA (United States); Office of Scientific and Technical Information (OSTI)
  2. Univ. of Washington, Seattle, WA (United States)
  3. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

This paper demonstrates a novel combination of program synthesis and verification to lift stencil computations from low-level Fortran code to a high-level summary expressed using a predicate language. The technique is sound and mostly automated, and leverages counter-example guided inductive synthesis (CEGIS) to find provably correct translations. Lifting existing code to a high-performance description language has a number of benefits, including maintainability and performance portability. For example, our experiments show that the lifted summaries can enable domain specific compilers to do a better job of parallelization as compared to an off-the-shelf compiler working on the original code, and can even support fully automatic migration to hardware accelerators such as GPUs. We have implemented verified lifting in a system called STNG and have evaluated it using microbenchmarks, mini-apps, and real-world applications. Here, we demonstrate the benefits of verified lifting by first automatically summarizing Fortran source code into a high-level predicate language, and subsequently translating the lifted summaries into Halide, with the translated code achieving median performance speedups of 4.1X and up to 24X for non-trivial stencils as compared to the original implementation.

Research Organization:
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
Grant/Contract Number:
SC0005288
OSTI ID:
1557573
Journal Information:
ACM SIGPLAN Notices, Journal Name: ACM SIGPLAN Notices Journal Issue: 6 Vol. 51; ISSN 0362-1340
Publisher:
ACMCopyright Statement
Country of Publication:
United States
Language:
English

References (80)

What’s Decidable About Arrays? book January 2005
Verification Condition Generation Via Theorem Proving book January 2006
ROSE: An Optimizing Transformation System for C++ Array-Class Libraries book January 1998
An Axiomatic Basis for Computer Programming book January 1978
ICE: A Robust Framework for Learning Invariants book January 2014
Adaptive Concretization for Parallel Program Synthesis book January 2015
Property-Directed Inference of Universal Invariants or Proving Their Absence book January 2015
Z3: An Efficient SMT Solver book January 2008
Anti-unification Algorithms and Their Applications in Program Analysis book January 2010
SAT-Based Model Checking without Unrolling book January 2011
Software Model Checking via IC3 book January 2012
Three-dimensional treatment of convective flow in the earth's mantle journal June 1985
Dataflow analysis of array and scalar references journal February 1991
Achieving Scalable Locality with Time Skewing journal January 2002
Parallel Finite Volume Method Simulation of Three-Dimensional Fluid Flow and Convective Heat Transfer for Viscoplastic Non-Newtonian Fluids journal July 2014
Dynamically discovering likely program invariants to support program evolution journal January 2001
Syntax-guided synthesis conference October 2013
An auto-tuning framework for parallel multicore stencil computations conference April 2010
PATUS: A Code Generation and Autotuning Framework for Parallel Iterative Stencil Computations on Modern Microarchitectures
  • Christen, Matthias; Schenk, Olaf; Burkhart, Helmar
  • Distributed Processing Symposium (IPDPS), 2011 IEEE International Parallel & Distributed Processing Symposium https://doi.org/10.1109/IPDPS.2011.70
conference May 2011
Tiling Optimizations for 3D Scientific Computations conference January 2000
Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures conference November 2008
Using hammock graphs to structure programs journal April 2004
The Relation Between Diamond Tiling and Hexagonal Tiling journal September 2014
Cache oblivious stencil computations conference January 2005
Combinatorial sketching for finite programs
  • Solar-Lezama, Armando; Tancau, Liviu; Bodik, Rastislav
  • Proceedings of the 12th international conference on Architectural support for programming languages and operating systems - ASPLOS-XII https://doi.org/10.1145/1168857.1168907
conference January 2006
Combinatorial sketching for finite programs journal October 2006
Combinatorial sketching for finite programs journal November 2006
Combinatorial sketching for finite programs journal October 2006
Sketching stencils
  • Solar-Lezama, Armando; Arnold, Gilad; Tancau, Liviu
  • Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation - PLDI '07 https://doi.org/10.1145/1250734.1250754
conference January 2007
The NAS parallel benchmarks---summary and preliminary results conference January 1991
Sketching stencils journal June 2007
Interprocedural query extraction for transparent persistence
  • Wiedermann, Ben; Ibrahim, Ali; Cook, William R.
  • Proceedings of the 23rd ACM SIGPLAN conference on Object oriented programming systems languages and applications - OOPSLA '08 https://doi.org/10.1145/1449764.1449767
conference January 2008
Interprocedural query extraction for transparent persistence journal October 2008
HadoopToSQL: a mapReduce query optimizer conference January 2010
Open source computer algebra systems: GAP journal June 2010
The pochoir stencil compiler conference January 2011
Open source computer algebra systems: SymPy journal January 2012
From relational verification to SIMD loop synthesis
  • Barthe, Gilles; Crespo, Juan Manuel; Kunz, Cesar
  • Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming - PPoPP '13 https://doi.org/10.1145/2442516.2442529
conference January 2013
Stochastic superoptimization
  • Schkufza, Eric; Sharma, Rahul; Aiken, Alex
  • Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems - ASPLOS '13 https://doi.org/10.1145/2451116.2451150
conference January 2013
Stochastic superoptimization journal March 2013
TRANSIT: specifying protocols with concolic snippets
  • Udupa, Abhishek; Raghavan, Arun; Deshmukh, Jyotirmoy V.
  • Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation - PLDI '13 https://doi.org/10.1145/2491956.2462174
conference January 2013
Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines
  • Ragan-Kelley, Jonathan; Barnes, Connelly; Adams, Andrew
  • Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation - PLDI '13 https://doi.org/10.1145/2491956.2462176
conference January 2013
Optimizing database-backed applications with query synthesis
  • Cheung, Alvin; Solar-Lezama, Armando; Madden, Samuel
  • Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation - PLDI '13 https://doi.org/10.1145/2491956.2462180
conference January 2013
When polyhedral transformations meet SIMD code generation conference January 2013
Stochastic superoptimization journal April 2013
Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines journal June 2013
Optimizing database-backed applications with query synthesis journal June 2013
When polyhedral transformations meet SIMD code generation journal June 2013
From relational verification to SIMD loop synthesis journal August 2013
Hybrid Hexagonal/Classical Tiling for GPUs
  • Grosser, Tobias; Cohen, Albert; Holewinski, Justin
  • CGO '14: 12th Annual IEEE/ACM International Symposium on Code Generation and Optimization, Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization https://doi.org/10.1145/2544137.2544160
conference October 2018
Hybrid Hexagonal/Classical Tiling for GPUs conference January 2014
Delite: A Compiler Architecture for Performance-Oriented Embedded Domain-Specific Languages journal April 2014
Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology conference January 2014
Stochastic optimization of floating-point programs with tunable precision conference January 2013
Chlorophyll: synthesis-aided compiler for low-power spatial architectures
  • Phothilimthana, Phitchaya Mangpo; Jelvis, Tikhon; Shah, Rohin
  • PLDI '14: ACM SIGPLAN Conference on Programming Language Design and Implementation, Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation https://doi.org/10.1145/2594291.2594339
conference June 2014
OpenTuner: an extensible framework for program autotuning
  • Ansel, Jason; Kamil, Shoaib; Veeramachaneni, Kalyan
  • Proceedings of the 23rd international conference on Parallel architectures and compilation - PACT '14 https://doi.org/10.1145/2628071.2628092
conference January 2014
Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology conference January 1997
Stochastic optimization of floating-point programs with tunable precision journal June 2014
Chlorophyll: synthesis-aided compiler for low-power spatial architectures journal June 2014
PolyMage: Automatic Optimization for Image Processing Pipelines
  • Mullapudi, Ravi Teja; Vasista, Vinay; Bondhugula, Uday
  • Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS '15 https://doi.org/10.1145/2694344.2694364
conference January 2015
Helium: lifting high-performance stencil kernels from stripped x86 binaries to halide DSL code
  • Mendis, Charith; Bosboom, Jeffrey; Wu, Kevin
  • PLDI '15: ACM SIGPLAN Conference on Programming Language Design and Implementation, Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation https://doi.org/10.1145/2737924.2737974
conference June 2015
PolyMage: Automatic Optimization for Image Processing Pipelines journal May 2015
PolyMage: Automatic Optimization for Image Processing Pipelines journal March 2015
Helium: lifting high-performance stencil kernels from stripped x86 binaries to halide DSL code journal June 2015
Property-Directed Inference of Universal Invariants or Proving Their Absence journal March 2017
Dynamically discovering likely program invariants to support program evolution conference January 1999
An axiomatic basis for computer programming journal January 1983
Superoptimizer: a look at the smallest program journal November 1987
Superoptimizer: a look at the smallest program journal October 1987
Superoptimizer: a look at the smallest program journal October 1987
Superoptimizer: a look at the smallest program
  • Massalin, Henry
  • Proceedings of the second international conference on Architectual support for programming languages and operating systems https://doi.org/10.1145/36206.36194
conference October 1987
An axiomatic basis for computer programming journal October 1969
Predicate abstraction for software verification conference January 2002
Denali: a goal-directed superoptimizer
  • Joshi, Rajeev; Nelson, Greg; Randall, Keith
  • Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation - PLDI '02 https://doi.org/10.1145/512529.512566
conference January 2002
Denali: a goal-directed superoptimizer journal May 2002
Predicate abstraction for software verification journal January 2002
Supernode partitioning conference January 1988
Parallel Programmability and the Chapel Language journal August 2007
Stochastic Superoptimization preprint January 2012
The Formal Semantics of Programming Languages: An Introduction book January 1993

Cited By (2)

Verifying spatial properties of array computations
  • Orchard, Dominic; Contrastin, Mistral; Danish, Matthew
  • Proceedings of the ACM on Programming Languages, Vol. 1, Issue OOPSLA https://doi.org/10.1145/3133899
journal October 2017
Verifying Spatial Properties of Array Computations text January 2017

Similar Records

Using Verified Lifting to Optimize Legacy Stencil Codes (Final Project Report)
Technical Report · Tue Feb 09 23:00:00 EST 2021 · OSTI ID:1765083

Helium: lifting high-performance stencil kernels from stripped x86 binaries to halide DSL code
Journal Article · Wed Jun 03 00:00:00 EDT 2015 · ACM SIGPLAN Notices · OSTI ID:1457399

Designing a stencil compiler for the Connection Machine model CM-5
Technical Report · Fri Dec 30 23:00:00 EST 1994 · OSTI ID:10119048