Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

A massively parallel and memory-efficient FEM toolbox with a hybrid total FETI solver with accelerator support

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [2];  [1];  [1];  [1];  [1];  [1];  [1];  [1]
  1. IT4Innovations, VSB-Technical University of Ostrava, Ostrava, Czech Republic
  2. IT4Innovations, VSB-Technical University of Ostrava, Ostrava, Czech Republic; Department of Applied Mathematics, VSB-Technical University of Ostrava, Ostrava, Czech Republic

In this article, we present the ExaScale PaRallel finite element tearing and interconnecting SOlver (ESPRESO) finite element method (FEM) library, which includes an FEM toolbox with interfaces to professional and open-source simulation tools, and a massively parallel hybrid total finite element tearing and interconnecting (HTFETI) solver which can fully utilize the Oak Ridge Leadership Computing Facility Titan supercomputer and achieve superlinear scaling. This article presents several new techniques for finite element tearing and interconnecting (FETI) solvers designed for efficient utilization of supercomputers with a focus on (i) performance—we present a fivefold reduction of solver runtime for the Laplace equation by redesigning the FETI solver and offloading the key workload to the accelerator. We compare Intel Xeon Phi 7120p and Tesla K80 and P100 accelerators to Intel Xeon E5-2680v3 and Xeon Phi 7210 central processing units; and (ii) memory efficiency—we present two techniques which increase the efficiency of the HTFETI solver 1.8 times and push the limits of the largest possible problem ESPRESO that can solve from 124 to 223 billion unknowns for problems with unstructured meshes. Finally, we show that by dynamically tuning hardware parameters, we can reduce energy consumption by up to 33%.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); UT-Battelle LLC/ORNL, Oak Ridge, TN (Unted States)
Sponsoring Organization:
USDOE Office of Science
DOE Contract Number:
AC05-00OR22725
OSTI ID:
1565782
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 4 Vol. 33; ISSN 1094-3420
Publisher:
SAGE
Country of Publication:
United States
Language:
English

References (14)

A Highly Scalable Parallel Implementation of Balancing Domain Decomposition by Constraints journal January 2014
Total FETI-an easier implementable variant of the FETI method for numerical solution of elliptic PDE journal June 2006
Accelerating sparse Cholesky factorization on GPUs journal November 2016
The READEX formalism for automatic tuning for energy efficiency journal January 2017
Matching-based preprocessing algorithms to the solution of saddle-point problems in large-scale nonconvex interior-point optimization journal February 2007
Optimal convergence properties of the FETI domain decomposition method journal May 1994
An Approximate Minimum Degree Ordering Algorithm journal October 1996
Highly scalable parallel domain decomposition methods with an application to biomechanics journal January 2010
Evaluation of the HPC Applications Dynamic Behavior in Terms of Energy Consumption
  • Vysocky, O.; Beseda, M.; Riha, L.
  • The Fifth International Conference on Parallel, Distributed, Grid and Cloud Computing for Engineering, Civil-Comp Proceedings https://doi.org/10.4203/ccp.111.3
conference May 2017
Parallel iterative solution of the incompressible Navier–Stokes equations with application to rotating wings journal November 2015
On Large-Scale Diagonalization Techniques for the Anderson Model of Localization journal January 2008
A Highly Scalable Implementation of Inexact Nonlinear FETI-DP Without Sparse Direct Solvers book January 2016
Non-overlapping domain decomposition methods in structural mechanics journal December 2006
An overview of the Trilinos project journal September 2005