 
Summary: ULTRASCALABLE IMPLICIT FINITE ELEMENT ANALYSES IN SOLID MECHANICS
WITH OVER A HALF A BILLION DEGREES OF FREEDOM
MARK F. ADAMS1 , HARUN H. BAYRAKTAR2,6 , TONY M. KEAVENY2,3,4
, AND PANAYIOTIS PAPADOPOULOS3,5
Abstract. The solution of elliptic diffusion operators is the computational bottleneck in many simulations in a wide range
of engineering and scientific disciplines. We present a truly scalableultrascalablealgebraic multigrid (AMG) linear solver
for the diffusion operator in unstructured elasticity problems. Scalability is demonstrated with speedup studies of a nonlinear
microfinite element analyses of a human vertebral body with over a half of a billion degrees of freedom on up to 4088 processors
on the ACSI White machine. This work is significant because in the domain of unstructured implicit finite element analysis
in solid mechanics with complex geometry, this is the first demonstration of a highly parallel and efficient application of a
mathematically optimal linear solution method on a common large scale computing platformthe IBM SP Power3.
1. Introduction. The availability of large high performance computers is providing scientists and
engineers with the opportunity to simulate a variety of complex physical systems with ever more accuracy
and thereby exploit the advantages of computer simulations over laboratory experiments. The finite element
(FE) method is widely used for these simulations. The finite element method requires that one or several
linearized systems of sparse unstructured algebraic equations (the stiffness matrix) be solved for static
analyses, or at each time step when implicit time integration is used. These linear system solves are the
computational bottleneck (once the simulation has been setup and before the results are interpreted) as
the scale of problems increases. Direct solvers (eg, LU decomposition) and one level solvers (eg, diagonally
preconditioned CG) have been popular in the past but, with the ever decreasing cost of computing, the
