skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit

Abstract

We analyze the potential performance benefits of estimating expected quantities in large eddy simulations of turbulent flows using true ensembles rather than ergodic time averaging. Multiple realizations of the same flow are simulated in parallel, using slightly perturbed initial conditions to create unique instantaneous evolutions of the flow field. Each realization is then used to calculate statistical quantities. Provided each instance is sufficiently de-correlated, this approach potentially allows considerable reduction in the time to solution beyond the strong scaling limit for a given accuracy. This study focuses on the theory and implementation of the methodology in Nek5000, a massively parallel open-source spectral element code.

Authors:
 [1];  [1];  [1];  [1];  [1]
  1. Argonne National Lab. (ANL), Lemont, IL (United States)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1393517
Grant/Contract Number:
AC02-06CH11357
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Computer Physics Communications
Additional Journal Information:
Journal Volume: 219; Journal Issue: C; Journal ID: ISSN 0010-4655
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; 97 MATHEMATICS AND COMPUTING; Ensemble averaging; Nek5000

Citation Formats

Makarashvili, Vakhtang, Merzari, Elia, Obabko, Aleksandr, Siegel, Andrew, and Fischer, Paul. A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit. United States: N. p., 2017. Web. doi:10.1016/j.cpc.2017.05.023.
Makarashvili, Vakhtang, Merzari, Elia, Obabko, Aleksandr, Siegel, Andrew, & Fischer, Paul. A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit. United States. doi:10.1016/j.cpc.2017.05.023.
Makarashvili, Vakhtang, Merzari, Elia, Obabko, Aleksandr, Siegel, Andrew, and Fischer, Paul. Wed . "A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit". United States. doi:10.1016/j.cpc.2017.05.023.
@article{osti_1393517,
title = {A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit},
author = {Makarashvili, Vakhtang and Merzari, Elia and Obabko, Aleksandr and Siegel, Andrew and Fischer, Paul},
abstractNote = {We analyze the potential performance benefits of estimating expected quantities in large eddy simulations of turbulent flows using true ensembles rather than ergodic time averaging. Multiple realizations of the same flow are simulated in parallel, using slightly perturbed initial conditions to create unique instantaneous evolutions of the flow field. Each realization is then used to calculate statistical quantities. Provided each instance is sufficiently de-correlated, this approach potentially allows considerable reduction in the time to solution beyond the strong scaling limit for a given accuracy. This study focuses on the theory and implementation of the methodology in Nek5000, a massively parallel open-source spectral element code.},
doi = {10.1016/j.cpc.2017.05.023},
journal = {Computer Physics Communications},
number = C,
volume = 219,
place = {United States},
year = {Wed Jun 07 00:00:00 EDT 2017},
month = {Wed Jun 07 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on June 7, 2018
Publisher's Version of Record

Save / Share:
  • Methods and computing hardware advances have enabled accurate predictions of complex compressible turbulence phenomena, such as the generation of jet noise that motivates the present effort. However, limited understanding of underlying physical mechanisms restricts the utility of such predictions since they do not, by themselves, indicate a route to design improvements. Gradient-based optimization using adjoints can circumvent the flow complexity to guide designs, though this is predicated on the availability of a sufficiently accurate solution of the forward and adjoint systems. These are challenging to obtain, since both the chaotic character of the turbulence and the typical use of discretizationsmore » near their resolution limits in order to efficiently represent its smaller scales will amplify any approximation errors made in the adjoint formulation. Formulating a practical exact adjoint that avoids such errors is especially challenging if it is to be compatible with state-of-the-art simulation methods used for the turbulent flow itself. Automatic differentiation (AD) can provide code to calculate a nominally exact adjoint, but existing general-purpose AD codes are inefficient to the point of being prohibitive for large-scale turbulence simulations. Here, we analyze the compressible flow equations as discretized using the same high-order workhorse methods used for many high-fidelity compressible turbulence simulations, and formulate a practical space–time discrete-adjoint method without changing the basic discretization. A key step is the definition of a particular discrete analog of the continuous norm that defines our cost functional; our selection leads directly to an efficient Runge–Kutta-like scheme, though it would be just first-order accurate if used outside the adjoint formulation for time integration, with finite-difference spatial operators for the adjoint system. Its computational cost only modestly exceeds that of the flow equations. We confirm that its accuracy is limited by computing precision, and we demonstrate it on the aeroacoustic control of a mixing layer with a challengingly broad range of turbulence scales. For comparison, the error from a corresponding discretization of the continuous-adjoint equations is quantified to potentially explain its limited success in past efforts to control jet noise. The differences are illuminating: the continuous-adjoint is shown to suffer from exponential error growth in (reverse) time even for the best-resolved largest turbulence scales. Implications for jet noise reduction and turbulence control in general are discussed.« less
  • Cited by 3
  • Abstract not provided.