skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Validation metrics for turbulent plasma transport

Journal Article · · Physics of Plasmas
DOI:https://doi.org/10.1063/1.4954151· OSTI ID:1259193

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnostics to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. Furthermore, the utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak, as part of a multi-year transport model validation activity.

Research Organization:
Univ. of California, San Diego, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Fusion Energy Sciences (FES)
Grant/Contract Number:
FG02-06ER54871; SC0006957
OSTI ID:
1259193
Alternate ID(s):
OSTI ID: 1420654; OSTI ID: 1434473
Journal Information:
Physics of Plasmas, Journal Name: Physics of Plasmas Vol. 23 Journal Issue: 6; ISSN 1070-664X
Publisher:
American Institute of PhysicsCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 42 works
Citation information provided by
Web of Science

References (156)

Measurements of the cross-phase angle between density and electron temperature fluctuations and comparison with gyrokinetic simulations journal May 2010
Validating simulations of core tokamak turbulence: current status and future directions journal July 2008
Approaching the investigation of plasma turbulence through a rigorous verification and validation procedure: A practical examplea) journal May 2015
Zonal flows in plasma—a review journal April 2005
The 2008 Public Release of the International Multi-tokamak Confinement Profile Database journal October 2008
Electron temperature gradient driven turbulence journal May 2000
Damped eigenmode saturation in plasma fluid turbulence journal January 2011
Simulation of plasma turbulence in scrape-off layer conditions: the GBS code, simulation results and code validation journal November 2012
Chapter 3: MHD stability, operational limits and disruptions journal December 1999
Multi-scale gyrokinetic simulation of tokamak plasmas: enhanced heat loss due to cross-scale coupling of plasma turbulence journal December 2015
Chapter 1: Overview and summary journal December 1999
A Landau fluid model for electromagnetic plasma microturbulence journal July 2001
Role of trapped electron mode turbulence in internal transport barrier control in the Alcator C-Mod Tokamak journal May 2004
Effects of E×B velocity shear and magnetic shear on turbulence and transport in magnetic confinement devices journal May 1997
Modelling of JET Diagnostics Using Bayesian Graphical Models journal March 2011
Verification, validation, and predictive capability in computational engineering and physics journal September 2004
Quantification of Uncertainty in Computational Fluid Dynamics journal January 1997
Edge-localized-modes in tokamaks journal September 2014
Toward a Better Understanding of Model Validation Metrics journal July 2011
Validation studies of gyrofluid and gyrokinetic predictions of transport and turbulence stiffness using the DIII-D tokamak journal July 2013
Verification and validation of linear gyrokinetic simulation of Alfvén eigenmodes in the DIII-D tokamak journal August 2012
Electron temperature critical gradient and transport stiffness in DIII-D journal July 2015
Comparison of turbulence measurements from DIII-D low-mode and high-performance plasmas to turbulence simulations and models journal May 2002
Investigation of fast particle driven instabilities by 2D electron cyclotron emission imaging on ASDEX Upgrade journal November 2011
Multi-scale gyrokinetic simulation of Alcator C-Mod tokamak discharges journal March 2014
An introduction to ROC analysis journal June 2006
A global collisionless PIC code in magnetic coordinates journal September 2007
Gyrokinetic studies of core turbulence features in ASDEX Upgrade H-mode plasmas journal April 2015
Comprehensive Approach to Verification and Validation of CFD Simulations—Part 1: Methodology and Procedures journal July 2001
Trapped electron mode driven electron heat transport in JET: experimental investigation and gyro-kinetic theory validation journal September 2015
Role of stable eigenmodes in saturated local plasma turbulence journal February 2006
Kinetic calculation of neoclassical transport including self-consistent electron and impurity dynamics journal July 2008
Finite-Resistivity Instabilities of a Sheet Pinch journal January 1963
Nonlinear electromagnetic gyrokinetic equation for plasmas with large mean flows journal July 1998
Multiscale features of density and frequency spectra from nonlinear gyrokinetics journal October 2008
A unified method for operator evaluation in local Grad–Shafranov plasma equilibria journal September 2009
Measures of agreement between computation and experiment: Validation metrics journal September 2006
Determination of Z eff by integrating measurements from x-ray tomography and charge exchange recombination spectroscopy journal November 2015
Reduced electron thermal transport in low collisionality H-mode plasmas in DIII-D and the importance of TEM/ETG-scale turbulence journal January 2012
A flux-matched gyrokinetic analysis of DIII-D L-mode turbulence journal December 2014
Study of the L-mode tokamak plasma “shortfall” with local and global nonlinear gyrokinetic δf particle-in-cell simulation journal November 2014
Advances in validating gyrokinetic turbulence models against L- and H-mode plasmas journal May 2011
Profile stiffness and global confinement journal July 2004
Basic physics of Alfvén instabilities driven by energetic particles in toroidally confined plasmas journal May 2008
Sizing up plasmas using dimensionless parameters journal August 2008
Comparisons and physics basis of tokamak transport models and turbulence simulations journal March 2000
Nonlinear gyrokinetic simulations of the I-mode high confinement regime and comparisons with experimenta) journal May 2015
Full linearized Fokker–Planck collisions in neoclassical transport simulations journal December 2011
3D toroidal physics: Testing the boundaries of symmetry breakinga) journal May 2015
L-mode validation studies of gyrokinetic turbulence simulations via multiscale and multifield turbulence measurements on the DIII-D tokamak journal May 2011
Langmuir probe-based observables for plasma-turbulence code validation and application to the TORPEX basic plasma physics experiment journal May 2009
Gyro‐Landau fluid models for toroidal geometry journal October 1992
Effect of finite aspect ratio on the neoclassical ion thermal conductivity in the banana regime journal January 1982
Fast Ion Induced Shearing of 2D Alfvén Eigenmodes Measured by Electron Cyclotron Emission Imaging journal February 2011
Fast-ion Dα measurements of the fast-ion distribution (invited) journal October 2010
Nonlinear gyrokinetic equations for low-frequency electromagnetic waves in general plasma equilibria journal January 1982
Fluid models of phase mixing, Landau damping, and nonlinear gyrokinetic dynamics journal March 1992
Verification and validation for magnetic fusion journal May 2010
Transport simulation on L-mode and improved confinement associated with current profile modification journal June 1995
Quantitative predictions of tokamak energy confinement from first‐principles simulations with kinetic effects journal June 1995
Comparing simulation of plasma turbulence with experiment. II. Gyrokinetic simulations journal December 2002
Bootstrap current and neoclassical transport in tokamaks of arbitrary collisionality and aspect ratio journal September 1997
Benchmarking of codes for electron cyclotron heating and electron cyclotron current drive under ITER conditions journal January 2008
Heating and current drive by electron cyclotron waves journal May 2004
Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic tokamak geometry journal September 2009
A theory-based transport model with comprehensive physics journal May 2007
A gyro-Landau-fluid transport model journal July 1997
Implementation and application of two synthetic diagnostics for validating simulations of core tokamak turbulence journal May 2009
Review of recent experimental and modeling progress in the lower hybrid range of frequencies at ITER relevant parameters journal June 2014
Verification and validation in computational fluid dynamics journal April 2002
Validation in fusion research: Towards guidelines and best practices journal June 2008
Chapter 2: Plasma confinement and transport journal December 1999
Comparing simulation of plasma turbulence with experiment journal January 2002
Fluid moment models for Landau damping with application to the ion-temperature-gradient instability journal June 1990
Chapter 2: Plasma confinement and transport journal June 2007
Rainbow Color Map (Still) Considered Harmful journal March 2007
A review of the dimensionless parameter scaling studies journal December 1996
Application of dimensionless parameter scaling techniques to the design and interpretation of magnetic fusion experiments journal March 2008
ITER: burning plasma physics experiment journal March 2003
Chapter 6: Steady state operation journal June 2007
Resonant magnetic perturbations of edge-plasmas in toroidal confinement devices journal November 2015
First scenario development with the JET new ITER-like wall journal December 2013
Theory of plasma transport in toroidal confinement systems journal April 1976
Drift waves and transport journal April 1999
Gyrofluid turbulence models with kinetic effects journal March 1993
Toroidal gyrofluid equations for simulations of tokamak turbulence journal November 1996
Gyro-Landau fluid equations for trapped and passing particles journal October 2005
Derivation via free energy conservation constraints of gyrofluid equations with finite-gyroradius electromagnetic nonlinearities journal October 2010
Full-F gyrofluid model journal July 2013
Interaction of tearing modes with external structures in cylindrical geometry (plasma) journal July 1993
Comparison of initial value and eigenvalue codes for kinetic toroidal plasma instabilities journal August 1995
An Eulerian gyrokinetic-Maxwell solver journal April 2003
The nonlinear gyro-kinetic flux tube code GKW journal December 2009
Computation-Communication Overlap Techniques for Parallel Spectral Calculations in Gyrokinetic Vlasov Simulations journal January 2013
Turbulent Transport Reduction by Zonal Flows: Massively Parallel Simulations journal September 1998
Nonlocal neoclassical transport in tokamak and spherical torus experiments journal August 2006
Electromagnetic gyrokinetic δf particle-in-cell turbulence simulation with realistic equilibrium profiles and geometry journal January 2007
Global full- f gyrokinetic simulations of plasma turbulence journal November 2007
Tokamak profile prediction using direct gyrokinetic and neoclassical simulation journal June 2009
Direct multiscale coupling of a transport code to gyrokinetic turbulence codes journal May 2010
Synergistic cross-scale coupling of turbulence in a tokamak plasma journal November 2014
Cross-Scale Interactions between Electron and Ion Scale Turbulence in a Tokamak Plasma journal June 2015
A new gyrokinetic quasilinear transport model applied to particle transport in tokamak plasmas journal November 2007
Physics basis of Multi-Mode anomalous transport module journal March 2013
Electron heat transport studies journal November 2006
Experimental Study of the Ion Critical-Gradient Length and Stiffness Level and the Impact of Rotation in the JET Tokamak journal April 2009
Electron profile stiffness and critical gradient studies journal August 2012
Observation of a Critical Gradient Threshold for Electron Temperature Fluctuations in the DIII-D Tokamak journal January 2013
The European turbulence code benchmarking effort: turbulence driven by thermal gradients in magnetically confined plasmas journal November 2008
A Key to Improved Ion Core Confinement in the JET Tokamak: Ion Stiffness Mitigation due to Combined Plasma Rotation and Low Magnetic Shear journal September 2011
Simulation of microtearing turbulence in national spherical torus experiment journal May 2012
Nonlinear Stabilization of Tokamak Microturbulence by Fast Ions journal October 2013
Ion temperature profile stiffness: non-linear gyrokinetic simulations and comparison with experiment journal February 2014
Impurity transport, turbulence transitions and intrinsic rotation in Alcator C-Mod plasmas journal November 2014
Electromagnetic stabilization of tokamak microturbulence in a high- β regime journal November 2014
A standard DT supershot simulation journal September 1994
Local transport in Joint European Tokamak edge-localized, high-confinement mode plasmas with H, D, DT, and T isotopes journal December 2000
The behaviour of fast ions in tokamak experiments journal April 1994
Fast electron bremsstrahlung in axisymmetric magnetic configuration journal September 2008
Fullwave Simulations of Lower Hybrid Waves Coupled to 3D Fokker-Planck Solver: Comparison with Alcator C-Mod Experiment journal July 2011
Time dependent evolution of RF-generated non-thermal particle distributions in fusion plasmas journal March 2014
Ion cyclotron range of frequencies mode conversion electron heating in deuterium–hydrogen plasmas in the Alcator C-Mod tokamak journal May 2003
Lower hybrid current drive experiments on Alcator C-Mod: Comparison with theory and simulation journal May 2008
Observation and modelling of ion cyclotron range of frequencies waves in the mode conversion region of Alcator C-Mod journal July 2005
Validation of full-wave simulations for mode conversion of waves in the ion cyclotron range of frequencies with phase contrast imaging in Alcator C-Mod journal August 2015
Measurements of core electron temperature and density fluctuations in DIII-D and comparison to nonlinear gyrokinetic simulations journal May 2008
Improved profile fitting and quantification of uncertainty in experimental measurements of impurity transport coefficients using Gaussian process regression journal January 2015
Integrated Data Analysis of Profile Diagnostics at ASDEX Upgrade journal October 2010
Integrated data analysis at TJ-II: The density profile journal July 2011
Edge radial electric field structure in quiescent H-mode plasmas in the DIII-D tokamak journal April 2004
Suppression of turbulence and transport by sheared flow journal January 2000
Ray tracing study of the electron cyclotron current drive in DIII-D using 60 GHz journal January 1989
Linear and nonlinear verification of gyrokinetic microstability codes journal December 2011
On the Validation of Gyrokinetic L-Mode Simulations journal April 2016
Coupled ion temperature gradient and trapped electron mode to electron temperature gradient mode gyrokinetic simulations journal May 2007
The effect of ion-scale dynamics on electron-temperature-gradient turbulence journal July 2007
Scale Separation between Electron and Ion Thermal Transport journal May 2008
Fidelity of reduced and realistic electron mass ratio multi-scale gyrokinetic simulations of tokamak discharges journal April 2015
Methodology for turbulence code validation: Quantification of simulation-experiment agreement and application to the TORPEX experiment journal March 2011
Summarizing multiple aspects of model performance in a single diagram journal April 2001
Electrostatic instabilities, turbulence and fast ion interactions in the TORPEX device journal November 2010
System for simulating fluctuation diagnostics for application to turbulence computations journal January 2006
Comparison of scrape-off layer turbulence simulations with experiments using a synthetic gas puff imaging diagnostic journal February 2011
High sensitivity beam emission spectroscopy for core plasma turbulence imaging (invited) journal October 2006
Spatial transfer function for the beam emission spectroscopy diagnostic on DIII-D journal October 2006
Saturation of Gyrokinetic Turbulence through Damped Eigenmodes journal March 2011
2D properties of core turbulence on DIII-D and comparison to gyrokinetic simulations journal March 2012
The Homogeneous Chaos journal October 1938
Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics journal January 2009
Chapter 3: MHD stability, operational limits and disruptions journal June 2007
Predictive capability of MHD stability limits in high performance DIII-D discharges journal July 2002
Theory and Simulation Basis for Magnetohydrodynamic Stability in DIII-D journal October 2005
Fast imaging of transients and coherent magnetohydrodynamic modes in DIII-D journal May 2009
Radial Structure of Alfvén Eigenmodes in the DIII-D Tokamak through Electron-Cyclotron-Emission Measurements journal September 2006
On the application of electron cyclotron emission imaging to the validation of theoretical models of magnetohydrodynamic activity journal May 2011
A design retrospective of the DIII-D tokamak journal May 2002

Cited By (11)

A modern perspective on flow instability and shockwave phenomena in reacting gas multiphase system excited by direct current journal November 2019
Shaping effects on scrape-off layer plasma turbulence: A rigorous validation of three-dimensional simulations against TCV measurements journal January 2020
Tractable flux-driven temperature, density, and rotation profile evolution with the quasilinear gyrokinetic transport model QuaLiKiz journal November 2017
Criteria for the importance of multi-scale interactions in turbulent transport simulations journal June 2019
Real-time-capable prediction of temperature and density profiles in a tokamak using RAPTOR and a first-principle-based transport model journal July 2018
Comparison between measured and predicted turbulence frequency spectra in ITG and TEM regimes journal May 2017
Validation study of GENE on ASDEX Upgrade using perturbative thermal diffusivity measured with partial sawtooth heat pulses journal September 2018
Propagation of input parameter uncertainties in transport models journal October 2018
Optimal MSE polarisation angle and q-profile estimation using Kalman filters and the plasma simulator RAPTOR journal February 2019
Main-ion intrinsic toroidal rotation across the ITG/TEM boundary in DIII-D discharges during ohmic and electron cyclotron heating journal April 2019
Experimental trends of reflectometry frequency spectra emerging from a systematic analysis of the Tore Supra database journal March 2019