skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION

Abstract

Here, we implement a variance-based distance metric (D n) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The D n metric is a gamma-distributed statistic that is more general than the χ 2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The D n statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the D n metric as a cost function and incorporating model parametric uncertainty as part of a scheme tomore » optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.« less

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2];  [2]; ORCiD logo [1];  [3];  [4]
  1. T-3 Fluid Dynamics and Solid Mechanics, Theoretical Division, Los Alamos National Laboratory, Los Alamos New Mexico USA
  2. CCS-2 Computational Physics and Methods, Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos New Mexico USA
  3. XCP-8 Verification and Analysis, X Computational Physics Division, Los Alamos National Laboratory, Los Alamos New Mexico USA
  4. Booker Scientific, Fredericksburg Texas USA
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1351197
Report Number(s):
LA-UR-16-29002
Journal ID: ISSN 2169-9275
Grant/Contract Number:  
AC52-06NA25396
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Journal of Geophysical Research. Oceans
Additional Journal Information:
Journal Volume: 122; Journal Issue: 4; Journal ID: ISSN 2169-9275
Publisher:
American Geophysical Union
Country of Publication:
United States
Language:
English
Subject:
58 GEOSCIENCES; Earth Sciences; Mathematics; sea ice model, validation metrics, model uncertainty

Citation Formats

Urrego-Blanco, Jorge R., Hunke, Elizabeth C., Urban, Nathan M., Jeffery, Nicole, Turner, Adrian K., Langenbrunner, James R., and Booker, Jane M. Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION. United States: N. p., 2017. Web. doi:10.1002/2016JC012602.
Urrego-Blanco, Jorge R., Hunke, Elizabeth C., Urban, Nathan M., Jeffery, Nicole, Turner, Adrian K., Langenbrunner, James R., & Booker, Jane M. Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION. United States. doi:10.1002/2016JC012602.
Urrego-Blanco, Jorge R., Hunke, Elizabeth C., Urban, Nathan M., Jeffery, Nicole, Turner, Adrian K., Langenbrunner, James R., and Booker, Jane M. Sat . "Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION". United States. doi:10.1002/2016JC012602. https://www.osti.gov/servlets/purl/1351197.
@article{osti_1351197,
title = {Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION},
author = {Urrego-Blanco, Jorge R. and Hunke, Elizabeth C. and Urban, Nathan M. and Jeffery, Nicole and Turner, Adrian K. and Langenbrunner, James R. and Booker, Jane M.},
abstractNote = {Here, we implement a variance-based distance metric (Dn) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The Dn metric is a gamma-distributed statistic that is more general than the χ2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The Dn statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the Dn metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.},
doi = {10.1002/2016JC012602},
journal = {Journal of Geophysical Research. Oceans},
issn = {2169-9275},
number = 4,
volume = 122,
place = {United States},
year = {2017},
month = {4}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Aircraft active and passive microwave validation of sea ice concentration from the Defense Meteorological Satellite Program special sensor microwave imager
journal, January 1991

  • Cavalieri, D. J.; Crawford, J. P.; Drinkwater, M. R.
  • Journal of Geophysical Research, Vol. 96, Issue C12
  • DOI: 10.1029/91JC02335

Global atmospheric forcing data for Arctic ice-ocean modeling
journal, January 2007

  • Hunke, Elizabeth C.; Holland, Marika M.
  • Journal of Geophysical Research, Vol. 112, Issue C4
  • DOI: 10.1029/2006JC003640

The global climatology of an interannually varying air–sea flux data set
journal, August 2008


Determination of sea ice parameters with the NIMBUS 7 SMMR
journal, June 1984

  • Cavalieri, D. J.; Gloersen, P.; Campbell, W. J.
  • Journal of Geophysical Research: Atmospheres, Vol. 89, Issue D4
  • DOI: 10.1029/JD089iD04p05355

Retrieval of Arctic Sea Ice Parameters by Satellite Passive Microwave Sensors: A Comparison of Eleven Sea Ice Concentration Algorithms
journal, November 2014

  • Ivanova, Natalia; Johannessen, Ola M.; Pedersen, Leif Toudal
  • IEEE Transactions on Geoscience and Remote Sensing, Vol. 52, Issue 11
  • DOI: 10.1109/TGRS.2014.2310136

Sea-ice extent and its trend provide limited metrics of model performance
journal, January 2014


A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing
journal, June 2011

  • Roy, Christopher J.; Oberkampf, William L.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 200, Issue 25-28
  • DOI: 10.1016/j.cma.2011.03.016

Assessment of sea ice simulations in the CMIP5 models
journal, January 2015


An Elastic–Viscous–Plastic Model for Sea Ice Dynamics
journal, September 1997


Viscous–Plastic Sea Ice Dynamics with the EVP Model: Linearization Issues
journal, June 2001


Predictability of the Arctic sea ice edge: PREDICTABILITY OF THE ARCTIC ICE EDGE
journal, February 2016

  • Goessling, H. F.; Tietsche, S.; Day, J. J.
  • Geophysical Research Letters, Vol. 43, Issue 4
  • DOI: 10.1002/2015GL067232

The Community Climate System Model Version 3 (CCSM3)
journal, June 2006

  • Collins, William D.; Bitz, Cecilia M.; Blackmon, Maurice L.
  • Journal of Climate, Vol. 19, Issue 11
  • DOI: 10.1175/JCLI3761.1

Trends in the sea ice cover using enhanced and compatible AMSR-E, SSM/I, and SMMR data
journal, January 2008

  • Comiso, Josefino C.; Nishio, Fumihiko
  • Journal of Geophysical Research, Vol. 113, Issue C2
  • DOI: 10.1029/2007JC004257

New Unified Sea Ice Thickness Climate Data Record
journal, January 2010

  • Lindsay, Ron
  • Eos, Transactions American Geophysical Union, Vol. 91, Issue 44
  • DOI: 10.1029/2010EO440001

Simulating the mass balance and salinity of Arctic and Antarctic sea ice. 1. Model description and validation
journal, January 2009


Improving Predictions of Arctic Sea Ice Extent
journal, June 2015

  • Stroeve, Julienne; Blanchard-Wrigglesworth, Ed; Guemas, Virginie
  • Eos, Vol. 96
  • DOI: 10.1029/2015EO031431

Deriving long-term time series of sea ice cover from satellite passive-microwave multisensor data sets
journal, July 1999

  • Cavalieri, D. J.; Parkinson, C. L.; Gloersen, P.
  • Journal of Geophysical Research: Oceans, Vol. 104, Issue C7
  • DOI: 10.1029/1999JC900081

Age characteristics in a multidecadal Arctic sea ice simulation
journal, January 2009

  • Hunke, Elizabeth C.; Bitz, Cecilia M.
  • Journal of Geophysical Research, Vol. 114, Issue C8
  • DOI: 10.1029/2008JC005186

Improved Sea Ice Shortwave Radiation Physics in CCSM4: The Impact of Melt Ponds and Aerosols on Arctic Sea Ice
journal, March 2012

  • Holland, Marika M.; Bailey, David A.; Briegleb, Bruce P.
  • Journal of Climate, Vol. 25, Issue 5
  • DOI: 10.1175/JCLI-D-11-00078.1

Skill metrics for evaluation and comparison of sea ice models
journal, September 2015

  • Dukhovskoy, Dmitry S.; Ubnoske, Jonathan; Blanchard‐Wrigglesworth, Edward
  • Journal of Geophysical Research: Oceans, Vol. 120, Issue 9
  • DOI: 10.1002/2015JC010989

The Role of Sea Ice Thickness Distribution in the Arctic Sea Ice Potential Predictability: A Diagnostic Approach with a Coupled GCM
journal, April 2012


Ridging, strength, and stability in high-resolution sea ice models
journal, January 2007

  • Lipscomb, William H.; Hunke, Elizabeth C.; Maslowski, Wieslaw
  • Journal of Geophysical Research, Vol. 112, Issue C3
  • DOI: 10.1029/2005JC003355

Level-ice melt ponds in the Los Alamos sea ice model, CICE
journal, November 2013


Forum for Arctic Modeling and Observational Synthesis (FAMOS): Past, current, and future activities: FAMOS PROJECT RESULTS
journal, June 2016

  • Proshutinsky, A.; Steele, M.; Timmermans, M. -L.
  • Journal of Geophysical Research: Oceans, Vol. 121, Issue 6
  • DOI: 10.1002/2016JC011898

An enhancement of the NASA Team sea ice algorithm
journal, May 2000

  • Markus, T.; Cavalieri, D. J.
  • IEEE Transactions on Geoscience and Remote Sensing, Vol. 38, Issue 3
  • DOI: 10.1109/36.843033

Uncertainty quantification and global sensitivity analysis of the Los Alamos sea ice model
journal, April 2016

  • Urrego‐Blanco, Jorge R.; Urban, Nathan M.; Hunke, Elizabeth C.
  • Journal of Geophysical Research: Oceans, Vol. 121, Issue 4
  • DOI: 10.1002/2015JC011558

The thickness distribution of sea ice
journal, November 1975

  • Thorndike, A. S.; Rothrock, D. A.; Maykut, G. A.
  • Journal of Geophysical Research, Vol. 80, Issue 33
  • DOI: 10.1029/JC080i033p04501

A sea-ice sensitivity study with a global ocean-ice model
journal, July 2012


Performance metrics for climate models
journal, January 2008

  • Gleckler, P. J.; Taylor, K. E.; Doutriaux, C.
  • Journal of Geophysical Research, Vol. 113, Issue D6
  • DOI: 10.1029/2007JD008972

The sea ice mass budget of the Arctic and its future change as simulated by coupled climate models
journal, November 2008

  • Holland, Marika M.; Serreze, Mark C.; Stroeve, Julienne
  • Climate Dynamics, Vol. 34, Issue 2-3
  • DOI: 10.1007/s00382-008-0493-4

Two modes of sea-ice gravity drainage: A parameterization for large-scale modeling: GRAVITY DRAINAGE
journal, May 2013

  • Turner, Adrian K.; Hunke, Elizabeth C.; Bitz, Cecilia M.
  • Journal of Geophysical Research: Oceans, Vol. 118, Issue 5
  • DOI: 10.1002/jgrc.20171