DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The Plumbing of Land Surface Models: Is Poor Performance a Result of Methodology or Data Quality?

Abstract

The PALS Land Surface Model Benchmarking Evaluation Project (PLUMBER) illustrated the value of prescribing a priori performance targets in model intercomparisons. It showed that the performance of turbulent energy flux predictions from different land surface models, at a broad range of flux tower sites using common evaluation metrics, was on average worse than relatively simple empirical models. For sensible heat fluxes, all land surface models were outperformed by a linear regression against downward shortwave radiation. For latent heat flux, all land surface models were outperformed by a regression against downward shortwave, surface air temperature and relative humidity. These results are explored here in greater detail and possible causes are investigated. We examine whether particular metrics or sites unduly influence the collated results, whether results change according to time-scale aggregation and whether a lack of energy conservation in flux tower data gives the empirical models an unfair advantage in the intercomparison. We demonstrate that energy conservation in the observational data is not responsible for these results. We also show that the partitioning between sensible and latent heat fluxes in LSMs, rather than the calculation of available energy, is the cause of the original findings. In conclusion, we present evidence suggesting thatmore » the nature of this partitioning problem is likely shared among all contributing LSMs. While we do not find a single candidate explanation for why land surface models perform poorly relative to empirical benchmarks in PLUMBER, we do exclude multiple possible explanations and provide guidance on where future research should focus.« less

Authors:
 [1];  [1];  [1];  [2];  [3];  [3];  [4];  [5];  [6];  [5];  [7];  [8];  [8];  [7];  [9];  [10];  [11];  [12];  [11];  [12] more »;  [13] « less
  1. ARC Centre of Excellence for Climate Systems Science, Sydney, NSW (Australia)
  2. ETH Zurich, Zurich (Switzerland)
  3. Met Office, Exeter (United Kingdom)
  4. ECMWF, Reading (United Kingdom)
  5. CNRM-GAME, Toulouse (France)
  6. Helmholtz Centre for Environmental Research (UFZ), Leipzig (Germany)
  7. George Mason Univ., Fairfax, VA (United States)
  8. NOAA/NCEP/EMC, College Park, MD (United States)
  9. CSIRO, Canberra, ACT (Australia)
  10. Royal Netherlands Meteorological Institute (KNMI), De Bilt (Netherlands)
  11. NASA Goddard Space Flight Center (GSFC), Greenbelt, MD (United States)
  12. CSIRO, Aspendale, VIC (Australia)
  13. Lab. des Sciences du Climat et de l’Environnement, Gif-sur-Yvette (France)
Publication Date:
Research Org.:
Oregon State Univ., Corvallis, OR (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER); Terrestrial Carbon Program
OSTI Identifier:
1467090
Grant/Contract Number:  
FG02-04ER63917; FG02-04ER63911
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Hydrometeorology
Additional Journal Information:
Journal Volume: 17; Journal Issue: 6; Journal ID: ISSN 1525-755X
Publisher:
American Meteorological Society
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES

Citation Formats

Haughton, Ned, Abramowitz, Gab, Pitman, Andy J., Or, Dani, Best, Martin J., Johnson, Helen R., Balsamo, Gianpaolo, Boone, Aaron, Cuntz, Matthias, Decharme, Bertrand, Dirmeyer, Paul A., Dong, Jairui, Ek, Michael, Guo, Zichang, Haverd, Vanessa, van den Hurk, Bart J. J., Nearing, Grey S., Pak, Bernard, Santanello, Jr., Joe A., Stevens, Lauren E., and Vuichard, Nicolas. The Plumbing of Land Surface Models: Is Poor Performance a Result of Methodology or Data Quality?. United States: N. p., 2016. Web. doi:10.1175/JHM-D-15-0171.1.
Haughton, Ned, Abramowitz, Gab, Pitman, Andy J., Or, Dani, Best, Martin J., Johnson, Helen R., Balsamo, Gianpaolo, Boone, Aaron, Cuntz, Matthias, Decharme, Bertrand, Dirmeyer, Paul A., Dong, Jairui, Ek, Michael, Guo, Zichang, Haverd, Vanessa, van den Hurk, Bart J. J., Nearing, Grey S., Pak, Bernard, Santanello, Jr., Joe A., Stevens, Lauren E., & Vuichard, Nicolas. The Plumbing of Land Surface Models: Is Poor Performance a Result of Methodology or Data Quality?. United States. https://doi.org/10.1175/JHM-D-15-0171.1
Haughton, Ned, Abramowitz, Gab, Pitman, Andy J., Or, Dani, Best, Martin J., Johnson, Helen R., Balsamo, Gianpaolo, Boone, Aaron, Cuntz, Matthias, Decharme, Bertrand, Dirmeyer, Paul A., Dong, Jairui, Ek, Michael, Guo, Zichang, Haverd, Vanessa, van den Hurk, Bart J. J., Nearing, Grey S., Pak, Bernard, Santanello, Jr., Joe A., Stevens, Lauren E., and Vuichard, Nicolas. Wed . "The Plumbing of Land Surface Models: Is Poor Performance a Result of Methodology or Data Quality?". United States. https://doi.org/10.1175/JHM-D-15-0171.1. https://www.osti.gov/servlets/purl/1467090.
@article{osti_1467090,
title = {The Plumbing of Land Surface Models: Is Poor Performance a Result of Methodology or Data Quality?},
author = {Haughton, Ned and Abramowitz, Gab and Pitman, Andy J. and Or, Dani and Best, Martin J. and Johnson, Helen R. and Balsamo, Gianpaolo and Boone, Aaron and Cuntz, Matthias and Decharme, Bertrand and Dirmeyer, Paul A. and Dong, Jairui and Ek, Michael and Guo, Zichang and Haverd, Vanessa and van den Hurk, Bart J. J. and Nearing, Grey S. and Pak, Bernard and Santanello, Jr., Joe A. and Stevens, Lauren E. and Vuichard, Nicolas},
abstractNote = {The PALS Land Surface Model Benchmarking Evaluation Project (PLUMBER) illustrated the value of prescribing a priori performance targets in model intercomparisons. It showed that the performance of turbulent energy flux predictions from different land surface models, at a broad range of flux tower sites using common evaluation metrics, was on average worse than relatively simple empirical models. For sensible heat fluxes, all land surface models were outperformed by a linear regression against downward shortwave radiation. For latent heat flux, all land surface models were outperformed by a regression against downward shortwave, surface air temperature and relative humidity. These results are explored here in greater detail and possible causes are investigated. We examine whether particular metrics or sites unduly influence the collated results, whether results change according to time-scale aggregation and whether a lack of energy conservation in flux tower data gives the empirical models an unfair advantage in the intercomparison. We demonstrate that energy conservation in the observational data is not responsible for these results. We also show that the partitioning between sensible and latent heat fluxes in LSMs, rather than the calculation of available energy, is the cause of the original findings. In conclusion, we present evidence suggesting that the nature of this partitioning problem is likely shared among all contributing LSMs. While we do not find a single candidate explanation for why land surface models perform poorly relative to empirical benchmarks in PLUMBER, we do exclude multiple possible explanations and provide guidance on where future research should focus.},
doi = {10.1175/JHM-D-15-0171.1},
journal = {Journal of Hydrometeorology},
number = 6,
volume = 17,
place = {United States},
year = {Wed May 25 00:00:00 EDT 2016},
month = {Wed May 25 00:00:00 EDT 2016}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 40 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Towards a public, standardized, diagnostic benchmarking system for land surface models
journal, January 2012


A study of land-atmosphere interactions during summertime rainfall using a mesoscale model
journal, January 2002


The Plumbing of Land Surface Models: Benchmarking Model Performance
journal, June 2015

  • Best, M. J.; Abramowitz, G.; Johnson, H. R.
  • Journal of Hydrometeorology, Vol. 16, Issue 3
  • DOI: 10.1175/JHM-D-14-0158.1

Climate model dependence and the replicate Earth paradigm
journal, December 2012


Extension of the Averaging Time in Eddy-Covariance Measurements and Its Effect on the Energy Balance Closure
journal, April 2014

  • Charuchittipan, Doojdao; Babel, Wolfgang; Mauder, Matthias
  • Boundary-Layer Meteorology, Vol. 152, Issue 3
  • DOI: 10.1007/s10546-014-9922-6

Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models: DIFFERENCES BETWEEN HYDROLOGICAL MODELS
journal, August 2008

  • Clark, Martyn P.; Slater, Andrew G.; Rupp, David E.
  • Water Resources Research, Vol. 44, Issue 12
  • DOI: 10.1029/2007WR006735

Forest water use and water use efficiency at elevated CO 2 : a model-data intercomparison at two contrasting temperate forest FACE sites
journal, March 2013

  • De Kauwe, Martin G.; Medlyn, Belinda E.; Zaehle, Sönke
  • Global Change Biology, Vol. 19, Issue 6
  • DOI: 10.1111/gcb.12164

A History and Review of the Global Soil Wetness Project (GSWP)
journal, October 2011


Exploring the hydrological robustness of model-parameter values with alpha shapes: Hydrological Robustness and Depth
journal, October 2013

  • Guerrero, José-Luis; Westerberg, Ida K.; Halldin, Sven
  • Water Resources Research, Vol. 49, Issue 10
  • DOI: 10.1002/wrcr.20533

GLACE: The Global Land–Atmosphere Coupling Experiment. Part II: Analysis
journal, August 2006

  • Guo, Zhichang; Dirmeyer, Paul A.; Koster, Randal D.
  • Journal of Hydrometeorology, Vol. 7, Issue 4
  • DOI: 10.1175/JHM511.1

Parameter estimation of a land surface scheme using multicriteria methods
journal, August 1999

  • Gupta, H. V.; Bastidas, L. A.; Sorooshian, S.
  • Journal of Geophysical Research: Atmospheres, Vol. 104, Issue D16
  • DOI: 10.1029/1999JD900154

Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling
journal, October 2009


Weighting climate model ensembles for mean and variance estimates
journal, February 2015


The Project for Intercomparison of Land-surface Parametrization Schemes (PILPS): 1992 to 1995
journal, November 1996

  • Henderson-Sellers, A.; McGuffie, K.; Pitman, A. J.
  • Climate Dynamics, Vol. 12, Issue 12
  • DOI: 10.1007/s003820050147

Comparison of Noah simulations with eddy covariance and soil water measurements at a winter wheat stand
journal, March 2011


Estimating random errors of eddy covariance data: An extended two-tower approach
journal, April 2013

  • Kessomkiat, Wittaya; Franssen, Harrie-Jan Hendricks; Graf, Alexander
  • Agricultural and Forest Meteorology, Vol. 171-172
  • DOI: 10.1016/j.agrformet.2012.11.019

Regions of Strong Coupling Between Soil Moisture and Precipitation
journal, August 2004


GLACE: The Global Land–Atmosphere Coupling Experiment. Part I: Overview
journal, August 2006

  • Koster, Randal D.; Sud, Y. C.; Guo, Zhichang
  • Journal of Hydrometeorology, Vol. 7, Issue 4
  • DOI: 10.1175/JHM510.1

On the Nature of Soil Moisture in Land Surface Models
journal, August 2009

  • Koster, Randal D.; Guo, Zhichang; Yang, Rongqian
  • Journal of Climate, Vol. 22, Issue 16
  • DOI: 10.1175/2009JCLI2832.1

Climate model genealogy: CLIMATE MODEL GENEALOGY
journal, April 2011


THE WCRP CMIP3 Multimodel Dataset: A New Era in Climate Change Research
journal, September 2007

  • Meehl, Gerald A.; Covey, Curt; Delworth, Thomas
  • Bulletin of the American Meteorological Society, Vol. 88, Issue 9
  • DOI: 10.1175/BAMS-88-9-1383

Are we unnecessarily constraining the agility of complex process-based models?
journal, January 2015

  • Mendoza, Pablo A.; Clark, Martyn P.; Barlage, Michael
  • Water Resources Research, Vol. 51, Issue 1
  • DOI: 10.1002/2014WR015820

The evolution of, and revolution in, land surface schemes designed for climate models
journal, January 2003

  • Pitman, A. J.
  • International Journal of Climatology, Vol. 23, Issue 5
  • DOI: 10.1002/joc.893

Key results and implications from phase 1(c) of the Project for Intercomparison of Land-surface Parametrization Schemes
journal, September 1999

  • Pitman, A. J.; Henderson-Sellers, A.; Desborough, C. E.
  • Climate Dynamics, Vol. 15, Issue 9
  • DOI: 10.1007/s003820050309

A test of the optimality approach to modelling canopy properties and CO 2 uptake by natural vegetation
journal, December 2007


Impact of soil moisture‐climate feedbacks on CMIP5 projections: First results from the GLACE‐CMIP5 experiment
journal, October 2013

  • Seneviratne, Sonia I.; Wilhelm, Micah; Stanelle, Tanja
  • Geophysical Research Letters, Vol. 40, Issue 19
  • DOI: 10.1002/grl.50956

A data-driven analysis of energy balance closure across FLUXNET research sites: The role of landscape scale heterogeneity
journal, April 2013


An Overview of CMIP5 and the Experiment Design
journal, April 2012

  • Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A.
  • Bulletin of the American Meteorological Society, Vol. 93, Issue 4
  • DOI: 10.1175/BAMS-D-11-00094.1

Acceleration of Land Surface Model Development over a Decade of Glass
journal, December 2011

  • van den Hurk, Bart; Best, Martin; Dirmeyer, Paul
  • Bulletin of the American Meteorological Society, Vol. 92, Issue 12
  • DOI: 10.1175/BAMS-D-11-00007.1

How Much Do Different Land Models Matter for Climate Simulation? Part I: Climatology and Variability
journal, June 2010

  • Wei, Jiangfeng; Dirmeyer, Paul A.; Guo, Zhichang
  • Journal of Climate, Vol. 23, Issue 11
  • DOI: 10.1175/2010JCLI3177.1

Energy balance closure at FLUXNET sites
journal, December 2002


Evaluation of 11 terrestrial carbon-nitrogen cycle models against observations from two temperate Free-Air CO 2 Enrichment studies
journal, January 2014

  • Zaehle, Sönke; Medlyn, Belinda E.; De Kauwe, Martin G.
  • New Phytologist, Vol. 202, Issue 3
  • DOI: 10.1111/nph.12697

Works referencing / citing this record:

Toward seamless hydrologic predictions across spatial scales
journal, January 2017

  • Samaniego, Luis; Kumar, Rohini; Thober, Stephan
  • Hydrology and Earth System Sciences, Vol. 21, Issue 9
  • DOI: 10.5194/hess-21-4323-2017

Evaluating the Interplay Between Biophysical Processes and Leaf Area Changes in Land Surface Models
journal, May 2018

  • Forzieri, Giovanni; Duveiller, Gregory; Georgievski, Goran
  • Journal of Advances in Modeling Earth Systems, Vol. 10, Issue 5
  • DOI: 10.1002/2018ms001284

Does predictability of fluxes vary between FLUXNET sites?
journal, January 2018

  • Haughton, Ned; Abramowitz, Gab; De Kauwe, Martin G.
  • Biogeosciences, Vol. 15, Issue 14
  • DOI: 10.5194/bg-15-4495-2018

Land surface models systematically overestimate the intensity, duration and magnitude of seasonal-scale evaporative droughts
journal, October 2016


Controls on evapotranspiration from jack pine forests in the Boreal Plains Ecozone
journal, December 2019

  • Nazarbakhsh, Mahtab; Ireson, Andrew M.; Barr, Alan G.
  • Hydrological Processes, Vol. 34, Issue 4
  • DOI: 10.1002/hyp.13674

Using phase lags to evaluate model biases in simulating the diurnal cycle of evapotranspiration: a case study in Luxembourg
journal, January 2019

  • Renner, Maik; Brenner, Claire; Mallick, Kaniska
  • Hydrology and Earth System Sciences, Vol. 23, Issue 1
  • DOI: 10.5194/hess-23-515-2019

Global Investigation of Soil Moisture and Latent Heat Flux Coupling Strength
journal, October 2018

  • Lei, Fangni; Crow, Wade T.; Holmes, Thomas R. H.
  • Water Resources Research, Vol. 54, Issue 10
  • DOI: 10.1029/2018wr023469

Application of a Regional Climate Model to Assess Changes in the Climatology of the Eastern United States and Cuba Associated With Historic Land Cover Change
journal, November 2019

  • Hostetler, S.; Reker, R.; Alder, J.
  • Journal of Geophysical Research: Atmospheres, Vol. 124, Issue 22
  • DOI: 10.1029/2019jd030965

On the predictability of land surface fluxes from meteorological variables
journal, January 2018

  • Haughton, Ned; Abramowitz, Gab; Pitman, Andy J.
  • Geoscientific Model Development, Vol. 11, Issue 1
  • DOI: 10.5194/gmd-11-195-2018

Derived Optimal Linear Combination Evapotranspiration (DOLCE): a global gridded synthesis ET estimate
journal, January 2018

  • Hobeichi, Sanaa; Abramowitz, Gab; Evans, Jason
  • Hydrology and Earth System Sciences, Vol. 22, Issue 2
  • DOI: 10.5194/hess-22-1317-2018

Comparing the Performance of the Maximum Entropy Production Model With a Land Surface Scheme in Simulating Surface Energy Fluxes
journal, March 2019

  • Alves, M.; Music, B.; Nadeau, D. F.
  • Journal of Geophysical Research: Atmospheres, Vol. 124, Issue 6
  • DOI: 10.1029/2018jd029282

Biases in Model-Simulated Surface Energy Fluxes During the Indian Monsoon Onset Period
journal, November 2018

  • Chakraborty, Tirthankar; Sarangi, Chandan; Krishnan, Mithun
  • Boundary-Layer Meteorology, Vol. 170, Issue 2
  • DOI: 10.1007/s10546-018-0395-x

Evaluating the Interplay Between Biophysical Processes and Leaf Area Changes in Land Surface Models
text, January 2018