18 Search Results

Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics-based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop Earth-system models (ESMs) capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes over long timescales. Building trust in ESMs is a much more difficult problem than for weather forecast models, not leastmore » because the model must represent the alternate (e.g., future or paleoclimatic) coupled states of the system for which there are no direct observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML-based models, demonstrating the credibility of ML-based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML-based ESMs to strengthen their credibility and promote their wider use.« less

Global and national efforts to deliver high-quality nuclear data to users have a wide-ranging impact, affecting applications in national security, reactor operations, basic science, medicine, and more. Cross-section evaluation is a major part of this effort, combining theory and experimentation to produce recommended values and uncertainties for reaction probabilities. Resonance region evaluation is a specialized type of nuclear data evaluation that can require significant manual effort and months of time from expert scientists. Here, in this article, nonconvex, nonlinear optimization methods are combined with concepts of inferential statistics to infer a resonance model from experimental data in an automated mannermore » that is not dependent on prior evaluation(s). This methodology aims to enhance the workflow of a resonance evaluator by minimizing time, effort, and the potential for bias from prior assumptions, while enhancing reproducibility and documentation, thereby addressing well-known challenges in the field.« less

Weather and climate forecasting, using climate models, have become essential tools and life-savers in the West African region; in spite of the fact that climate models do not fully comply with attributes of forecast qualities—RASAP: reliability, association, skill, accuracy, and precision. The objective of this paper is to quantitatively evaluate, in comparison to CRU and ERA5 datasets, the RASAP compliance-level of the weather@home2 modeling system (w@h2). Findings from some statistical evaluations show that, to a moderately significant extent, w@h2 model provides useful information during the monsoon seasons; skills to capture the Little Dry Season over the Guinea zone; predictive skillsmore » for the onset season; ability to reproduce all the annual characteristics of the surface maximum air temperature over the region; as well as skill to detect heat waves that usually ravage West Africa during the boreal spring. The model displays traces of attributes that are needed for seasonal climate predictions and applications. Deficiencies in the quantitative reproducibility point to the facts that the model does provide a reliability akin to that of regional climate models. This paper further furnishes a prospective user with information on whether the model might be “useful or not” for a particular application.« less

Estimating soil organic carbon (SOC) stocks in agricultural fields is essential for environmental and agronomic research, management, and policy. Stratified sampling is a classic strategy for estimating mean soil properties, and has recently been codified in SOC monitoring protocols. However, for the specific task of estimating the SOC stock of an agricultural field, concrete guidance is needed for which covariates to stratify on and how much stratification can improve estimation efficiency. It is also unknown how stratified sampling of SOC stocks compares to modern alternatives, notably doubly balanced sampling. To address these gaps, we collected high-density (average of 7 samplesmore » ha^-1) and deep (average of 75 cm) measurements of SOC stocks at eight commercial fields under maize-soybean production in two US Midwestern states. We combined these measurements with a Bayesian geostatistical model to evaluate stratified and balanced sampling strategies that use a set of readily-available geographic, topographic, spectroscopic, and soil survey data. We examined the number of samples needed to achieve a given level of SOC stock estimation accuracy. While stratified sampling using these variables enables an average sample size reduction of 17% (95% CI, 11% to 23%) compared to simple random sampling, doubly balanced sampling is consistently more efficient, reducing sample sizes by 32% (95% CI, 25% to 37%). The data most important to these efficiency gains are a remotely-sensed SOC index, SSURGO estimates of SOC stocks, and the topographic wetness index. We conclude that in order to meet the urgent challenge of climate change, SOC stocks in agricultural fields could be more efficiently estimated by taking advantage of this readily-available data, especially with doubly balanced sampling.« less

While there are numerous performance metrics that have been developed for the evaluation of distracted driving prevention programs, there is little information on how to select them depending on the requirements and/or objectives of the study. Here, this paper describes a systematic literature review that was conducted on the metrics for evaluating distracted driving countermeasures in order to bridge this research gap. A summary of the evaluation metrics used for the existing distracted driving countermeasures was provided. Guidance for choosing an evaluation measure was provided by analyzing the metrics from the perspectives of functionality, spatial-temporal dimension, and equity. Three examplesmore » of distracted driving countermeasure evaluation processes were thoroughly reviewed in order to offer insight into metric selection and measurement. This paper contributes to the body of knowledge by discussing the implications for policy on how to enhance the thoroughness and accuracy of the evaluation of distracted driving countermeasures. Analysis from multiple angles, the development of data collection tools and direct behavior indicators, taking into account temporal dimensions, and equity considerations, are proposed.« less

Abstract. Seasonal snow has crucial impacts on climate, ecosystems, and humans, but it is vulnerable to global warming. The land component (ELM) of the Energy Exascale Earth System Model (E3SM) mechanistically simulates snow processes from accumulation, canopy interception, compaction, and snow aging to melt. Although high-quality field measurements, remote sensing snow products, and data assimilation products with high spatio-temporal resolution are available, there has been no systematic evaluation of the snow properties and phenology in ELM. This study comprehensively evaluates ELM snow simulations over the western United States at 0.125∘ resolution during 2001–2019 using the Snow Telemetry (SNOTEL) in situ networks, MODIS remotemore » sensing products (i.e., MCD43 surface albedo product), the spatially and temporally complete (STC) snow-covered area and grain size (MODSCAG) and MODIS dust and radiative forcing in snow (MODDRFS) products (STC-MODSCAG/STC-MODDRFS), and the snow property inversion from remote sensing (SPIReS) product and two data assimilation products of snow water equivalent and snow depth – i.e., University of Arizona (UA) and SNOw Data Assimilation System (SNODAS). Overall the ELM simulations are consistent with the benchmarking datasets and reproduce the spatio-temporal patterns, interannual variability, and elevation gradients for different snow properties including snow cover fraction (fsno), surface albedo (αsur) over snow cover regions, snow water equivalent (SWE), and snow depth (Dsno). However, there are large biases of fsno with dense forest cover and αsur in the Rocky Mountains and Sierra Nevada in winter, compared to the MODIS products. There are large discrepancies of snow albedo, snow grain size, and light-absorbing particle-induced snow albedo reduction between ELM and the MODIS products, attributed to uncertainties in the aerosol forcing data, snow aging processes in ELM, and remote sensing retrievals. Against UA and SNODAS, ELM has a mean bias of −20.7 mm (−35.9 %) and −20.4 mm (−35.5 %), respectively, for spring, and −13.8 mm (−27.8 %) and −10.2 mm (−22.2 %), respectively, for winter. ELM shows a relatively high correlation with SNOTEL SWE, with mean correlation coefficients of 0.69 but negative mean biases of −122.7 mm. Compared to the snow phenology of STC-MODSCAG and SPIReS, ELM shows delayed snow accumulation onset dates by 17.3 and 12.4 d, earlier snow end dates by 35.5 and 26.8 d, and shorter snow durations by 52.9 and 39.5 d, respectively. This study underscores the need for diagnosing model biases and improving ELM representations of snow properties and snow phenology in mountainous areas for more credible simulation and future projection of mountain snowpack.« less

We report connected and automated vehicles (CAVs) can bring safety, mobility, and energy benefits to transportation systems. Ideally, CAV applications would be fully evaluated and validated prior to real-world implementation. However, many technical challenges in both software and hardware hinder the process. To comprehensively evaluate all aspects of CAV applications, an integrated evaluation environment is needed with various simulation tools from different domains. In the current literature, there lacks a well-developed interface to enable multi-resolution simulation of vehicle, traffic, virtual environment, and hardware-in-the-loop (HIL) simulation. In this work, a modular and flexible interface is developed to enable multi-resolution vehicle andmore » traffic co-simulation for CAV applications. This interface is built upon Oak Ridge National Laboratory’s (ORNL’s) Real-Sim approach, can support various simulation tools and real-time X-in-the-loop (XIL) simulations, and is based on network communication protocols. The network communication delays are analyzed for simultaneous connection of up to 20 simulators. Then, two example applications are studied to demonstrate the potential usage of the Real-Sim interface: (1) a centralized merging scenario where the dynamics of two ego vehicles are emulated with detailed vehicle and powertrain dynamics models and (2) a signal-controller-in-the-loop (SCIL) evaluation where one intersection of the simulated traffic scenario is controlled by a physical signal controller. This Real-Sim interface is a tool that allows researchers to bring real, tangible hardware and software into simulated environments to comprehensively evaluate and support a wide variety of CAV applications.« less

The Regional Atmosphere (RA) configuration of the Met Office Unified Model currently requires different cloud fraction parametrizations (CFP) for tropical and midlatitude simulations. To explore the scope for unification of these two RA configurations, this paper presents a detailed evaluation of simulations over tropical, midlatitude and arctic domains, with two different diagnostic CFPs, a prognostic CFP, and no CFP at all. Furthermore, a novel, hybrid approach was used that treats liquid cloud diagnostically and ice cloud prognostically. Using observations from three U.S. Department of Energy Atmospheric Radiation Measurement supersites, it is shown that none of these CFPs stands out asmore » superior over all domains. Over the frequently overcast Arctic, the all-or-nothing approach best captures the cloud radiative properties. Conversely, CFPs are of benefit in regions with frequent partial cloudiness, such as the midlatitudes and the tropics. However, their improved cloud radiative properties often hide an error compensation. All models underestimate overcast, low-base cloud with small water paths in convective environments. In addition, mid-latitude overcast, low-base, optically thick clouds in the morning, possibly associated with overnight convection, are frequently too broken. Diagnostic schemes compensate for these errors by producing spurious, scattered afternoon cloud, which could be due to a correct cloud response to too eager convective initiation. Winter clouds over the midlatitudes are improved when liquid cloud is represented diagnostically with a bimodal saturation-departure PDF, without error compensation. While it is difficult to unify the RA across the globe around a single CFP scheme, the newly proposed Hybrid scheme performs reasonably well for cloud cover across all regions. It also exhibits SW biases that are smaller than most other configurations and is less affected by excessive liquid water paths and compensating errors than fully diagnostic schemes. Surface precipitation is fairly insensitive to the CFP in the simulations shown here.« less

In 2017, the National Institute of Public Health in Cambodia collaborated with the U.S. Centers for Disease Control and Prevention to provide management and leadership training for 20 managers and senior staff from 10 health centers. We conducted a mixed methods evaluation of the program's outcomes and impact on the graduates and health centers. From June 2018 (baseline) to January 2019 (endpoint), we collected data from a competency assessment, observational visits, and interviews. From baseline to endpoint, all 20 participants reported increased competence in seven management areas. Comparing baseline and endpoint observational visits, we found improvements in leadership and governance,more » health workforce, water, sanitation, and hygiene, and health centers' use of medical products and technologies. When evaluating the improvements made by participants against the World Health Organization's key components of a well-functioning health system, the program positively contributed toward building four of the six components—leadership and governance, health information systems, human resources for health, and service delivery. While these findings are specific to the context of Cambodian health centers, we hope this evaluation adds to the growing body of research around the impact of skilled public health management on health systems.« less

To meet the exascale I/O requirements for the High-Performance Computing (HPC), a new I/O subsystem, Burst Buffer, based on solid state drives (SSD), has been developed. However, the diverse HPC workloads and the bursty I/O pattern cause severe data fragmentation that requires costly garbage collection (GC) and increases the number of bytes written to the SSD. To address this data fragmentation challenge, a new multi-stream feature has been developed for SSDs. In this work, we develop an I/O Separation scheme called BIOS to leverage this multi-stream feature to group the I/O streams based on the user IDs. We propose amore » stream-aware scheduling policy based on burst buffer pools in the workload manager, and integrate the BIOS with the workload manager to optimize the I/O separation scheme in burst buffer. We evaluate the proposed framework with a burst buffer I/O traces from Cori Supercomputer including a diverse set of applications. Experimental results show that the BIOS could improve the performance by 1.44x on average and reduce the Write Amplification Factor (WAF) by up to 1.20x. Finally, these demonstrate the potential benefits of the I/O separation scheme for solid state storage systems.« less