32 Search Results

The high performance computing (HPC) community is in a period of transition. The rise of AI/ML coupled with a changing landscape of resources deems portability a new metric of performance, and methods to move between on-premises and cloud environments and assess compatibility are paramount. Here we design and test a strategy for bridging the gap between traditional HPC and Kubernetes environments – first containerizing applications, providing automated orchestration to run studies, and packaging the setup with automated means to assess performance using low overhead eXtended Berkeley Packet Filter (eBPF) programs. We first assess different designs for eBPF collection, demonstrating amore » tradeoff between number of programs deployed on a node and overhead added. We develop 5 low overhead eBPF programs that combine with streaming ML models to assess CPU, futex, TCP, shared memory, and file access across four different builds of an HPC application for CPU and GPU. We use eBPF data to generate insights into the possible underlying etiology of scaling issues. We then assess compatibility of a well-known benchmark, HPCG, across matrices of micro-architectures and optimization levels (217 containers across 24 instance types and over 7500 runs). We provide to the community 30 applications to deploy in our automated setup and perform a scaling study from 4 to a maximum of 256 nodes for both CPU and GPU applications. Finally, we use our gained knowledge about performance to generate compatibility artifacts that are used by a newly developed Kubernetes controller to intelligently select instance type based on optimizing a figure of merit. Along with insights to scaling in this environment with a collection of applications and templates to work from, we provide an overall strategy for approaching HPC application deployment and image selection based on compatibility in cloud.« less

We evaluate cloud simulations using satellite simulators against multiple observational data sets. These simulators have been run within the Geophysical Fluid Dynamics Laboratory's Atmosphere Model version 4.0 (AM4.0), as well as an alternative configuration where a fully two‐moment Morrison‐Gettelman cloud microphysical parameterization with prognostic precipitation (MG2) is applied, denoted as AM4‐MG2. The modeled cloud spatial distributions, vertical profiles, phase partitioning, cloud‐to‐precipitation transitions, and radiative effects compare reasonably well with satellite observations. Model biases include the under‐prediction of total and low‐level clouds, especially optically thin/intermediate clouds with cloud optical depth of less than 23, but the over‐prediction of thick clouds, indicatingmore » “too few, too bright” biases. These biases counteract each other, and give rise to reasonable estimates of cloud radiative effects. The underestimate of low‐level clouds is associated with too early and too frequent drizzle/precipitation formation. The precipitation bias is improved in AM4‐MG2, where the autoconversion scheme initiates the precipitation more realistically. There also exist discrepancies between models and observations for midlevel and high‐level clouds. Additional biases include the underestimate of liquid cloud fraction and the overestimate of ice cloud fraction.« less

The accurate representation of interactions between clouds and planetary boundary layer (PBL) is a persistent challenge in climate models, critical for simulating surface energy budget. The emergence of kilometer-grid-scale global storm resolving models (GSRMs) offers the potential for enhanced details of PBL processes in these complex interactions. This study evaluates the representation of PBL-coupled and decoupled clouds in nine GSRM simulations against extensive ground-based observations by the Department of Energy Atmospheric Radiation Measurement (ARM) program, across six sites encompassing diverse regimes such as marine and continental environments in tropics and midlatitude. By differentiating coupling based on the relative positions betweenmore » cloud bases and PBL tops, our analysis focuses on the simulation of PBL height, cloud frequency, position and vertical extent. The GSRMs generally exhibit commendable agreements with observed cloud structures and PBL diurnal cycles across different ARM sites. In contrast to the relatively consistent representation of decoupled clouds, discrepancies exist between the simulated and the observed coupled clouds, particularly in areas of intense convection, for example, over tropical rainforests and mountainous regions. These biases are probably associated with the models' tendency to underestimate the boundary layer humidity and the frequency of coupled clouds within different ranges of PBL heights. This study underscores the importance for continuous improvements in the representation of boundary layer and convection within these global kilometer-grid-scale models.« less

In numerical atmospheric models that treat cloud and rain droplet populations as separate condensate categories, precipitation initiation in warm clouds is often represented by an autoconversion rate (Au), which is the rate of formation of new rain droplets through the collisions of cloud droplets. Being a function of the cloud droplet size distribution (DSD), the local Au is commonly parameterized as a function of DSD moments: cloud droplet number (n_c) and mass (q_c) concentrations. When applied in a large-scale model, the grid-mean Au must also include a correction, or enhancement factor, to account for the horizontal variability of the cloudmore » properties across the model grid. In this study, we evaluate the Au representation in the Energy Exascale Earth System Model version 2 (E3SMv2) climate model using large-eddy simulations (LES), which explicitly resolve cloud droplet spectra, and therefore the local Au, as well as its spatial variability. The analysis of an ensemble of warm low-level cloud cases shows that the E3SMv2 formulation represents the Au reasonably well compared to the horizontally averaged explicitly computed rate from LES. The agreement, however, comes from a combination of an underestimated E3SM-tuned local Au rate and an overestimated subgrid cloud variability enhancement factor. The latter bias is traced to neglecting the horizontal variability of n_c and its co-variability with q_c in parameterizing the grid-mean Au.« less

Aerosols and clouds are key components of the marine atmosphere, impacting the Earth’s radiative budget with a net cooling effect over the industrial era that counterbalances greenhouse gas warming, yet with an uncertain amplitude. Here we report recent advances in our understanding of how open ocean aerosol sources are modulated by ocean biogeochemistry and how they, in turn, shape cloud coverage and properties. We organize these findings in successive steps from ocean biogeochemical processes to particle formation by nucleation and sea spray emissions, further particle growth by condensation of gases, the potential to act as cloud condensation nuclei or icemore » nucleating particles, and finally, their effects on cloud formation, optical properties, and life cycle. We discuss how these processes may be impacted in a warming climate and the potential for ocean biogeochemistry—climate feedbacks through aerosols and clouds.« less

Abstract. Ice-nucleating particles (INPs) are an essential class of aerosols found worldwide that have far-reaching but poorly quantified climate feedback mechanisms through interaction with clouds and impacts on precipitation. These particles can have highly variable physicochemical properties in the atmosphere, and it is crucial to continuously monitor their long-term concentration relative to total ambient aerosol populations at a wide variety of sites to comprehensively understand aerosol–cloud interactions in the atmosphere. Hence, our study applied an in situ forced expansion cooling device to measure ambient INP concentrations and test its automated continuous measurements at atmospheric observatories, where complementary aerosol instruments aremore » heavily equipped. Using collocated aerosol size, number, and composition measurements from these sites, we analyzed the correlation between sources and abundance of INPs in different environments. Toward this aim, we have measured ground-level INP concentrations at two contrasting sites, one in the Southern Great Plains (SGP) region of the United States with a substantial terrestrially influenced aerosol population and one in the Eastern North Atlantic Ocean (ENA) region with a primarily marine-influenced aerosol population. These measurements examined INPs mainly formed through immersion freezing and were performed at a ≤ 12 min resolution and with a wide range of heterogeneous freezing temperatures (Ts above −31 °C) for at least 45 d at each site. The associated INP data analysis was conducted in a consistent manner. We also explored the additional offline characterization of ambient aerosol particle samples from both locations in comparison to in situ data. From our ENA data, on average, INP abundance ranges from ≈ 1 to ≈ 20 L−1 (−30 °C ≤ T ≤ −20 °C) during October–November 2020. Backward air mass trajectories reveal a strong marine influence at ENA with 75.7 % of air masses originating over the Atlantic Ocean and 96.6 % of air masses traveling over open water, but analysis of particle chemistry suggests an additional INP source besides maritime aerosols (e.g., sea spray aerosols) at ENA. In contrast, 90.8 % of air masses at the SGP location originated from the North American continent, and 96.1 % of the time, these air masses traveled over land. As a result, organic-rich SGP aerosols from terrestrial sources exhibited notably high INP abundance from ≈ 1 to ≈ 100 L−1 (−30 °C ≤ T ≤ −15 °C) during October–November 2019. The probability density function of aerosol surface area-scaled immersion freezing efficiency (ice nucleation active surface site density; ns) was assessed for selected freezing temperatures. While the INP concentrations measured at SGP are higher than those of ENA, the ns(T) values of SGP (≈ 105 to ≈ 107 m−2 for −30 °C ≤ T ≤ −15 °C) are reciprocally lower than ENA for approximately 2 orders of magnitude (≈ 107 to ≈ 109 m−2 for −30 °C ≤ T ≤ −15 °C). The observed difference in ns(T) mainly stems from varied available aerosol surface areas, Saer, from two sites (Saer,SGP > Saer,ENA). INP parameterizations were developed as a function of examined freezing temperatures from SGP and ENA for our study periods.« less

Abstract Riming is a key process of precipitation formation in ice‐containing clouds, but quantifying riming from observations is challenging, limiting our ability to evaluate the riming process in numerical weather models. One challenge for radar observations is that riming changes both the physical properties (mass, area cross‐section) and scattering properties of ice particles. These changes need to be implemented consistently as a function of riming in radar forward operators, which are required for retrievals and model evaluation in observation space. In this study, mass–size, cross‐section area–size, and backscattering cross‐section relations are developed as a function of the normalized rime massmore » for aggregates composed of various monomer types (columns, dendrites, needles, plates, and rosettes). The proposed framework allows us to simulate scattering properties of aggregated ice particles consistently as a function of riming in retrievals and radar forward operators. The parameterizations are developed from a large data set of simulated rimed aggregates of different sizes and monomer crystal types. The backscattering cross‐section parameterization (the “riming‐dependent parameterization”) is evaluated for radar frequencies of 35.6 and 94.0 GHz and is based on the Self‐Similar Rayleigh–Gans approximation (SSRGA), which is increasingly used to calculate microwave scattering of ice crystals and snowflakes. Compared with parameterizations from the literature that do not consider riming, the riming‐dependent parameterization leads to significantly smaller biases in terms of backscattering cross‐section. When using the particle masses and scattering properties of the individual particles simulated by the aggregation and riming model as a reference, the bias of our parameterization is below 1 dB when integrating over an exponential particle size distribution with sizes from 0.1–10 mm.« less

Abstract The effect of aerosols on the properties of clouds is a large source of uncertainty in predictions of weather and climate. These aerosol‐cloud interactions depend critically on the ability of aerosol particles to form cloud droplets. A challenge in modeling aerosol‐cloud interactions is the representation of interactions between turbulence and cloud microphysics. Turbulent mixing leads to small‐scale fluctuations in water vapor and temperature that are unresolved in large‐scale atmospheric models. To quantify the impact of turbulent fluctuations on cloud condensation nuclei (CCN) activation, we used a high‐resolution Large Eddy Simulation of a convective cloud chamber to drive particle‐based cloudmore » microphysics simulations. We show small‐scale fluctuations strongly impact CCN activity. Once activated, the relatively long timescales of evaporation compared to fluctuations causes droplets to persist in subsaturated regions, which further increases droplet concentrations.« less

The proliferation of low-cost sensors and industrial data solutions has continued to push the frontier of manufacturing technology. Machine learning and other advanced statistical techniques stand to provide tremendous advantages in production capabilities, optimization, monitoring, and efficiency. The tremendous volume of data gathered continues to grow, and the methods for storing the data are critical underpinnings for advancing manufacturing technology. This work aims to investigate the ramifications and design tradeoffs within a decoupled architecture of two prominent database management systems (DBMS): sql and NoSQL. A representative comparison is carried out with Amazon Web Services (AWS) DynamoDB and AWS Aurora MySQL.more » The technologies and accompanying design constraints are investigated, and a side-by-side comparison is carried out through high-fidelity industrial data simulated load tests using metrics from a major US manufacturer. The results support the use of simulated client load testing for comparing the latency of database management systems as a system scales up from the prototype stage into production. As a result of complex query support, MySQL is favored for higher-order insights, while NoSQL can reduce system latency for known access patterns at the expense of integrated query flexibility. Here, by reviewing this work, a manufacturer can observe that the use of high-fidelity load testing can reveal tradeoffs in IoTfM write/ingestion performance in terms of latency that are not observable through prototype-scale testing of commercially available cloud DB solutions.« less

Ubiquitous shallow cumulus clouds are associated with complex variability in surface solar irradiance (SSI). Aerosol embedded in the cloud field typically has a much smaller overall radiative effect, but can significantly perturb the shape of the SSI probability density function (PDF). These perturbations have important implications for several applications that utilize SSI, but are poorly quantified and are the subject of this study. Multiple cases of shallow cumulus cloud fields with embedded aerosol are simulated at the Southern Great Plains Atmospheric Observatory using large eddy simulation (LES). The LES-derived cloud and aerosol fields are then ingested into Monte Carlo three-dimensionalmore » (3D) radiative transfer to simulate SSI. We find a variety of perturbations to the SSI PDF that depend on aerosol presence and optical properties. The processes leading to these perturbations include extinction of the direct beam that often increases from the clear-sky region toward cloud edge due to aerosol hygroscopic growth, and scattering of radiation by aerosol into cloud shadows. The ability to predict the SSI PDF in the presence of aerosol is assessed by adding three representative aerosol optical properties into an existing machine learning framework. We show that machine learning accurately predicts the SSI PDF across a wide range of conditions with negligible computational expense. Importance metrics reveal the relatively high influence of aerosol optical properties in making the predictions. These new findings highlight the important role that aerosol plays in SSI variability for highly 3D cloud-aerosol environments and provides a computationally efficient route forward for its simulation.« less