Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm
Abstract
Abstract Droplet-level interactions in clouds are often parameterized by a modified gamma fitted to a “global” droplet size distribution. Do “local” droplet size distributions of relevance to microphysical processes look like these average distributions? This paper describes an algorithm to search and classify characteristic size distributions within a cloud. The approach combines hypothesis testing, specifically, the Kolmogorov–Smirnov (KS) test, and a widely used class of machine learning algorithms for identifying clusters of samples with similar properties: density-based spatial clustering of applications with noise (DBSCAN) is used as the specific example for illustration. The two-sample KS test does not presume any specific distribution, is parameter free, and avoids biases from binning. Importantly, the number of clusters is not an input parameter of the DBSCAN-type algorithms but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the KS test results, and hence spatial correlation is not required for a cluster. The method is explored using data obtained from the Holographic Detector for Clouds (HOLODEC) deployed during the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The algorithm identifies evidence of the existence of clusters of nearly identical local size distributions. Itmore »
- Authors:
-
- a Michigan Technological University, Houghton, Michigan
- a Michigan Technological University, Houghton, Michigan, b College of Charleston, Charleston, South Carolina
- c Brigham Young University, Provo, Utah
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Atmospheric Radiation Measurement (ARM) Data Center
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER); National Science Foundation (NSF)
- OSTI Identifier:
- 1887478
- Alternate Identifier(s):
- OSTI ID: 1884127
- Grant/Contract Number:
- SC0020053; AGS-2001490
- Resource Type:
- Published Article
- Journal Name:
- Artificial Intelligence for the Earth Systems
- Additional Journal Information:
- Journal Name: Artificial Intelligence for the Earth Systems Journal Volume: 1 Journal Issue: 3; Journal ID: ISSN 2769-7525
- Publisher:
- American Meteorological Society
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 54 ENVIRONMENTAL SCIENCES; Cloud droplets; Cloud microphysics; In situ atmospheric observations; Machine learning; Microscale processes/variability
Citation Formats
Allwayin, Nithin, Larsen, Michael L., Shaw, Alexander G., and Shaw, Raymond A. Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm. United States: N. p., 2022.
Web. doi:10.1175/AIES-D-22-0003.1.
Allwayin, Nithin, Larsen, Michael L., Shaw, Alexander G., & Shaw, Raymond A. Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm. United States. https://doi.org/10.1175/AIES-D-22-0003.1
Allwayin, Nithin, Larsen, Michael L., Shaw, Alexander G., and Shaw, Raymond A. Fri .
"Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm". United States. https://doi.org/10.1175/AIES-D-22-0003.1.
@article{osti_1887478,
title = {Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm},
author = {Allwayin, Nithin and Larsen, Michael L. and Shaw, Alexander G. and Shaw, Raymond A.},
abstractNote = {Abstract Droplet-level interactions in clouds are often parameterized by a modified gamma fitted to a “global” droplet size distribution. Do “local” droplet size distributions of relevance to microphysical processes look like these average distributions? This paper describes an algorithm to search and classify characteristic size distributions within a cloud. The approach combines hypothesis testing, specifically, the Kolmogorov–Smirnov (KS) test, and a widely used class of machine learning algorithms for identifying clusters of samples with similar properties: density-based spatial clustering of applications with noise (DBSCAN) is used as the specific example for illustration. The two-sample KS test does not presume any specific distribution, is parameter free, and avoids biases from binning. Importantly, the number of clusters is not an input parameter of the DBSCAN-type algorithms but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the KS test results, and hence spatial correlation is not required for a cluster. The method is explored using data obtained from the Holographic Detector for Clouds (HOLODEC) deployed during the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The algorithm identifies evidence of the existence of clusters of nearly identical local size distributions. It is found that cloud segments have as few as one and as many as seven characteristic size distributions. To validate the algorithm’s robustness, it is tested on a synthetic dataset and successfully identifies the predefined distributions at plausible noise levels. The algorithm is general and is expected to be useful in other applications, such as remote sensing of cloud and rain properties. Significance Statement A typical cloud can have billions of drops spread over tens or hundreds of kilometers in space. Keeping track of the sizes, positions, and interactions of all of these droplets is impractical, and, as such, information about the relative abundance of large and small drops is typically quantified with a “size distribution.” Droplets in a cloud interact locally, however, so this work is motivated by the question of whether the cloud droplet size distribution is different in different parts of a cloud. A new method, based on hypothesis testing and machine learning, determines how many different size distributions are contained in a given cloud. This is important because the size distribution describes processes such as cloud droplet growth and light transmission through clouds.},
doi = {10.1175/AIES-D-22-0003.1},
journal = {Artificial Intelligence for the Earth Systems},
number = 3,
volume = 1,
place = {United States},
year = {Fri Jul 01 00:00:00 EDT 2022},
month = {Fri Jul 01 00:00:00 EDT 2022}
}
https://doi.org/10.1175/AIES-D-22-0003.1
Works referenced in this record:
Disdrometer Network Observations of Finescale Spatial–Temporal Clustering in Rain
journal, March 2015
- Jameson, A. R.; Larsen, M. L.; Kostinski, A. B.
- Journal of the Atmospheric Sciences, Vol. 72, Issue 4
The Importance of the Shape of Cloud Droplet Size Distributions in Shallow Cumulus Clouds. Part II: Bulk Microphysics Simulations
journal, January 2017
- Igel, Adele L.; van den Heever, Susan C.
- Journal of the Atmospheric Sciences, Vol. 74, Issue 1
On the Detection of Statistical Heterogeneity in Rain Measurements
journal, July 2018
- Jameson, A. R.; Larsen, M. L.; Kostinski, A. B.
- Journal of Atmospheric and Oceanic Technology, Vol. 35, Issue 7
Holographic Observations of Centimeter-Scale Nonuniformities within Marine Stratocumulus Clouds
journal, January 2020
- Glienke, Susanne; Kostinski, Alexander B.; Shaw, Raymond A.
- Journal of the Atmospheric Sciences, Vol. 77, Issue 2
Design of an in-line, digital holographic imaging system for airborne measurement of clouds
journal, January 2011
- Spuler, Scott M.; Fugal, Jacob
- Applied Optics, Vol. 50, Issue 10
Holographic measurements of inhomogeneous cloud mixing at the centimeter scale
journal, October 2015
- Beals, Matthew J.; Fugal, Jacob P.; Shaw, Raymond A.
- Science, Vol. 350, Issue 6256
Subgrid variations of the cloud water and droplet number concentration over the tropical ocean: satellite observations and implications for warm rain simulations in climate models
journal, January 2019
- Zhang, Zhibo; Song, Hua; Ma, Po-Lun
- Atmospheric Chemistry and Physics, Vol. 19, Issue 2
OPTICS: ordering points to identify the clustering structure
journal, June 1999
- Ankerst, Mihael; Breunig, Markus M.; Kriegel, Hans-Peter
- ACM SIGMOD Record, Vol. 28, Issue 2
Observations and Analysis of Uncorrelated Rain
journal, November 2005
- Larsen, Michael L.; Kostinski, Alexander B.; Tokay, Ali
- Journal of the Atmospheric Sciences, Vol. 62, Issue 11
Airborne digital holographic system for cloud particle measurements
journal, January 2004
- Fugal, Jacob P.; Shaw, Raymond A.; Saw, Ewe Wei
- Applied Optics, Vol. 43, Issue 32
Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA)
journal, February 2022
- Wang, Jian; Wood, Rob; Jensen, Michael P.
- Bulletin of the American Meteorological Society, Vol. 103, Issue 2
Cloud particle size distributions measured with an airborne digital in-line holographic instrument
journal, January 2009
- Fugal, J. P.; Shaw, R. A.
- Atmospheric Measurement Techniques, Vol. 2, Issue 1
On the Variability of Drop Size Distributions over Areas
journal, March 2015
- Jameson, A. R.; Larsen, M. L.; Kostinski, A. B.
- Journal of the Atmospheric Sciences, Vol. 72, Issue 4
Sensitivity of the Earth's radiation budget to changes in low clouds
journal, January 1990
- Slingo, A.
- Nature, Vol. 343, Issue 6253
Cloud Droplet Size Distributions in Low-Level Stratiform Clouds
journal, January 2000
- Miles, Natasha L.; Verlinde, Johannes; Clothiaux, Eugene E.
- Journal of the Atmospheric Sciences, Vol. 57, Issue 2
Experimental Quantification of the Sampling Uncertainty Associated with Measurements from PARSIVEL Disdrometers
journal, June 2011
- Jaffrain, Joël; Berne, Alexis
- Journal of Hydrometeorology, Vol. 12, Issue 3
Practical methods for automated reconstruction and characterization of particles in digital in-line holograms
journal, May 2009
- Fugal, Jacob P.; Schulz, Timothy J.; Shaw, Raymond A.
- Measurement Science and Technology, Vol. 20, Issue 7
Fluctuation Properties of Precipitation. Part VI: Observations of Hyperfine Clustering and Drop Size Distribution Structures in Three-Dimensional Rain
journal, February 2000
- Jameson, A. R.; Kostinski, A. B.
- Journal of the Atmospheric Sciences, Vol. 57, Issue 3
Fine-Scale Droplet Clustering in Atmospheric Clouds: 3D Radial Distribution Function from Airborne Digital Holography
journal, November 2018
- Larsen, Michael L.; Shaw, Raymond A.; Kostinski, Alexander B.
- Physical Review Letters, Vol. 121, Issue 20
Sampling variability effects in drop-resolving disdrometer observations: SAMPLING VARIABILITY IN DISDROMETER DATA
journal, October 2016
- Larsen, Michael L.; O'Dell, Katelyn A.
- Journal of Geophysical Research: Atmospheres, Vol. 121, Issue 19