A globally sampled high-resolution hand-labeled validation dataset for evaluating surface water extent maps

Journal Article · 23 September 2024 · Earth System Science Data (Online)

DOI: https://doi.org/10.5194/essd-16-4311-2024 · OSTI ID:2467419

Mukherjee, Rohit ^[1]; Policelli, Frederick ^[2]; Wang, Ruixue ^[3]; Arellano-Thompson, Elise ^[3]; Tellman, Beth ^[3]; Sharma, Prashanti ^[3]; Zhang, Zhijie ^[3]; Giezendanner, Jonathan ^[4]

---

1. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Univ. of Arizona, Tucson, AZ (United States)
2. NASA Goddard Space Flight Center (GSFC), Greenbelt, MD (United States)
3. Univ. of Arizona, Tucson, AZ (United States)
4. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

Effective monitoring of global water resources is increasingly critical due to climate change and population growth. Advancements in remote sensing technology, specifically in spatial, spectral, and temporal resolutions, are revolutionizing water resource monitoring, leading to more frequent and high-quality surface water extent maps using various techniques such as traditional image processing and machine learning algorithms. However, satellite imagery datasets contain trade-offs that result in inconsistencies in performance, such as disparities in measurement principles between optical (e.g., Sentinel-2) and radar (e.g., Sentinel-1) sensors and differences in spatial and spectral resolutions among optical sensors. Therefore, developing accurate and robust surface water mapping solutions requires independent validations from multiple datasets to identify potential biases within the imagery and algorithms. However, high-quality validation datasets are expensive to build, and few contain information on water resources. For this purpose, we introduce a globally sampled, high-spatial-resolution dataset labeled using 3 m PlanetScope imagery. Our surface water extent dataset comprises 100 images, each with a size of 1024×1024 pixels, which were sampled using a stratified random sampling strategy covering all 14 biomes. We highlighted urban and rural regions, lakes, and rivers, including braided rivers and coastal regions. We evaluated two surface water extent mapping methods using our dataset – Dynamic World, based on Sentinel-2, and the NASA IMPACT model, based on Sentinel-1. Dynamic World achieved a mean intersection over union (IoU) of 72.16 % and F1 score of 79.70 %, while the NASA IMPACT model had a mean IoU of 57.61 % and F1 score of 65.79 %. Performance varied substantially across biomes, highlighting the importance of evaluating models on diverse landscapes to assess their generalizability and robustness. Our dataset can be used to analyze satellite products and methods, providing insights into their advantages and drawbacks. Our dataset offers a unique tool for analyzing satellite products, aiding the development of more accurate and robust surface water monitoring solutions. The dataset can be accessed via https://doi.org/10.25739/03nt-4f29.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE; National Aeronautics and Space Administration (NASA)
• Grant/Contract Number: AC05-76RL01830
• OSTI ID: 2467419
• Report Number(s): PNNL-SA--204138
• Journal Information: Earth System Science Data (Online), Journal Name: Earth System Science Data (Online) Journal Issue: 9 Vol. 16; ISSN 1866-3516
• Publisher: Copernicus PublicationsCopyright Statement
• Country of Publication: United States
• Language: English