Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems

Liu, Licheng; Zhou, Wang; Guan, Kaiyu; Peng, Bin; Xu, Shaoming; Tang, Jinyun; Zhu, Qing; Till, Jessica; Jia, Xiaowei; Jiang, Chongya; Wang, Sheng; Qin, Ziqi; Kong, Hui; Grant, Robert; Mezbahuddin, Symon; Kumar, Vipin; Jin, Zhenong

doi:10.1038/s41467-023-43860-5

Title: Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems

Journal Article · 07 January 2024 · Nature Communications

DOI: https://doi.org/10.1038/s41467-023-43860-5 · OSTI ID:2279084

; Zhou, Wang; Guan, Kaiyu;

; Xu, Shaoming;

;

; Till, Jessica; Jia, Xiaowei; Jiang, Chongya;

; Qin, Ziqi; Kong, Hui;

;

; Kumar, Vipin;

Abstract Accurate and cost-effective quantification of the carbon cycle for agroecosystems at decision-relevant scales is critical to mitigating climate change and ensuring sustainable food production. However, conventional process-based or data-driven modeling approaches alone have large prediction uncertainties due to the complex biogeochemical processes to model and the lack of observations to constrain many key state and flux variables. Here we propose a Knowledge-Guided Machine Learning (KGML) framework that addresses the above challenges by integrating knowledge embedded in a process-based model, high-resolution remote sensing observations, and machine learning (ML) techniques. Using the U.S. Corn Belt as a testbed, we demonstrate that KGML can outperform conventional process-based and black-box ML models in quantifying carbon cycle dynamics. Our high-resolution approach quantitatively reveals 86% more spatial detail of soil organic carbon changes than conventional coarse-resolution approaches. Moreover, we outline a protocol for improving KGML via various paths, which can be generalized to develop hybrid models to better predict complex earth system dynamics.

View Journal Article

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: National Aeronautics and Space Administration (NASA); National Science Foundation (NSF); USDOE; USDOE Advanced Research Projects Agency - Energy (ARPA-E); USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office

Grant/Contract Number:: AC02-05CH11231; AR0001382

OSTI ID:: 2279084

Journal Information:: Nature Communications, Journal Name: Nature Communications Journal Issue: 1 Vol. 15; ISSN 2041-1723

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (66)

Hybrid Modeling of Evapotranspiration: Inferring Stomatal and Aerodynamic Resistances Using Combined Physics-Based and Machine Learning ElGhawi, Reda; Kraft, Basil; Reimers, Christian ESS Open Archive https://doi.org/10.1002/essoar.10512258.1	preprint	August 2022
Quantifying soil carbon change in a long‐term tillage and crop rotation study across Iowa landscapes Al‐Kaisi, Mahdi M.; Kwaw‐Mensah, David Soil Science Society of America Journal, Vol. 84, Issue 1 https://doi.org/10.1002/saj2.20003	journal	January 2020
Principles of Terrestrial Ecosystem Ecology Chapin, F. Stuart; Matson, Pamela A.; Vitousek, Peter M. https://doi.org/10.1007/978-1-4419-9504-9	book	September 2011
The global carbon cycle and the biosphere Reichle, David E. The Global Carbon Cycle and Climate Change: Scaling Ecological Energetics from Organism to the Biosphere, p. 183-208 https://doi.org/10.1016/B978-0-12-820244-9.00010-X	book	January 2020
Examining strategies to improve the carbon balance of corn/soybean agriculture using eddy covariance and mass balance techniques Baker, J. M.; Griffis, T. J. Agricultural and Forest Meteorology, Vol. 128, Issue 3-4 https://doi.org/10.1016/j.agrformet.2004.11.005	journal	February 2005
Quantifying carbon budget, crop yields and their responses to environmental variability using the ecosys model for U.S. Midwestern agroecosystems Zhou, Wang; Guan, Kaiyu; Peng, Bin Agricultural and Forest Meteorology, Vol. 307 https://doi.org/10.1016/j.agrformet.2021.108521	journal	September 2021
Two for one: Partitioning CO2 fluxes and understanding the relationship between solar-induced chlorophyll fluorescence and gross primary productivity using machine learning Zhan, Weiwei; Yang, Xi; Ryu, Youngryel Agricultural and Forest Meteorology, Vol. 321 https://doi.org/10.1016/j.agrformet.2022.108980	journal	June 2022
Distinct driving mechanisms of non-growing season N2O emissions call for spatial-specific mitigation strategies in the US Midwest Yang, Yufeng; Liu, Licheng; Zhou, Wang Agricultural and Forest Meteorology, Vol. 324 https://doi.org/10.1016/j.agrformet.2022.109108	journal	September 2022
Improving predictions of evapotranspiration by integrating multi-source observations and land surface model He, Xinlei; Liu, Shaomin; Xu, Tongren Agricultural Water Management, Vol. 272 https://doi.org/10.1016/j.agwat.2022.107827	journal	October 2022
On the spatial variability and influencing factors of soil organic carbon and total nitrogen stocks in a desert oasis ecotone of northwestern China Cao, Qiqi; Li, Junran; Wang, Guan CATENA, Vol. 206 https://doi.org/10.1016/j.catena.2021.105533	journal	November 2021
A scalable framework for quantifying field-level agricultural carbon outcomes Guan, Kaiyu; Jin, Zhenong; Peng, Bin Earth-Science Reviews, Vol. 243 https://doi.org/10.1016/j.earscirev.2023.104462	journal	August 2023
The implication of input data aggregation on up-scaling soil organic carbon changes Grosz, Balázs; Dechow, Rene; Gebbert, Sören Environmental Modelling & Software, Vol. 96 https://doi.org/10.1016/j.envsoft.2017.06.046	journal	October 2017
Assessing the impacts of cover crops on maize and soybean yield in the U.S. Midwestern agroecosystems Qin, Ziqi; Guan, Kaiyu; Zhou, Wang Field Crops Research, Vol. 273 https://doi.org/10.1016/j.fcr.2021.108264	journal	November 2021
Assessing the impacts of pre-growing-season weather conditions on soil nitrogen dynamics and corn productivity in the U.S. Midwest Li, Ziyi; Guan, Kaiyu; Zhou, Wang Field Crops Research, Vol. 284 https://doi.org/10.1016/j.fcr.2022.108563	journal	August 2022
Valid inferences about soil carbon in heterogeneous landscapes Stanley, Paige; Spertus, Jacob; Chiartas, Jessica Geoderma, Vol. 430 https://doi.org/10.1016/j.geoderma.2022.116323	journal	February 2023
Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data Wang, Sheng; Guan, Kaiyu; Zhang, Chenhui Remote Sensing of Environment, Vol. 285 https://doi.org/10.1016/j.rse.2022.113366	journal	February 2023
Recent Rapid Increase of Cover Crop Adoption Across the U.S. Midwest Detected by Fusing Multi‐Source Satellite Data Zhou, Qu; Guan, Kaiyu; Wang, Sheng Geophysical Research Letters, Vol. 49, Issue 22 https://doi.org/10.1029/2022GL100249	journal	November 2022
Combining Remotely Sensed Evapotranspiration and an Agroecosystem Model to Estimate Center‐Pivot Irrigation Water Use at High Spatio‐Temporal Resolution Zhang, Jingwen; Guan, Kaiyu; Zhou, Wang Water Resources Research, Vol. 59, Issue 3 https://doi.org/10.1029/2022WR032967	journal	February 2023
Uncertainty in simulating wheat yields under climate change Asseng, S.; Ewert, F.; Rosenzweig, C. Nature Climate Change, Vol. 3, Issue 9 https://doi.org/10.1038/nclimate1916	journal	June 2013
Expansion of high-latitude deciduous forests driven by interactions between climate warming and fire Mekonnen, Zelalem A.; Riley, William J.; Randerson, James T. Nature Plants, Vol. 5, Issue 9 https://doi.org/10.1038/s41477-019-0495-8	journal	August 2019
Deep learning and process understanding for data-driven Earth system science Reichstein, Markus; Camps-Valls, Gustau; Stevens, Bjorn Nature, Vol. 566, Issue 7743 https://doi.org/10.1038/s41586-019-0912-1	journal	February 2019
Mapping twenty years of corn and soybean across the US Midwest using the Landsat archive Wang, Sherrie; Di Tommaso, Stefania; Deines, Jillian M. Scientific Data, Vol. 7, Issue 1 https://doi.org/10.1038/s41597-020-00646-4	journal	September 2020
The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data Pastorello, Gilberto; Trotta, Carlo; Canfora, Eleonora Scientific Data, Vol. 7, Issue 1 https://doi.org/10.1038/s41597-020-0534-3	journal	July 2020
Soil carbon science for policy and practice Bradford, Mark A.; Carey, Chelsea J.; Atwood, Lesley Nature Sustainability, Vol. 2, Issue 12 https://doi.org/10.1038/s41893-019-0431-y	journal	November 2019
The role of soil carbon in natural climate solutions Bossio, D. A.; Cook-Patton, S. C.; Ellis, P. W. Nature Sustainability, Vol. 3, Issue 5 https://doi.org/10.1038/s41893-020-0491-z	journal	March 2020
Towards neural Earth system modelling by integrating artificial intelligence in Earth system science Irrgang, Christopher; Boers, Niklas; Sonnewald, Maike Nature Machine Intelligence, Vol. 3, Issue 8 https://doi.org/10.1038/s42256-021-00374-3	journal	August 2021
The digital revolution of Earth-system science Bauer, Peter; Dueben, Peter D.; Hoefler, Torsten Nature Computational Science, Vol. 1, Issue 2 https://doi.org/10.1038/s43588-021-00023-0	journal	February 2021
Deep learning to represent subgrid processes in climate models Rasp, Stephan; Pritchard, Michael S.; Gentine, Pierre Proceedings of the National Academy of Sciences, Vol. 115, Issue 39 https://doi.org/10.1073/pnas.1810286115	journal	September 2018
Enforcing Analytic Constraints in Neural Networks Emulating Physical Systems Beucler, Tom; Pritchard, Michael; Rasp, Stephan Physical Review Letters, Vol. 126, Issue 9 https://doi.org/10.1103/PhysRevLett.126.098302	journal	March 2021
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data Karpatne, Anuj; Atluri, Gowtham; Faghmous, James H. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, Issue 10 https://doi.org/10.1109/TKDE.2017.2720168	journal	October 2017
Reducing emissions from agriculture to meet the 2 °C target Wollenberg, Eva; Richards, Meryl; Smith, Pete Global Change Biology, Vol. 22, Issue 12 https://doi.org/10.1111/gcb.13340	journal	July 2016
How to measure, report and verify soil carbon change to realize the potential of soil carbon sequestration for atmospheric greenhouse gas removal Smith, Pete; Soussana, Jean‐Francois; Angers, Denis Global Change Biology, Vol. 26, Issue 1 https://doi.org/10.1111/gcb.14815	journal	October 2019
Informing Nature‐based Climate Solutions for the United States with the best‐available science Novick, Kimberly A.; Metzger, Stefan; Anderegg, William R. L. Global Change Biology, Vol. 28, Issue 12 https://doi.org/10.1111/gcb.16156	journal	April 2022
Magnitude and sources of uncertainties in soil organic carbon (SOC) stock assessments at various scales Goidts, E.; van Wesemael, B.; Crucifix, M. European Journal of Soil Science, Vol. 60, Issue 5 https://doi.org/10.1111/j.1365-2389.2009.01157.x	journal	October 2009
Natural climate solutions for the United States Fargione, Joseph E.; Bassett, Steven; Boucher, Timothy Science Advances, Vol. 4, Issue 11 https://doi.org/10.1126/sciadv.aat1869	journal	November 2018
Global food system emissions could preclude achieving the 1.5° and 2°C climate change targets Clark, Michael A.; Domingo, Nina G. G.; Colgan, Kimberly Science, Vol. 370, Issue 6517 https://doi.org/10.1126/science.aba7357	journal	November 2020
Crediting agricultural soil carbon sequestration Oldfield, Emily E.; Eagle, Alison J.; Rubin, Rachel L. Science, Vol. 375, Issue 6586 https://doi.org/10.1126/science.abl7991	journal	March 2022
Taught to the test Hutson, Matthew Science, Vol. 376, Issue 6593 https://doi.org/10.1126/science.abq7833	journal	May 2022
Graph-based Reinforcement Learning for Active Learning in Real Time: An Application in Modeling River Networks Jia, Xiaowei; Lin, Beiyu; Zwart, Jacob Proceedings of the 2021 SIAM International Conference on Data Mining (SDM) https://doi.org/10.1137/1.9781611976700.70	book	April 2021
Mini-Batch Learning Strategies for modeling long term temporal dependencies: A study in environmental applications Xu, Shaoming; Khandelwal, Ankush; Li, Xiang Proceedings of the 2023 SIAM International Conference on Data Mining (SDM) https://doi.org/10.1137/1.9781611977653.ch73	journal	April 2023
Curriculum learning Bengio, Yoshua; Louradour, Jérôme; Collobert, Ronan Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09 https://doi.org/10.1145/1553374.1553380	conference	January 2009
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems Willard, Jared; Jia, Xiaowei; Xu, Shaoming ACM Computing Surveys, Vol. 55, Issue 4 https://doi.org/10.1145/3514228	journal	November 2022
Robust Inverse Framework using Knowledge-guided Self-Supervised Learning Ghosh, Rahul; Renganathan, Arvind; Tayal, Kshitij Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining https://doi.org/10.1145/3534678.3539448	conference	August 2022
Long Short-Term Memory Hochreiter, Sepp; Schmidhuber, Jürgen Neural Computation, Vol. 9, Issue 8 https://doi.org/10.1162/neco.1997.9.8.1735	journal	November 1997
Knowledge-Guided Machine Learning Karpatne, Anuj; Kannan, Ramakrishnan; Kumar, Vipin https://doi.org/10.1201/9781003143376	book	August 2022
Maximum soil organic carbon storage in Midwest U.S. cropping systems when crops are optimally nitrogen-fertilized Poffenbarger, Hanna J.; Barker, Daniel W.; Helmers, Matthew J. PLOS ONE, Vol. 12, Issue 3 https://doi.org/10.1371/journal.pone.0172293	journal	March 2017
Net Biome Productivity of Irrigated and Rainfed Maize–Soybean Rotations: Modeling vs. Measurements Grant, R. F.; Arkebauer, T. J.; Dobermann, A. Agronomy Journal, Vol. 99, Issue 6 https://doi.org/10.2134/agronj2006.0308	journal	January 2007
Controlled Warming Effects on Wheat Growth and Yield: Field Measurements and Modeling Grant, R. F.; Kimball, B. A.; Conley, M. M. Agronomy Journal, Vol. 103, Issue 6 https://doi.org/10.2134/agronj2011.0158	journal	January 2011
Soil Organic Carbon Storage under Biofuel Cropping Systems in a Humid, Continental Climate Ibrahim, Mostafa A.; Chua‐Ona, Teresita; Liebman, Matt Agronomy Journal, Vol. 110, Issue 5 https://doi.org/10.2134/agronj2018.03.0204	journal	September 2018
Tillage and Residue Management Effects on Soil Carbon and Nitrogen Under Irrigated Continuous Corn Schmer, M. R.; Jin, V. L.; Wienhold, B. J. Soil Science Society of America Journal, Vol. 78, Issue 6 https://doi.org/10.2136/sssaj2014.04.0166	journal	January 2014
Twelve Years of Stover Removal Increases Soil Erosion Potential without Impacting Yield Jin, Virginia L.; Schmer, Marty R.; Wienhold, Brian J. Soil Science Society of America Journal, Vol. 79, Issue 4 https://doi.org/10.2136/sssaj2015.02.0053	journal	January 2015
Chapter 7: The Earth’s energy budget, climate feedbacks, and climate sensitivity Forster, Piers; Storelvmo, Trude; Armour, Kyle Open Access Victoria University of Wellington \| Te Herenga Waka https://doi.org/10.25455/WGTN.16869671.V1	other	January 2021
Assessing Effects of Agronomic Nitrogen Management on Crop Nitrogen Use and Nitrogen Losses in the Western Canadian Prairies Mezbahuddin, Symon; Spiess, David; Hildebrand, David Frontiers in Sustainable Food Systems, Vol. 4 https://doi.org/10.3389/fsufs.2020.512292	journal	September 2020
Long-Term Effects of Cover Crops on Crop Yields, Soil Organic Carbon Stocks and Sequestration Olson, Kenneth; Ebelhar, Stephen A.; Lang, James M. Open Journal of Soil Science, Vol. 04, Issue 08 https://doi.org/10.4236/ojss.2014.48030	journal	January 2014
Physics Guided Machine Learning Methods for Hydrology Khandelwal, Ankush; Xu, Shaoming; Li, Xiang arXiv https://doi.org/10.48550/ARXIV.2012.02854	preprint	January 2020
Neural Machine Translation by Jointly Learning to Align and Translate Bahdanau, Dzmitry; Cho, Kyunghyun; Bengio, Yoshua arXiv https://doi.org/10.48550/arXiv.1409.0473	preprint	January 2014
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Cho, Kyunghyun; van Merrienboer, Bart; Bahdanau, Dzmitry arXiv https://doi.org/10.48550/arXiv.1409.1259	preprint	January 2014
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling Chung, Junyoung; Gulcehre, Caglar; Cho, KyungHyun arXiv https://doi.org/10.48550/arXiv.1412.3555	preprint	January 2014
Recent trends and drivers of regional sources and sinks of carbon dioxide Sitch, S.; Friedlingstein, P.; Gruber, N. Biogeosciences, Vol. 12, Issue 3 https://doi.org/10.5194/bg-12-653-2015	journal	January 2015
Scaling carbon fluxes from eddy covariance sites to globe: synthesis and evaluation of the FLUXCOM approach Jung, Martin; Schwalm, Christopher; Migliavacca, Mirco Biogeosciences, Vol. 17, Issue 5 https://doi.org/10.5194/bg-17-1343-2020	journal	January 2020
Modelling nitrification inhibitor effects on N₂O emissions after fall- and spring-applied slurry by reducing nitrifier NH₄⁺ oxidation rate Grant, Robert F.; Lin, Sisi; Hernandez-Ramirez, Guillermo Biogeosciences, Vol. 17, Issue 7 https://doi.org/10.5194/bg-17-2021-2020	journal	January 2020
Technical note: A view from space on global flux towers by MODIS and Landsat: the FluxnetEO data set Walther, Sophia; Besnard, Simon; Nelson, Jacob Allen Biogeosciences, Vol. 19, Issue 11 https://doi.org/10.5194/bg-19-2805-2022	journal	June 2022
KGML-ag: a modeling framework of knowledge-guided machine learning to simulate agroecosystems: a case study of estimating N₂O emission using data from mesocosm experiments Liu, Licheng; Xu, Shaoming; Tang, Jinyun Geoscientific Model Development, Vol. 15, Issue 7 https://doi.org/10.5194/gmd-15-2839-2022	journal	April 2022
Towards hybrid modeling of the global hydrological cycle Kraft, Basil; Jung, Martin; Körner, Marco Hydrology and Earth System Sciences, Vol. 26, Issue 6 https://doi.org/10.5194/hess-26-1579-2022	journal	January 2022
SoilGrids 2.0: producing soil information for the globe with quantified spatial uncertainty Poggio, Laura; de Sousa, Luis M.; Batjes, Niels H. SOIL, Vol. 7, Issue 1 https://doi.org/10.5194/soil-7-217-2021	journal	January 2021
The Log Generalized Lindley-Weibull Distribution with Application Oluyede, Broderick O.; Mutiso, Fedelis; Huang, Shujiao Journal of Data Science, Vol. 13, Issue 2 https://doi.org/10.6339/JDS.201504_13(2).0004	journal	April 2021