Title: Implementing Spatio-Temporal 3D-Convolution Neural Networks and UAV Time Series Imagery to Better Predict Lodging Damage in Sorghum

Abstract

Unmanned aerial vehicle (UAV)-based remote sensing is gaining momentum in a variety of agricultural and environmental applications. Very-high-resolution remote sensing image sets collected repeatedly throughout a crop growing season are becoming increasingly common. Analytical methods able to learn from both spatial and time dimensions of the data may allow for an improved estimation of crop traits, as well as the effects of genetics and the environment on these traits. Multispectral and geometric time series imagery was collected by UAV on 11 dates, along with ground-truth data, in a field trial of 866 genetically diverse biomass sorghum accessions. We compared the performance of Convolution Neural Network (CNN) architectures that used image data from single dates (two spatial dimensions, 2D) versus multiple dates (two spatial dimensions + temporal dimension, 3D) to estimate lodging detection and severity. Lodging was detected with 3D-CNN analysis of time series imagery with 0.88 accuracy, 0.92 Precision, and 0.83 Recall. This outperformed the best 2D-CNN on a single date with 0.85 accuracy, 0.84 Precision, and 0.76 Recall. The variation in lodging severity was estimated by the best 3D-CNN analysis with 9.4% mean absolute error (MAE), 11.9% root mean square error (RMSE), and goodness-of-fit (R2) of 0.76. This wasmore » a significant improvement over the best 2D-CNN analysis with 11.84% MAE, 14.91% RMSE, and 0.63 R2. The success of the improved 3D-CNN analysis approach depended on the inclusion of “before and after” data, i.e., images collected on dates before and after the lodging event. The integration of geometric and spectral features with 3D-CNN architecture was also key to the improved assessment of lodging severity, which is an important and difficult-to-assess phenomenon in bioenergy feedstocks such as biomass sorghum. This demonstrates that spatio-temporal CNN architectures based on UAV time series imagery have significant potential to enhance plant phenotyping capabilities in crop breeding and Precision agriculture applications.« less

Authors:

Varela, Sebastian ;

• Search DOE PAGES for author "Varela, Sebastian"
• Search DOE PAGES for ORCID "0000-0001-8064-3636"

Pederson, Taylor L.; Leakey, Andrew D. B.

Publication Date:

Fri Feb 04 00:00:00 EST 2022

Research Org.:

Center for Advanced Bioenergy and Bioproducts Innovation (CABBI), Urbana, IL (United States)

Sponsoring Org.:

USDOE Office of Science (SC), Biological and Environmental Research (BER)

OSTI Identifier:

1843857

Alternate Identifier(s):

OSTI ID: 1996350

Grant/Contract Number:  

SC0018420

Resource Type:

Published Article

Journal Name:

Remote Sensing

Additional Journal Information:

Journal Name: Remote Sensing Journal Volume: 14 Journal Issue: 3; Journal ID: ISSN 2072-4292

Publisher:

MDPI AG

Country of Publication:

Switzerland

Language:

English

Subject:

47 OTHER INSTRUMENTATION; 3D-convolution neural networks; time series; lodging; sorghum