GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields
Abstract
Acquired imagery by unmanned aerial vehicles (UAVs) has been widely used for three-dimensional (3D) reconstruction/modeling in various digital agriculture applications, such as phenotyping, crop monitoring, and yield prediction. 3D reconstruction from well-textured UAV-based images has matured and the user community has access to several commercial and opensource tools that provide accurate products at a high level of automation. However, in some applications, such as digital agriculture, due to repetitive image patterns, these approaches are not always able to produce reliable/complete products. The main limitation of these techniques is their inability to establish a sufficient number of correctly matched features among overlapping images, causing incomplete and/or inaccurate 3D reconstruction. This paper provides two structure from motion (SfM) strategies, which use trajectory information provided by an onboard survey-grade global navigation satellite system/inertial navigation system (GNSS/INS) and system calibration parameters. The main difference between the proposed strategies is that the first one—denoted as partially GNSS/INS-assisted SfM—implements the four stages of an automated triangulation procedure, namely, imaging matching, relative orientation parameters (ROPs) estimation, exterior orientation parameters (EOPs) recovery, and bundle adjustment (BA). The second strategy— denoted as fully GNSS/INS-assisted SfM—removes the EOPs estimation step while introducing a random sample consensus (RANSAC)-based strategy for removingmore »
- Authors:
- Publication Date:
- Research Org.:
- Purdue Univ., West Lafayette, IN (United States)
- Sponsoring Org.:
- USDOE Advanced Research Projects Agency - Energy (ARPA-E)
- OSTI Identifier:
- 1592811
- Alternate Identifier(s):
- OSTI ID: 1799143
- Grant/Contract Number:
- AR0000593; AR0001135
- Resource Type:
- Published Article
- Journal Name:
- Remote Sensing
- Additional Journal Information:
- Journal Name: Remote Sensing Journal Volume: 12 Journal Issue: 3; Journal ID: ISSN 2072-4292
- Publisher:
- MDPI AG
- Country of Publication:
- Switzerland
- Language:
- English
- Subject:
- 54 ENVIRONMENTAL SCIENCES; 47 OTHER INSTRUMENTATION; Environmental Sciences & Ecology; Geology; Remote Sensing; Imaging Science & Photographic Technology; structure from motion; automatic aerial triangulation; unmanned aerial vehicles; GNSS/INS-assisted mapping; image matching; epipolar geometry
Citation Formats
Hasheminasab, Seyyed Meghdad, Zhou, Tian, and Habib, Ayman. GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields. Switzerland: N. p., 2020.
Web. doi:10.3390/rs12030351.
Hasheminasab, Seyyed Meghdad, Zhou, Tian, & Habib, Ayman. GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields. Switzerland. https://doi.org/10.3390/rs12030351
Hasheminasab, Seyyed Meghdad, Zhou, Tian, and Habib, Ayman. Tue .
"GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields". Switzerland. https://doi.org/10.3390/rs12030351.
@article{osti_1592811,
title = {GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields},
author = {Hasheminasab, Seyyed Meghdad and Zhou, Tian and Habib, Ayman},
abstractNote = {Acquired imagery by unmanned aerial vehicles (UAVs) has been widely used for three-dimensional (3D) reconstruction/modeling in various digital agriculture applications, such as phenotyping, crop monitoring, and yield prediction. 3D reconstruction from well-textured UAV-based images has matured and the user community has access to several commercial and opensource tools that provide accurate products at a high level of automation. However, in some applications, such as digital agriculture, due to repetitive image patterns, these approaches are not always able to produce reliable/complete products. The main limitation of these techniques is their inability to establish a sufficient number of correctly matched features among overlapping images, causing incomplete and/or inaccurate 3D reconstruction. This paper provides two structure from motion (SfM) strategies, which use trajectory information provided by an onboard survey-grade global navigation satellite system/inertial navigation system (GNSS/INS) and system calibration parameters. The main difference between the proposed strategies is that the first one—denoted as partially GNSS/INS-assisted SfM—implements the four stages of an automated triangulation procedure, namely, imaging matching, relative orientation parameters (ROPs) estimation, exterior orientation parameters (EOPs) recovery, and bundle adjustment (BA). The second strategy— denoted as fully GNSS/INS-assisted SfM—removes the EOPs estimation step while introducing a random sample consensus (RANSAC)-based strategy for removing matching outliers before the BA stage. Both strategies modify the image matching by restricting the search space for conjugate points. They also implement a linear procedure for ROPs’ refinement. Finally, they use the GNSS/INS information in modified collinearity equations for a simpler BA procedure that could be used for refining system calibration parameters. Eight datasets over six agricultural fields are used to evaluate the performance of the developed strategies. In comparison with a traditional SfM framework and Pix4D Mapper Pro, the proposed strategies are able to generate denser and more accurate 3D point clouds as well as orthophotos without any gaps.},
doi = {10.3390/rs12030351},
journal = {Remote Sensing},
number = 3,
volume = 12,
place = {Switzerland},
year = {2020},
month = {1}
}
https://doi.org/10.3390/rs12030351
Web of Science
Works referenced in this record:
Improving Orthorectification of UAV-Based Push-Broom Scanner Imagery Using Derived Orthophotos From Frame Cameras
journal, January 2017
- Habib, Ayman; Xiong, Weifeng; He, Fangning
- IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 10, Issue 1
Unmanned Aerial Vehicle Remote Sensing for Field-Based Crop Phenotyping: Current Status and Perspectives
journal, June 2017
- Yang, Guijun; Liu, Jiangang; Zhao, Chunjiang
- Frontiers in Plant Science, Vol. 8
Automated Aerial Triangulation for UAV-Based Mapping
journal, December 2018
- He, Fangning; Zhou, Tian; Xiong, Weifeng
- Remote Sensing, Vol. 10, Issue 12
Is There a Need for a More Sustainable Agriculture?
journal, January 2011
- Gomiero, Tiziano; Pimentel, David; Paoletti, Maurizio G.
- Critical Reviews in Plant Sciences, Vol. 30, Issue 1-2
Automated Relative Orientation of UAV-Based Imagery in the Presence of Prior Information for the Flight Trajectory
journal, November 2016
- He, Fangning; Habib, Ayman
- Photogrammetric Engineering & Remote Sensing, Vol. 82, Issue 11
Evaluation of UAV LiDAR for Mapping Coastal Environments
journal, December 2019
- Lin, Yi-Chun; Cheng, Yi-Ting; Zhou, Tian
- Remote Sensing, Vol. 11, Issue 24
Corn Plant Counting Using Deep Learning and UAV Images
journal, January 2019
- Kitano, Bruno T.; Mendes, Caio C. T.; Geus, Andre R.
- IEEE Geoscience and Remote Sensing Letters
A computer algorithm for reconstructing a scene from two projections
journal, September 1981
- Longuet-Higgins, H. C.
- Nature, Vol. 293, Issue 5828
Feeding 10 billion people under climate change: How large is the production gap of current agricultural systems?
journal, September 2014
- Sakschewski, Boris; von Bloh, Werner; Huber, Veronika
- Ecological Modelling, Vol. 288
Deep Learning-Based Damage Detection from Aerial SfM Point Clouds
journal, August 2019
- Mohammadi,
- Drones, Vol. 3, Issue 3
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography
journal, June 1981
- Fischler, Martin A.; Bolles, Robert C.
- Communications of the ACM, Vol. 24, Issue 6
Drift detection and removal for sequential structure from motion algorithms
journal, October 2004
- Cornelis, K.; Verbiest, F.; Van Gool, L.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Issue 10
Rotation Averaging
journal, January 2013
- Hartley, Richard; Trumpf, Jochen; Dai, Yuchao
- International Journal of Computer Vision, Vol. 103, Issue 3
Boresight Calibration of GNSS/INS-Assisted Push-Broom Hyperspectral Scanners on UAV Platforms
journal, May 2018
- Habib, Ayman; Zhou, Tian; Masjedi, Ali
- IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, Issue 5
Calibration and accuracy assessment in a direct georeferencing system for UAS photogrammetry
journal, February 2018
- Gabrlik, Petr; Cour-Harbo, Anders la; Kalvodova, Petra
- International Journal of Remote Sensing, Vol. 39, Issue 15-16
Simultaneous System Calibration of a Multi-LiDAR Multicamera Mobile Mapping Platform
journal, May 2018
- Ravi, Radhika; Lin, Yun-Jou; Elbahnasawy, Magdy
- IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, Issue 5
An Automated Technique for Generating Georectified Mosaics from Ultra-High Resolution Unmanned Aerial Vehicle (UAV) Imagery, Based on Structure from Motion (SfM) Point Clouds
journal, May 2012
- Turner, Darren; Lucieer, Arko; Watson, Christopher
- Remote Sensing, Vol. 4, Issue 5
An efficient solution to the five-point relative pose problem
journal, June 2004
- Nister, D.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Issue 6
Unmanned Aerial Vehicle-Based Phenotyping Using Morphometric and Spectral Analysis Can Quantify Responses of Wild Tomato Plants to Salinity Stress
journal, March 2019
- Johansen, Kasper; Morton, Mitchell J. L.; Malbeteau, Yoann M.
- Frontiers in Plant Science, Vol. 10
Unmanned Aerial Vehicles for High-Throughput Phenotyping and Agronomic Research
journal, July 2016
- Shi, Yeyin; Thomasson, J. Alex; Murray, Seth C.
- PLOS ONE, Vol. 11, Issue 7
Thermal and Narrowband Multispectral Remote Sensing for Vegetation Monitoring From an Unmanned Aerial Vehicle
journal, March 2009
- Berni, J.; Zarco-Tejada, P. J.; Suarez, L.
- IEEE Transactions on Geoscience and Remote Sensing, Vol. 47, Issue 3
A performance evaluation of local descriptors
journal, October 2005
- Mikolajczyk, K.; Schmid, C.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 10
New Strategies for Time Delay Estimation During System Calibration for UAV-Based GNSS/INS-Assisted Imaging Systems
journal, August 2019
- LaForest,
- Remote Sensing, Vol. 11, Issue 15
Distinctive Image Features from Scale-Invariant Keypoints
journal, November 2004
- Lowe, David G.
- International Journal of Computer Vision, Vol. 60, Issue 2, p. 91-110
Big Data in Smart Farming – A review
journal, May 2017
- Wolfert, Sjaak; Ge, Lan; Verdouw, Cor
- Agricultural Systems, Vol. 153
In defense of the eight-point algorithm
journal, June 1997
- Hartley, R. I.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, Issue 6
Using unmanned aerial vehicle-based multispectral, RGB and thermal imagery for phenotyping of forest genetic trials: A case study in Pinus halepensis
journal, January 2019
- Santini, Filippo; Kefauver, Shawn C.; Resco de Dios, Victor
- Annals of Applied Biology, Vol. 174, Issue 2
Accurate Calibration Scheme for a Multi-Camera Mobile Mapping System
journal, November 2019
- Khoramshahi,
- Remote Sensing, Vol. 11, Issue 23
Rapid Mosaicking of Unmanned Aerial Vehicle (UAV) Images for Crop Growth Monitoring Using the SIFT Algorithm
journal, May 2019
- Zhao, Jianqing; Zhang, Xiaohu; Gao, Chenxi
- Remote Sensing, Vol. 11, Issue 10
Establishment of Plot-Yield Prediction Models in Soybean Breeding Programs Using UAV-Based Hyperspectral Remote Sensing
journal, November 2019
- Zhang, Xiaoyan; Zhao, Jinming; Yang, Guijun
- Remote Sensing, Vol. 11, Issue 23
Building Damage Extraction from Post-earthquake Airborne LiDAR Data
journal, August 2016
- Aixia, Dou; Zongjin, Ma; Shusong, Huang
- Acta Geologica Sinica - English Edition, Vol. 90, Issue 4
Phenotyping of Corn Plants Using Unmanned Aerial Vehicle (UAV) Images
journal, August 2019
- Su, Wei; Zhang, Mingzheng; Bian, Dahong
- Remote Sensing, Vol. 11, Issue 17
Acquisition of NIR-Green-Blue Digital Photographs from Unmanned Aircraft for Crop Monitoring
journal, January 2010
- Hunt, E. Raymond; Hively, W. Dean; Fujikawa, Stephen
- Remote Sensing, Vol. 2, Issue 1
DEM orientation based on local feature correspondence with global DEMs
journal, August 2017
- Sedaghat, Amin; Alizadeh Naeini, Amin
- GIScience & Remote Sensing, Vol. 55, Issue 1
Three-point-based solution for automated motion parameter estimation of a multi-camera indoor mapping system with planar motion constraint
journal, August 2018
- He, Fangning; Habib, Ayman
- ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 142
A Deep Learning Semantic Segmentation-Based Approach for Field-Level Sorghum Panicle Counting
journal, December 2019
- Malambo, Lonesome; Popescu, Sorin; Ku, Nian-Wei
- Remote Sensing, Vol. 11, Issue 24
Relative orientation
journal, January 1990
- Horn, Berthold K. P.
- International Journal of Computer Vision, Vol. 4, Issue 1
Food Security: The Challenge of Feeding 9 Billion People
journal, January 2010
- Godfray, H. C. J.; Beddington, J. R.; Crute, I. R.
- Science, Vol. 327, Issue 5967
Assessment of Unmanned Aerial Vehicles Imagery for Quantitative Monitoring of Wheat Crop in Small Plots
journal, May 2008
- Lelong, Camille; Burger, Philippe; Jubelin, Guillaume
- Sensors, Vol. 8, Issue 5
Indoor robot motion based on monocular images
journal, April 2001
- Ortín, D.; Montiel, J. M. M.
- Robotica, Vol. 19, Issue 3