skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields

Abstract

Acquired imagery by unmanned aerial vehicles (UAVs) has been widely used for three-dimensional (3D) reconstruction/modeling in various digital agriculture applications, such as phenotyping, crop monitoring, and yield prediction. 3D reconstruction from well-textured UAV-based images has matured and the user community has access to several commercial and opensource tools that provide accurate products at a high level of automation. However, in some applications, such as digital agriculture, due to repetitive image patterns, these approaches are not always able to produce reliable/complete products. The main limitation of these techniques is their inability to establish a sufficient number of correctly matched features among overlapping images, causing incomplete and/or inaccurate 3D reconstruction. This paper provides two structure from motion (SfM) strategies, which use trajectory information provided by an onboard survey-grade global navigation satellite system/inertial navigation system (GNSS/INS) and system calibration parameters. The main difference between the proposed strategies is that the first one—denoted as partially GNSS/INS-assisted SfM—implements the four stages of an automated triangulation procedure, namely, imaging matching, relative orientation parameters (ROPs) estimation, exterior orientation parameters (EOPs) recovery, and bundle adjustment (BA). The second strategy— denoted as fully GNSS/INS-assisted SfM—removes the EOPs estimation step while introducing a random sample consensus (RANSAC)-based strategy for removingmore » matching outliers before the BA stage. Both strategies modify the image matching by restricting the search space for conjugate points. They also implement a linear procedure for ROPs’ refinement. Finally, they use the GNSS/INS information in modified collinearity equations for a simpler BA procedure that could be used for refining system calibration parameters. Eight datasets over six agricultural fields are used to evaluate the performance of the developed strategies. In comparison with a traditional SfM framework and Pix4D Mapper Pro, the proposed strategies are able to generate denser and more accurate 3D point clouds as well as orthophotos without any gaps.« less

Authors:
ORCiD logo; ORCiD logo; ORCiD logo
Publication Date:
Sponsoring Org.:
USDOE Advanced Research Projects Agency - Energy (ARPA-E)
OSTI Identifier:
1592811
Grant/Contract Number:  
AR0000593
Resource Type:
Published Article
Journal Name:
Remote Sensing
Additional Journal Information:
Journal Name: Remote Sensing Journal Volume: 12 Journal Issue: 3; Journal ID: ISSN 2072-4292
Publisher:
MDPI AG
Country of Publication:
Switzerland
Language:
English

Citation Formats

Hasheminasab, Seyyed Meghdad, Zhou, Tian, and Habib, Ayman. GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields. Switzerland: N. p., 2020. Web. doi:10.3390/rs12030351.
Hasheminasab, Seyyed Meghdad, Zhou, Tian, & Habib, Ayman. GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields. Switzerland. doi:10.3390/rs12030351.
Hasheminasab, Seyyed Meghdad, Zhou, Tian, and Habib, Ayman. Tue . "GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields". Switzerland. doi:10.3390/rs12030351.
@article{osti_1592811,
title = {GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields},
author = {Hasheminasab, Seyyed Meghdad and Zhou, Tian and Habib, Ayman},
abstractNote = {Acquired imagery by unmanned aerial vehicles (UAVs) has been widely used for three-dimensional (3D) reconstruction/modeling in various digital agriculture applications, such as phenotyping, crop monitoring, and yield prediction. 3D reconstruction from well-textured UAV-based images has matured and the user community has access to several commercial and opensource tools that provide accurate products at a high level of automation. However, in some applications, such as digital agriculture, due to repetitive image patterns, these approaches are not always able to produce reliable/complete products. The main limitation of these techniques is their inability to establish a sufficient number of correctly matched features among overlapping images, causing incomplete and/or inaccurate 3D reconstruction. This paper provides two structure from motion (SfM) strategies, which use trajectory information provided by an onboard survey-grade global navigation satellite system/inertial navigation system (GNSS/INS) and system calibration parameters. The main difference between the proposed strategies is that the first one—denoted as partially GNSS/INS-assisted SfM—implements the four stages of an automated triangulation procedure, namely, imaging matching, relative orientation parameters (ROPs) estimation, exterior orientation parameters (EOPs) recovery, and bundle adjustment (BA). The second strategy— denoted as fully GNSS/INS-assisted SfM—removes the EOPs estimation step while introducing a random sample consensus (RANSAC)-based strategy for removing matching outliers before the BA stage. Both strategies modify the image matching by restricting the search space for conjugate points. They also implement a linear procedure for ROPs’ refinement. Finally, they use the GNSS/INS information in modified collinearity equations for a simpler BA procedure that could be used for refining system calibration parameters. Eight datasets over six agricultural fields are used to evaluate the performance of the developed strategies. In comparison with a traditional SfM framework and Pix4D Mapper Pro, the proposed strategies are able to generate denser and more accurate 3D point clouds as well as orthophotos without any gaps.},
doi = {10.3390/rs12030351},
journal = {Remote Sensing},
number = 3,
volume = 12,
place = {Switzerland},
year = {2020},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.3390/rs12030351

Save / Share:

Works referenced in this record:

Improving Orthorectification of UAV-Based Push-Broom Scanner Imagery Using Derived Orthophotos From Frame Cameras
journal, January 2017

  • Habib, Ayman; Xiong, Weifeng; He, Fangning
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 10, Issue 1
  • DOI: 10.1109/JSTARS.2016.2520929

Unmanned Aerial Vehicle Remote Sensing for Field-Based Crop Phenotyping: Current Status and Perspectives
journal, June 2017


Automated Aerial Triangulation for UAV-Based Mapping
journal, December 2018

  • He, Fangning; Zhou, Tian; Xiong, Weifeng
  • Remote Sensing, Vol. 10, Issue 12
  • DOI: 10.3390/rs10121952

Is There a Need for a More Sustainable Agriculture?
journal, January 2011

  • Gomiero, Tiziano; Pimentel, David; Paoletti, Maurizio G.
  • Critical Reviews in Plant Sciences, Vol. 30, Issue 1-2
  • DOI: 10.1080/07352689.2011.553515

Automated Relative Orientation of UAV-Based Imagery in the Presence of Prior Information for the Flight Trajectory
journal, November 2016

  • He, Fangning; Habib, Ayman
  • Photogrammetric Engineering & Remote Sensing, Vol. 82, Issue 11
  • DOI: 10.14358/PERS.82.11.879

Evaluation of UAV LiDAR for Mapping Coastal Environments
journal, December 2019

  • Lin, Yi-Chun; Cheng, Yi-Ting; Zhou, Tian
  • Remote Sensing, Vol. 11, Issue 24
  • DOI: 10.3390/rs11242893

Corn Plant Counting Using Deep Learning and UAV Images
journal, January 2019

  • Kitano, Bruno T.; Mendes, Caio C. T.; Geus, Andre R.
  • IEEE Geoscience and Remote Sensing Letters
  • DOI: 10.1109/LGRS.2019.2930549

A computer algorithm for reconstructing a scene from two projections
journal, September 1981


Feeding 10 billion people under climate change: How large is the production gap of current agricultural systems?
journal, September 2014


Deep Learning-Based Damage Detection from Aerial SfM Point Clouds
journal, August 2019


Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography
journal, June 1981

  • Fischler, Martin A.; Bolles, Robert C.
  • Communications of the ACM, Vol. 24, Issue 6
  • DOI: 10.1145/358669.358692

Drift detection and removal for sequential structure from motion algorithms
journal, October 2004

  • Cornelis, K.; Verbiest, F.; Van Gool, L.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Issue 10
  • DOI: 10.1109/TPAMI.2004.85

Rotation Averaging
journal, January 2013

  • Hartley, Richard; Trumpf, Jochen; Dai, Yuchao
  • International Journal of Computer Vision, Vol. 103, Issue 3
  • DOI: 10.1007/s11263-012-0601-0

Boresight Calibration of GNSS/INS-Assisted Push-Broom Hyperspectral Scanners on UAV Platforms
journal, May 2018

  • Habib, Ayman; Zhou, Tian; Masjedi, Ali
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, Issue 5
  • DOI: 10.1109/JSTARS.2018.2813263

Calibration and accuracy assessment in a direct georeferencing system for UAS photogrammetry
journal, February 2018

  • Gabrlik, Petr; Cour-Harbo, Anders la; Kalvodova, Petra
  • International Journal of Remote Sensing, Vol. 39, Issue 15-16
  • DOI: 10.1080/01431161.2018.1434331

Simultaneous System Calibration of a Multi-LiDAR Multicamera Mobile Mapping Platform
journal, May 2018

  • Ravi, Radhika; Lin, Yun-Jou; Elbahnasawy, Magdy
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, Issue 5
  • DOI: 10.1109/JSTARS.2018.2812796

An efficient solution to the five-point relative pose problem
journal, June 2004

  • Nister, D.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Issue 6
  • DOI: 10.1109/TPAMI.2004.17

Unmanned Aerial Vehicle-Based Phenotyping Using Morphometric and Spectral Analysis Can Quantify Responses of Wild Tomato Plants to Salinity Stress
journal, March 2019

  • Johansen, Kasper; Morton, Mitchell J. L.; Malbeteau, Yoann M.
  • Frontiers in Plant Science, Vol. 10
  • DOI: 10.3389/fpls.2019.00370

Unmanned Aerial Vehicles for High-Throughput Phenotyping and Agronomic Research
journal, July 2016


Thermal and Narrowband Multispectral Remote Sensing for Vegetation Monitoring From an Unmanned Aerial Vehicle
journal, March 2009

  • Berni, J.; Zarco-Tejada, P. J.; Suarez, L.
  • IEEE Transactions on Geoscience and Remote Sensing, Vol. 47, Issue 3
  • DOI: 10.1109/TGRS.2008.2010457

A performance evaluation of local descriptors
journal, October 2005

  • Mikolajczyk, K.; Schmid, C.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 10
  • DOI: 10.1109/TPAMI.2005.188

Distinctive Image Features from Scale-Invariant Keypoints
journal, November 2004


Big Data in Smart Farming – A review
journal, May 2017


In defense of the eight-point algorithm
journal, June 1997

  • Hartley, R. I.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, Issue 6
  • DOI: 10.1109/34.601246

Using unmanned aerial vehicle-based multispectral, RGB and thermal imagery for phenotyping of forest genetic trials: A case study in Pinus halepensis
journal, January 2019

  • Santini, Filippo; Kefauver, Shawn C.; Resco de Dios, Victor
  • Annals of Applied Biology, Vol. 174, Issue 2
  • DOI: 10.1111/aab.12484

Accurate Calibration Scheme for a Multi-Camera Mobile Mapping System
journal, November 2019


Rapid Mosaicking of Unmanned Aerial Vehicle (UAV) Images for Crop Growth Monitoring Using the SIFT Algorithm
journal, May 2019

  • Zhao, Jianqing; Zhang, Xiaohu; Gao, Chenxi
  • Remote Sensing, Vol. 11, Issue 10
  • DOI: 10.3390/rs11101226

Establishment of Plot-Yield Prediction Models in Soybean Breeding Programs Using UAV-Based Hyperspectral Remote Sensing
journal, November 2019

  • Zhang, Xiaoyan; Zhao, Jinming; Yang, Guijun
  • Remote Sensing, Vol. 11, Issue 23
  • DOI: 10.3390/rs11232752

Building Damage Extraction from Post-earthquake Airborne LiDAR Data
journal, August 2016

  • Aixia, Dou; Zongjin, Ma; Shusong, Huang
  • Acta Geologica Sinica - English Edition, Vol. 90, Issue 4
  • DOI: 10.1111/1755-6724.12781

Phenotyping of Corn Plants Using Unmanned Aerial Vehicle (UAV) Images
journal, August 2019

  • Su, Wei; Zhang, Mingzheng; Bian, Dahong
  • Remote Sensing, Vol. 11, Issue 17
  • DOI: 10.3390/rs11172021

Acquisition of NIR-Green-Blue Digital Photographs from Unmanned Aircraft for Crop Monitoring
journal, January 2010

  • Hunt, E. Raymond; Hively, W. Dean; Fujikawa, Stephen
  • Remote Sensing, Vol. 2, Issue 1
  • DOI: 10.3390/rs2010290

DEM orientation based on local feature correspondence with global DEMs
journal, August 2017


Three-point-based solution for automated motion parameter estimation of a multi-camera indoor mapping system with planar motion constraint
journal, August 2018


A Deep Learning Semantic Segmentation-Based Approach for Field-Level Sorghum Panicle Counting
journal, December 2019

  • Malambo, Lonesome; Popescu, Sorin; Ku, Nian-Wei
  • Remote Sensing, Vol. 11, Issue 24
  • DOI: 10.3390/rs11242939

Relative orientation
journal, January 1990

  • Horn, Berthold K. P.
  • International Journal of Computer Vision, Vol. 4, Issue 1
  • DOI: 10.1007/BF00137443

Food Security: The Challenge of Feeding 9 Billion People
journal, January 2010


Assessment of Unmanned Aerial Vehicles Imagery for Quantitative Monitoring of Wheat Crop in Small Plots
journal, May 2008

  • Lelong, Camille; Burger, Philippe; Jubelin, Guillaume
  • Sensors, Vol. 8, Issue 5
  • DOI: 10.3390/s8053557

Indoor robot motion based on monocular images
journal, April 2001