skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: 3-D Semantic Information Inference from Airborne Video. Final Report

Abstract

We developed a full 3D modeling pipeline focusing on an important application scenario: city-scale 3D reconstruction from aerial imagery. We propose an approach to solve camera pose estimation and dense reconstruction from Wide Area Aerial Surveillance (WAAS) videos captured by an airborne platform. Our approach solves them in an online fashion: it incrementally updates a sparse 3D map and estimates the camera pose as each new frame arrives; depth maps of selected key frames are computed using a variational method and integrated to produce a full 3D model via volumetric reconstruction. In practice, aerial imagery is usually captured using a multicamera system. We propose an approach for camera pose estimation of multi-camera aerial imagery which is parallelized on multiple GPUs for efficiency. The approach is also extended for progressive 3D model acquisition with a hand-held camera. We also developed an offline approach since online approach is not a necessity and accuracy has higher priority over efficiency in many scenarios. We presented MeshRecon, a mesh-based offline system composed of three modules: a dense point cloud is generated using multi-resolution plane sweep method; an initial mesh model is extracted from the point cloud via global optimization considering visibility information of all images;more » the mesh model is then iteratively refined to capture structural details by optimizing the photometric consistency and spatial regularization. The major processes are parallelized on GPU for efficiency. For the aerial imagery case, we evaluate our system on several real-world multicamera aerial imagery datasets, each covering an urban scenario of several square kilometers. Quantitative result shows that the reconstructed geometric 3D model is highly accurate with error smaller than 1 meter over the entire city. Besides aerial imagery, we also evaluate its performance on general geometric 3D model acquisition of real-world objects. Result shows that the system is robust and flexible for various types of objects at different scales in both indoor and outdoor environments. Based on city 3D models reconstructed at different times, we present a system for city-scale geometry change detection by performing comparisons at the 3D geometry level. Our system is able to detect geometry changes at different scales, ranging from a building cluster to small-scale vegetation changes, with high accuracy.« less

Authors:
 [1]
  1. Univ. of Southern California, Los Angeles, CA (United States)
Publication Date:
Research Org.:
Univ. of Southern California, Los Angeles, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1457372
Report Number(s):
DOE-NA0001683
DOE Contract Number:  
NA0001683
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Medioni, Gerard. 3-D Semantic Information Inference from Airborne Video. Final Report. United States: N. p., 2018. Web. doi:10.2172/1457372.
Medioni, Gerard. 3-D Semantic Information Inference from Airborne Video. Final Report. United States. doi:10.2172/1457372.
Medioni, Gerard. Mon . "3-D Semantic Information Inference from Airborne Video. Final Report". United States. doi:10.2172/1457372. https://www.osti.gov/servlets/purl/1457372.
@article{osti_1457372,
title = {3-D Semantic Information Inference from Airborne Video. Final Report},
author = {Medioni, Gerard},
abstractNote = {We developed a full 3D modeling pipeline focusing on an important application scenario: city-scale 3D reconstruction from aerial imagery. We propose an approach to solve camera pose estimation and dense reconstruction from Wide Area Aerial Surveillance (WAAS) videos captured by an airborne platform. Our approach solves them in an online fashion: it incrementally updates a sparse 3D map and estimates the camera pose as each new frame arrives; depth maps of selected key frames are computed using a variational method and integrated to produce a full 3D model via volumetric reconstruction. In practice, aerial imagery is usually captured using a multicamera system. We propose an approach for camera pose estimation of multi-camera aerial imagery which is parallelized on multiple GPUs for efficiency. The approach is also extended for progressive 3D model acquisition with a hand-held camera. We also developed an offline approach since online approach is not a necessity and accuracy has higher priority over efficiency in many scenarios. We presented MeshRecon, a mesh-based offline system composed of three modules: a dense point cloud is generated using multi-resolution plane sweep method; an initial mesh model is extracted from the point cloud via global optimization considering visibility information of all images; the mesh model is then iteratively refined to capture structural details by optimizing the photometric consistency and spatial regularization. The major processes are parallelized on GPU for efficiency. For the aerial imagery case, we evaluate our system on several real-world multicamera aerial imagery datasets, each covering an urban scenario of several square kilometers. Quantitative result shows that the reconstructed geometric 3D model is highly accurate with error smaller than 1 meter over the entire city. Besides aerial imagery, we also evaluate its performance on general geometric 3D model acquisition of real-world objects. Result shows that the system is robust and flexible for various types of objects at different scales in both indoor and outdoor environments. Based on city 3D models reconstructed at different times, we present a system for city-scale geometry change detection by performing comparisons at the 3D geometry level. Our system is able to detect geometry changes at different scales, ranging from a building cluster to small-scale vegetation changes, with high accuracy.},
doi = {10.2172/1457372},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jun 25 00:00:00 EDT 2018},
month = {Mon Jun 25 00:00:00 EDT 2018}
}

Technical Report:

Save / Share: