skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system

Abstract

Purpose: To develop a robust and real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system. Methods: The authors have developed a robust and fast surface reconstruction method on point clouds acquired by the photogrammetry system, without explicitly solving the partial differential equation required by a typical variational approach. Taking advantage of the overcomplete nature of the acquired point clouds, their method solves and propagates a sparse linear relationship from the point cloud manifold to the surface manifold, assuming both manifolds share similar local geometry. With relatively consistent point cloud acquisitions, the authors propose a sparse regression (SR) model to directly approximate the target point cloud as a sparse linear combination from the training set, assuming that the point correspondences built by the iterative closest point (ICP) is reasonably accurate and have residual errors following a Gaussian distribution. To accommodate changing noise levels and/or presence of inconsistent occlusions during the acquisition, the authors further propose a modified sparse regression (MSR) model to model the potentially large and sparse error built by ICP with a Laplacian prior. The authors evaluated the proposed method on both clinical point clouds acquired under consistent acquisition conditions and on point cloudsmore » with inconsistent occlusions. The authors quantitatively evaluated the reconstruction performance with respect to root-mean-squared-error, by comparing its reconstruction results against that from the variational method. Results: On clinical point clouds, both the SR and MSR models have achieved sub-millimeter reconstruction accuracy and reduced the reconstruction time by two orders of magnitude to a subsecond reconstruction time. On point clouds with inconsistent occlusions, the MSR model has demonstrated its advantage in achieving consistent and robust performance despite the introduced occlusions. Conclusions: The authors have developed a fast and robust surface reconstruction method on point clouds captured from a 3D surface photogrammetry system, with demonstrated sub-millimeter reconstruction accuracy and subsecond reconstruction time. It is suitable for real-time motion tracking in radiotherapy, with clear surface structures for better quantifications.« less

Authors:
 [1];  [2];  [3];  [4]
  1. Department of Bioengineering, University of California, Los Angeles, Los Angeles, California 90095 (United States)
  2. Department of Radiation Oncology, University of Texas Southwestern, Dallas, Texas 75390 (United States)
  3. Department of Radiation Oncology, University of Texas Southwestern, Dallas, Texas, 75390 and Department of Radiation Oncology, University of Maryland, College Park, Maryland 20742 (United States)
  4. Department of Bioengineering, University of California, Los Angeles, Los Angeles, California 90095 and Department of Radiation Oncology, University of California, Los Angeles, Los Angeles, California 90095 (United States)
Publication Date:
OSTI Identifier:
22620890
Resource Type:
Journal Article
Resource Relation:
Journal Name: Medical Physics; Journal Volume: 43; Journal Issue: 5; Other Information: (c) 2016 American Association of Physicists in Medicine; Country of input: International Atomic Energy Agency (IAEA)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; 61 RADIATION PROTECTION AND DOSIMETRY; 60 APPLIED LIFE SCIENCES; ACCURACY; ERRORS; GAUSS FUNCTION; ITERATIVE METHODS; LAPLACIAN; PARTIAL DIFFERENTIAL EQUATIONS; PERFORMANCE; RADIOTHERAPY; VARIATIONAL METHODS

Citation Formats

Liu, Wenyang, Cheung, Yam, Sawant, Amit, and Ruan, Dan, E-mail: druan@mednet.ucla.edu. A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system. United States: N. p., 2016. Web. doi:10.1118/1.4945695.
Liu, Wenyang, Cheung, Yam, Sawant, Amit, & Ruan, Dan, E-mail: druan@mednet.ucla.edu. A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system. United States. doi:10.1118/1.4945695.
Liu, Wenyang, Cheung, Yam, Sawant, Amit, and Ruan, Dan, E-mail: druan@mednet.ucla.edu. 2016. "A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system". United States. doi:10.1118/1.4945695.
@article{osti_22620890,
title = {A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system},
author = {Liu, Wenyang and Cheung, Yam and Sawant, Amit and Ruan, Dan, E-mail: druan@mednet.ucla.edu},
abstractNote = {Purpose: To develop a robust and real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system. Methods: The authors have developed a robust and fast surface reconstruction method on point clouds acquired by the photogrammetry system, without explicitly solving the partial differential equation required by a typical variational approach. Taking advantage of the overcomplete nature of the acquired point clouds, their method solves and propagates a sparse linear relationship from the point cloud manifold to the surface manifold, assuming both manifolds share similar local geometry. With relatively consistent point cloud acquisitions, the authors propose a sparse regression (SR) model to directly approximate the target point cloud as a sparse linear combination from the training set, assuming that the point correspondences built by the iterative closest point (ICP) is reasonably accurate and have residual errors following a Gaussian distribution. To accommodate changing noise levels and/or presence of inconsistent occlusions during the acquisition, the authors further propose a modified sparse regression (MSR) model to model the potentially large and sparse error built by ICP with a Laplacian prior. The authors evaluated the proposed method on both clinical point clouds acquired under consistent acquisition conditions and on point clouds with inconsistent occlusions. The authors quantitatively evaluated the reconstruction performance with respect to root-mean-squared-error, by comparing its reconstruction results against that from the variational method. Results: On clinical point clouds, both the SR and MSR models have achieved sub-millimeter reconstruction accuracy and reduced the reconstruction time by two orders of magnitude to a subsecond reconstruction time. On point clouds with inconsistent occlusions, the MSR model has demonstrated its advantage in achieving consistent and robust performance despite the introduced occlusions. Conclusions: The authors have developed a fast and robust surface reconstruction method on point clouds captured from a 3D surface photogrammetry system, with demonstrated sub-millimeter reconstruction accuracy and subsecond reconstruction time. It is suitable for real-time motion tracking in radiotherapy, with clear surface structures for better quantifications.},
doi = {10.1118/1.4945695},
journal = {Medical Physics},
number = 5,
volume = 43,
place = {United States},
year = 2016,
month = 5
}
  • Purpose: To accurately and efficiently reconstruct a continuous surface from noisy point clouds captured by a surface photogrammetry system (VisionRT). Methods: The authors have developed a level-set based surface reconstruction method on point clouds captured by a surface photogrammetry system (VisionRT). The proposed method reconstructs an implicit and continuous representation of the underlying patient surface by optimizing a regularized fitting energy, offering extra robustness to noise and missing measurements. By contrast to explicit/discrete meshing-type schemes, their continuous representation is particularly advantageous for subsequent surface registration and motion tracking by eliminating the need for maintaining explicit point correspondences as in discretemore » models. The authors solve the proposed method with an efficient narrowband evolving scheme. The authors evaluated the proposed method on both phantom and human subject data with two sets of complementary experiments. In the first set of experiment, the authors generated a series of surfaces each with different black patches placed on one chest phantom. The resulting VisionRT measurements from the patched area had different degree of noise and missing levels, since VisionRT has difficulties in detecting dark surfaces. The authors applied the proposed method to point clouds acquired under these different configurations, and quantitatively evaluated reconstructed surfaces by comparing against a high-quality reference surface with respect to root mean squared error (RMSE). In the second set of experiment, the authors applied their method to 100 clinical point clouds acquired from one human subject. In the absence of ground-truth, the authors qualitatively validated reconstructed surfaces by comparing the local geometry, specifically mean curvature distributions, against that of the surface extracted from a high-quality CT obtained from the same patient. Results: On phantom point clouds, their method achieved submillimeter reconstruction RMSE under different configurations, demonstrating quantitatively the faith of the proposed method in preserving local structural properties of the underlying surface in the presence of noise and missing measurements, and its robustness toward variations of such characteristics. On point clouds from the human subject, the proposed method successfully reconstructed all patient surfaces, filling regions where raw point coordinate readings were missing. Within two comparable regions of interest in the chest area, similar mean curvature distributions were acquired from both their reconstructed surface and CT surface, with mean and standard deviation of (μ{sub recon} = − 2.7 × 10{sup −3} mm{sup −1}, σ{sub recon} = 7.0 × 10{sup −3} mm{sup −1}) and (μ{sub CT} = − 2.5 × 10{sup −3} mm{sup −1}, σ{sub CT} = 5.3 × 10{sup −3} mm{sup −1}), respectively. The agreement of local geometry properties between the reconstructed surfaces and the CT surface demonstrated the ability of the proposed method in faithfully representing the underlying patient surface. Conclusions: The authors have integrated and developed an accurate level-set based continuous surface reconstruction method on point clouds acquired by a 3D surface photogrammetry system. The proposed method has generated a continuous representation of the underlying phantom and patient surfaces with good robustness against noise and missing measurements. It serves as an important first step for further development of motion tracking methods during radiotherapy.« less
  • Efficient detection and high-fidelity quantification of surface changes resulting from underground activities are important national and global security efforts. In this investigation, a team performed field-based topographic characterization by gathering high-quality photographs at very low altitudes from an unmanned aerial system (UAS)-borne camera platform. The data collection occurred shortly before and after a controlled underground chemical explosion as part of the United States Department of Energy’s Source Physics Experiments (SPE-5) series. The high-resolution overlapping photographs were used to create 3D photogrammetric models of the site, which then served to map changes in the landscape down to 1-cm-scale. Separate models weremore » created for two areas, herein referred to as the test table grid region and the nearfield grid region. The test table grid includes the region within ~40 m from surface ground zero, with photographs collected at a flight altitude of 8.5 m above ground level (AGL). The near-field grid area covered a broader area, 90–130 m from surface ground zero, and collected at a flight altitude of 22 m AGL. The photographs, processed using Agisoft Photoscan® in conjunction with 125 surveyed ground control point targets, yielded a 6-mm pixel-size digital elevation model (DEM) for the test table grid region. This provided the ≤3 cm resolution in the topographic data to map in fine detail a suite of features related to the underground explosion: uplift, subsidence, surface fractures, and morphological change detection. The near-field grid region data collection resulted in a 2-cm pixel-size DEM, enabling mapping of a broader range of features related to the explosion, including: uplift and subsidence, rock fall, and slope sloughing. This study represents one of the first works to constrain, both temporally and spatially, explosion-related surface damage using a UAS photogrammetric platform; these data will help to advance the science of underground explosion detection.« less
  • It is essential to obtain realistic brain surface images, in which sulci and gyri are easily recognized, when examining the correlation between functional (PET or SPECT) and anatomical (MRI) brain studies. The volume rendering technique (VRT) is commonly employed to make three-dimensional (3D) brain surface images. This technique, however, takes considerable time to make only one 3D image. Therefore it has not been practical to make the brain surface images in arbitrary directions on a real-time basis using ordinary work stations or personal computers. The surface rendering technique (SRT), on the other hand, is much less computationally demanding, but themore » quality of resulting images is not satisfactory for our purpose. A new computer algorithm has been developed to make 3D brain surface MR images very quickly using a volume-surface rendering technique (VSRT), in which the quality of resulting images is comparable to that of VRT and computation time to SRT. In VSRT the process of volume rendering is done only once to the direction of the normal vector of each surface point, rather than each time a new view point is determined as in VRT. Subsequent reconstruction of the 3D image uses a similar algorithm to that of SRT. Thus we can obtain brain surface MR images of sufficient quality viewed from any direction on a real-time basis using an easily available personal computer (Macintosh Quadra 800). The calculation time to make a 3D image is less than 1 sec. in VSRT, while that is more than 15 sec. in the conventional VRT. The difference of resulting image quality between VSRT and VRT is almost imperceptible. In conclusion, our new technique for real-time reconstruction of 3D brain surface MR image is very useful and practical in the functional and anatomical correlation study.« less
  • An algorithm is presented that allows for the control of multileaf collimation (MLC) leaves based entirely on real-time calculations of the intensity delivered over the target. The algorithm is capable of efficiently correcting generalized delivery errors without requiring the interruption of delivery (self-correcting trajectories), where a generalized delivery error represents anything that causes a discrepancy between the delivered and intended intensity profiles. The intensity actually delivered over the target is continually compared to its intended value. For each pair of leaves, these comparisons are used to guide the control of the following leaf and keep this discrepancy below a user-specifiedmore » value. To demonstrate the basic principles of the algorithm, results of corrected delivery are shown for a leading leaf positional error during dynamic-MLC (DMLC) IMRT delivery over a rigid moving target. It is then shown that, with slight modifications, the algorithm can be used to track moving targets in real time. The primary results of this article indicate that the algorithm is capable of accurately delivering DMLC IMRT over a rigid moving target whose motion is (1) completely unknown prior to delivery and (2) not faster than the maximum MLC leaf velocity over extended periods of time. These capabilities are demonstrated for clinically derived intensity profiles and actual tumor motion data, including situations when the target moves in some instances faster than the maximum admissible MLC leaf velocity. The results show that using the algorithm while calculating the delivered intensity every 50 ms will provide a good level of accuracy when delivering IMRT over a rigid moving target translating along the direction of MLC leaf travel. When the maximum velocities of the MLC leaves and target were 4 and 4.2 cm/s, respectively, the resulting error in the two intensity profiles used was 0.1{+-}3.1% and -0.5{+-}2.8% relative to the maximum of the intensity profiles. For the same target motion, the error was shown to increase rapidly as (1) the maximum MLC leaf velocity was reduced below 75% of the maximum target velocity and (2) the system response time was increased.« less
  • Propagating the equations of motion (EOM) for the one-electron reduced-density matrix (1-RDM) requires knowledge of the corresponding two-electron RDM (2-RDM). We show that the indeterminacy of this expression can be removed through a constrained optimization that resembles the variational optimization of the ground-state 2-RDM subject to a set of known N-representability conditions. Electronic excitation energies can then be obtained by propagating the EOM for the 1-RDM and following the dipole moment after the system interacts with an oscillating external electric field. For simple systems with well-separated excited states whose symmetry differs from that of the ground state, excitation energies obtainedmore » from this method are comparable to those obtained from full configuration interaction computations. Although the optimized 2-RDM satisfies necessary N-representability conditions, the procedure cannot guarantee a unique mapping from the 1-RDM to the 2-RDM. This deficiency is evident in the mean-field-quality description of transitions to states of the same symmetry as the ground state, as well as in the inability of the method to describe Rabi oscillations.« less