Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics

Journal Article · · Computers and Electronics in Agriculture
 [1];  [2];  [3];  [4];  [4];  [4];  [2]
  1. University of Arizona, Tucson, AZ (United States); Donald Danforth Plant Science Center
  2. Purdue University, West Lafayette, LA (United States)
  3. University of Nebraska–Lincoln, NE (United States)
  4. University of Arizona, Tucson, AZ (United States)
+ Show Author Affiliations
In this study, we introduce PlantSegNet, a novel neural network model for instance segmentation of nearby objects with similar geometric structures. Our work addresses the challenges of instance segmentation of plant point clouds, including the difficulty of annotating and labeling point clouds, the loss of local structural information in neural network components, and the generation of large numbers of incorrect small clusters due to poor choices of the loss function. One of the key contributions of our approach is a digital twin of sorghum, i.e., a procedural sorghum model, which was used to generate point clouds of sorghum fields. This allowed us to create a large-scale, annotated, synthetic dataset of sorghum plants that we used to train our PlantSegNet model. We demonstrated the effectiveness of our method in segmenting instances of sorghum leaves grown in outdoor field settings. To the best of our knowledge, this is the first study to address this specific instance segmentation problem for plants grown in such a setting. We compared our proposed method with other state-of-the-art methods for indoor settings, including SGPN and TreePartNet, on both synthetic and real data. Furthermore, our results show that PlantSegNet outperforms these methods regarding accuracy, robustness, and efficiency.
Research Organization:
Donald Danforth Plant Science Center, St. Louis, MO (United States)
Sponsoring Organization:
USDOE Advanced Research Projects Agency - Energy (ARPA-E); USDOE Office of Science (SC), Biological and Environmental Research (BER)
Grant/Contract Number:
AR0000594; AR0001101; SC0020401; SC0023305
OSTI ID:
2339863
Journal Information:
Computers and Electronics in Agriculture, Journal Name: Computers and Electronics in Agriculture Vol. 221; ISSN 0168-1699
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (34)

RegGeoNet: Learning Regular Representations for Large-Scale 3D Point Clouds journal September 2022
Plant-part segmentation using deep learning and multi-view vision journal November 2019
Automatic organ-level point cloud segmentation of maize shoots by integrating high-throughput data acquisition and deep learning journal February 2022
PlantNet: A dual-function point cloud segmentation network for multiple plant species journal February 2022
VIBUS: Data-efficient 3D scene parsing with VIewpoint Bottleneck and Uncertainty-Spectrum modeling journal December 2022
Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots journal May 2021
Dynamic quantification of canopy structure to characterize early plant vigour in wheat genotypes journal June 2016
3D Semantic Parsing of Large-Scale Indoor Spaces conference June 2016
OctNet: Learning Deep 3D Representations at High Resolutions conference July 2017
Multi-view Harmonized Bilinear Network for 3D Object Recognition conference June 2018
SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation conference June 2018
PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding conference June 2019
GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud conference June 2019
Associatively Segmenting Instances and Semantics in Point Clouds conference June 2019
3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans conference June 2019
JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds With Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields conference June 2019
View-GCN: View-Based Graph Convolutional Network for 3D Shape Analysis conference June 2020
Grid-GCN for Fast and Scalable Point Cloud Learning conference June 2020
Stratified Transformer for 3D Point Cloud Segmentation conference June 2022
Point Cloud Pre-training with Natural 3D Structures conference June 2022
3D Point Cloud Instance Segmentation of Lettuce Based on PartNet conference June 2022
Multi-view Convolutional Neural Networks for 3D Shape Recognition conference December 2015
Adaptive Graph Convolution for Point Cloud Analysis conference October 2021
Point Transformer conference October 2021
3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation journal July 2020
MegaStitch: Robust Large-Scale Image Stitching journal January 2022
Deep Learning for 3D Point Clouds: A Survey journal December 2021
PointVST: Self-Supervised Pre-Training for 3D Point Clouds Via View-Specific Point-to-Image Translation journal January 2023
Dynamic Graph CNN for Learning on Point Clouds journal October 2019
TreePartNet journal December 2021
Rhizomorph: The Coordinated Function of Shoots and Roots journal July 2023
Latent L-systems: Transformer-based Tree Generator journal November 2023
3D Tree Reconstruction from Simulated Small Footprint Waveform Lidar journal December 2013
MVS-Pheno: A Portable and Low-Cost Phenotyping Platform for Maize Shoots Using Multiview Stereo 3D Reconstruction journal January 2020

Figures / Tables (13)

Fig. 1(p. 2)figure Fig. 1
Fig. 2(p. 5)figure Fig. 2
Fig. 3(p. 5)figure Fig. 3
Fig. 4(p. 5)figure Fig. 4
Fig. 5(p. 6)figure Fig. 5
Fig. 6(p. 7)figure Fig. 6
Fig. 7(p. 8)figure Fig. 7
Table 1(p. 8)table Table 1
Fig. 8(p. 9)figure Fig. 8
Table 2(p. 9)table Table 2
Fig. 9(p. 10)figure Fig. 9
Table 3(p. 10)table Table 3
Fig. 10(p. 11)figure Fig. 10

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING
Digital twins
Phenotyping
Plant geometry
Point clouds
Procedural modeling
Sorghum