Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

Journal Article · · Nature Methods
 [1];  [2];  [3];  [4];  [5];  [1];  [6];  [7];  [1];  [8];  [9];  [10];  [11];  [12];  [13];  [14];  [15];  [16];  [17];  [18] more »;  [19];  [20];  [21];  [22];  [23];  [24];  [25];  [25];  [26];  [27];  [7];  [28];  [29];  [30];  [31];  [32];  [33];  [34];  [35];  [36];  [37];  [38];  [39];  [40];  [41];  [42];  [43];  [44];  [10];  [45];  [46];  [47];  [48];  [10];  [49];  [42];  [50];  [51];  [52];  [53];  [44];  [44];  [54];  [7];  [1] « less
  1. Southeast University, Nanjing (China)
  2. Microsoft Corporation, Redmond, WA (United States)
  3. Tencent AI Lab, Bellevue, WA (United States)
  4. Argonne National Laboratory (ANL), Argonne, IL (United States)
  5. Kaya Medical, Seattle, WA (United States)
  6. University of Cassino and Southern Lazio (Italy)
  7. George Mason University, Fairfax, VA (United States)
  8. Beijing University of Technology (China); Beijing International Collaboration Base on Brain Informatics and Wisdom Services (China)
  9. Nuctech Netherlands, Rotterdam (Netherlands)
  10. Howard Hughes Medical Institute, Ashburn, VA (United States)
  11. University of Alberta, Edmonton, AB (Canada)
  12. Southeast University, Nanjing (China); Anhui University, Hefei (China)
  13. Paige AI, New York, NY (United States)
  14. Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); University of California, Berkeley, CA (United States)
  15. RIKEN, Tokyo (Japan). Center for Advanced Integrated Intelligence Platform Project (AIP); The University of Tokyo (Japan)
  16. University of Sydney, NSW (Australia)
  17. Texas A & M University, College Station, TX (United States)
  18. University of Houston, TX (United States)
  19. National Taiwan University of Science and Technology, Taipei (Taiwan)
  20. Bournemouth University, Poole (United Kingdom)
  21. PROPHESEE, Paris (France)
  22. Columbia University, New York, NY (United States)
  23. Northern Illinois University, DeKalb, IL (United States)
  24. AWS AI, Seattle, WA (United Sttaes)
  25. Universidade Do Porto (Portugal)
  26. Massachusetts General Hospital and Harvard Medical School, Boston, MA (United States)
  27. Allen Institute for Brain Science, Seattle, WA (United States); The University of Georgia, Athens, GA (United States)
  28. Florida State University College of Medicine, Tallahassee, FL (United States)
  29. University of Cambridge (United Kingdom)
  30. Ecole Polytechnique Federale Lausanne (EPFL) (Switzerland)
  31. University of Michigan, Ann Arbor, MI (United States)
  32. Georgetown University, Washington, DC (United States)
  33. Georg-August-University Göttingen (Germany)
  34. Harvard University, Cambridge, MA (United States)
  35. Georgia State University, Atlanta, GA (United States)
  36. 42 ENS Paris-Saclay, CNRS, CentraleSupélec, LuMIn, Université Paris-Saclay, Gif-sur-Yvette (France)
  37. Max Planck Institute for Neurobiology of Behavior – caesar, Bonn (Germany)
  38. Howard Hughes Medical Institute, Ashburn, VA (United States); MRC Laboratory of Molecular Biology, Cambridge (United Kingdom); University of Cambridge (United Kingdom)
  39. Howard Hughes Medical Institute, Ashburn, VA (United States); University of Tokyo (Japan); University of Cologne (Germany)
  40. University of Minnesota, St Paul, MN (United States)
  41. Korea Institute of Science and Technology (KIST), Seoul (Korea, Republic of)
  42. University of Washington, Seattle, WA (United States)
  43. The Scripps Research Institute, La Jolla, CA (United States)
  44. Allen Institute for Brain Science, Seattle, WA (United States)
  45. National Tsing Hua University, Hsinchu (Taiwan)
  46. Tencent AI Lab, Shenzhen (China)
  47. Allen Institute for Brain Science, Seattle, WA (United States); University of British Columbia, Vancouver, BC (Canada)
  48. UCL Great Ormond Street Institute of Child Health, London (United Kingdom)
  49. The University of Georgia, Athens, GA (United States)
  50. Imperial College London (United Kingdom)
  51. Beijing University of Technology (China); Beijing International Collaboration Base on Brain Informatics and Wisdom Services (China); Maebashi Institute of Technology (Japan)
  52. Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). BioEnergy Science Center (BESC)
  53. Centre for Addiction and Mental Health, Toronto, ON (Canada); University of Toronto, Toronto, ON (Canada)
  54. University of New South Wales, Sydney, NSW (Australia)
+ Show Author Affiliations
BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.
Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
National Institutes of Health (NIH); USDOE Office of Science (SC)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
2229298
Journal Information:
Nature Methods, Journal Name: Nature Methods Journal Issue: 6 Vol. 20; ISSN 1548-7091
Publisher:
Nature Publishing GroupCopyright Statement
Country of Publication:
United States
Language:
English

References (92)

BigNeuron fMOST showcase image data dataset January 2023
Intense and specialized dendritic localization of the fragile X mental retardation protein in binaural brainstem neurons: A comparative study in the alligator, chicken, gerbil, and human journal April 2014
Neuron tracing in perspective journal March 2010
Shortest Connection Networks And Some Generalizations journal November 1957
Quality Control for High-Throughput Imaging Experiments Using Machine Learning in Cellprofiler book October 2017
Morphological determinants of dendritic arborization neurons in Drosophila larva journal November 2017
Principal Curves as Skeletons of Tubular Objects journal February 2011
A Brief History of Neuronal Reconstruction journal February 2011
A Broadly Applicable 3-D Neuron Tracing Method Based on Open-Curve Snake journal March 2011
The DIADEM Metric: Comparing Multiple Reconstructions of the Same Neuron journal April 2011
Automated Reconstruction of Neuronal Morphology Based on Local Geometrical and Global Structural Models journal May 2011
Automatic Morphological Reconstruction of Neurons from Multiphoton and Confocal Microscopy Images Using 3D Tubular Models journal January 2015
From DIADEM to BigNeuron journal April 2015
TReMAP: Automatic 3D Neuron Reconstruction Based on Tracing, Reverse Mapping and Assembling of 2D Projections journal August 2015
Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking journal May 2016
Ensemble Neuron Tracer for 3D Neuron Reconstruction journal February 2017
Integrating the Allen Brain Institute Cell Types Database into Automated Neuroscience Workflow journal August 2017
A Topological Representation of Branching Neuronal Morphologies journal October 2017
FMST: an Automatic Neuron Tracing Method Based on Fast Marching and Minimum Spanning Tree journal July 2018
Automated Neuron Reconstruction from 3D Fluorescence Microscopy Images Using Sequential Monte Carlo Estimation journal December 2018
Petabyte-Scale Multi-Morphometry of Single Neurons for Whole Brains journal February 2022
SmartTracing: self-learning-based Neuron reconstruction journal August 2015
Reconstructing the brain: from image stacks to neuron synthesis journal February 2016
Fast assembling of neuron fragments in serial 3D sections journal April 2017
An automatic neuron reconstruction system journal August 1983
A system for quantitative morphological measurement and electrotonic modelling of neurons: three-dimensional reconstruction journal April 1993
A Suite of Transgenic Driver and Reporter Mouse Lines with Enhanced Brain-Cell-Type Targeting and Functionality journal July 2018
Reconstruction of 1,000 Projection Neurons Reveals New Cell Types and Organization of Long-Range Connectivity in the Mouse Brain journal September 2019
Super-Resolution Mapping of Neuronal Circuitry With an Index-Optimized Clearing Agent journal March 2016
Dissecting Sholl Analysis into Its Functional Components journal June 2019
Three-Dimensional Reconstruction of Brain-wide Wiring Networks in Drosophila at Single-Cell Resolution journal January 2011
Visual Space Is Represented by Nonmatching Topographies of Distinct Mouse Retinal Ganglion Cell Types journal February 2014
Reconstruction and simulation for three-dimensional morphological structure of insect neurons journal June 2006
Differential alterations in the morphology and electrophysiology of layer II pyramidal cells in the primary visual cortex of a mouse model prenatally exposed to LPS journal March 2015
Large scale similarity search across digital reconstructions of neural morphology journal August 2022
3D BrainCV: Simultaneous visualization and analysis of cells and capillaries in a whole mouse brain with one-micron voxel resolution journal February 2014
In Vivo Imaging Reveals Dendritic Targeting of Laminated Afferents by Zebrafish Retinal Ganglion Cells journal November 2006
The Brain Activity Map Project and the Challenge of Functional Connectomics journal June 2012
Neuronal Morphology Goes Digital: A Research Hub for Cellular and System Neuroscience journal March 2013
Structured Synaptic Connectivity between Hippocampal Regions journal February 2014
BigNeuron: Large-Scale 3D Neuron Reconstruction from Optical Microscopy Images journal July 2015
Experience-Dependent Bimodal Plasticity of Inhibitory Neurons in Early Development journal June 2016
NBLAST: Rapid, Sensitive Comparison of Neuronal Structure and Construction of Neuron Family Databases journal July 2016
The BRAIN Initiative Cell Census Consortium: Lessons Learned toward Generating a Comprehensive Brain Cell Atlas journal November 2017
Bridging faultlines by valuing diversity: Diversity beliefs, information elaboration, and performance in diverse work groups. journal January 2007
Imagining the future of bioimage analysis journal December 2016
High-throughput dual-colour precision imaging for brain-wide connectome with cytoarchitectonic landmarks at the cellular level journal July 2016
Transmission from the dominant input shapes the stereotypic ratio of photoreceptor inputs onto horizontal cells journal May 2014
Improved tools for the Brainbow toolbox journal May 2013
CLARITY for mapping the nervous system journal June 2013
Automatic tracing of ultra-volumes of neuronal images journal March 2017
High-resolution three-dimensional imaging of large specimens with light sheet–based microscopy journal March 2007
Scale: a chemical approach for fluorescence imaging and reconstruction of transparent mouse brain journal August 2011
Human cerebral cortex development from pluripotent stem cells to functional excitatory synapses journal February 2012
Whole-body tissue stabilization and selective extractions via tissue-hydrogel hybrids for high-resolution intact circuit mapping and phenotyping journal October 2015
Neuronal cell-type classification: challenges, opportunities and the path forward journal August 2017
The Blue Brain Project journal February 2006
Neuroscience thinks big (and collaboratively) journal August 2013
Why rankings of biomedical image analysis competitions should be interpreted with care journal December 2018
Precise segmentation of densely interweaving neuron clusters using G-Cut journal April 2019
TeraVR empowers precise reconstruction of complete 3-D neuronal morphology in the whole brain journal August 2019
Morphological diversity of single neurons in molecularly defined cell types journal October 2021
Cellular anatomy of the mouse primary motor cortex journal October 2021
High-definition imaging using line-illumination modulation microscopy journal March 2021
An open repository for single-cell reconstructions of the brain forest journal February 2018
NeuroGPS: automated localization of neurons for brain circuits using L1 minimization model journal April 2013
Tools for neuroanatomy and neurogenetics in Drosophila journal July 2008
Optimized tools for multicolor stochastic labeling reveal diverse stereotyped cell arrangements in the fly visual system journal May 2015
Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model journal June 2010
Automatic 3D neuron tracing using all-path pruning journal June 2011
APP2: automatic tracing of 3D neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree journal April 2013
Characterizing VIP Neurons in the Barrel Cortex of VIPcre/tdTomato Mice Reveals Layer-Specific Differences journal September 2015
Learning to Boost Filamentary Structure Segmentation conference December 2015
Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction journal July 2017
Automated 3-D Neuron Tracing With Precise Branch Erasing and Confidence Controlled Back Tracking journal November 2018
Multiscale Centerline Detection journal July 2016
Optical Sectioning Deep Inside Live Embryos by Selective Plane Illumination Microscopy journal August 2004
Micro-Optical Sectioning Tomography to Obtain a High-Resolution Atlas of the Mouse Brain journal December 2010
Development of a Scheme and Tools to Construct a Standard Moth Brain for Neural Network Simulations journal January 2012
A distance-field based automatic neuron tracing method journal March 2013
M-AMST: an automatic 3D neuron tracing method based on mean shift and adapted minimum spanning tree journal March 2017
Estimating the Dimension of a Model journal March 1978
Visualization of brain circuits using two-photon fluorescence micro-optical sectioning tomography journal April 2013
High-throughput Computer Method for 3D Neuronal Structure Reconstruction from the Image Stack of the Drosophila Brain and Its Applications journal September 2012
Transgenic Mouse Models Enabling Photolabeling of Individual Neurons In Vivo journal April 2013
Metrics for comparing neuronal tree shapes based on persistent homology journal August 2017
Neuronize: a tool for building realistic neuronal cell morphologies journal January 2013
Statistical analysis and data mining of digital reconstructions of dendritic morphologies journal December 2014
NeuTu: Software for Collaborative, Large-Scale, Segmentation-Based Connectome Reconstruction journal November 2018
Semi-Supervised Learning in Medical Images Through Graph-Embedded Random Forest journal November 2020
Visualization of brain circuits using two-photon fluorescence micro-optical sectioning tomography collection January 2017
A searchable image resource of Drosophila GAL4 driver expression patterns with single neuron resolution journal February 2023

Similar Records

Resolving local ambiguity using semantics of shape.
Conference · Sat May 01 00:00:00 EDT 2010 · OSTI ID:1012748
An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models
Journal Article · Tue Jun 08 20:00:00 EDT 2021 · Integrating Materials and Manufacturing Innovation · OSTI ID:1798722
Automated Tracing and Segmentation Tool for Migrating Neurons in 4D Confocal Imagery
Conference · Mon Dec 31 23:00:00 EST 2012 · OSTI ID:1092287

Related Subjects

97 MATHEMATICS AND COMPUTING
Computational neuroscience
Computational platforms and environments