Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data

Journal Article · · G3
 [1];  [2];  [2];  [2];  [2];  [2];  [2];  [2]
  1. Univ. of Wisconsin, Madison, WI (United States); OSTI
  2. Univ. of Wisconsin, Madison, WI (United States)
+ Show Author Affiliations
Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on seven published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.
Research Organization:
Univ. of Wisconsin, Madison, WI (United States)
Sponsoring Organization:
National Institutes of Health (NIH); National Science Foundation (NSF); USDOE Office of Science (SC)
Grant/Contract Number:
SC0021052
OSTI ID:
2472383
Journal Information:
G3, Journal Name: G3 Journal Issue: 3 Vol. 13; ISSN 2160-1836
Publisher:
Genetics Society of AmericaCopyright Statement
Country of Publication:
United States
Language:
English

References (60)

Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data dataset January 2019
TBX3 Promotes Human Embryonic Stem Cell Proliferation and Neuroepithelial Differentiation in a Differentiation Stage-Dependent Manner journal September 2012
Core Transcriptional Regulatory Circuitry in Human Embryonic Stem Cells journal September 2005
An Extended Transcriptional Network for Pluripotency of Embryonic Stem Cells journal March 2008
Control of the Embryonic Stem Cell State journal March 2011
Single-Cell Expression Analyses during Cellular Reprogramming Reveal an Early Stochastic and a Late Hierarchic Phase journal September 2012
Determination and Inference of Eukaryotic Transcription Factor Sequence Specificity journal September 2014
Perturb-Seq: Dissecting Molecular Circuits with Scalable Single-Cell RNA Profiling of Pooled Genetic Screens journal December 2016
Recovering Gene Interactions from Single-Cell Data Using Data Diffusion journal July 2018
Nac1 Coordinates a Sub-network of Pluripotency Factors to Regulate Embryonic Stem Cell Differentiation journal February 2016
Defining Reprogramming Checkpoints from Single-Cell Analyses of Induced Pluripotency journal May 2019
Functional and Evolutionary Diversification of Otx2 and Crx in Vertebrate Retinal Photoreceptor and Bipolar Cell Development journal January 2020
Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures journal September 2017
Single-Cell RNA-Seq Reveals Dynamic Early Embryonic-like Programs during Chemical Reprogramming journal July 2018
An integrated encyclopedia of DNA elements in the human genome journal September 2012
Discovery of directional and nondirectional pioneer transcription factors by modeling DNase profile magnitude and shape journal January 2014
Reversed graph embedding resolves complex single-cell trajectories journal August 2017
SCENIC: single-cell regulatory network inference and clustering journal October 2017
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data journal January 2020
A gene regulatory network in mouse embryonic stem cells journal October 2007
Network component analysis: Reconstruction of regulatory signals in biological systems journal December 2003
Genomic Expression Programs in the Response of Yeast Cells to Environmental Changes journal December 2000
Mapping gene regulatory networks from single-cell omics data journal January 2018
A comprehensive survey of regulatory network inference methods using single cell RNA sequencing data journal September 2020
Inferring TF activities and activity regulators from gene expression data with constraints from TF perturbation data journal December 2020
CIS: compound importance sampling method for protein–DNA binding site p-value estimation journal September 2004
Robust data-driven incorporation of prior knowledge into the inference of dynamic regulatory networks journal March 2013
LEAP: constructing gene co-expression networks for single-cell RNA-sequencing data using pseudotime ordering journal December 2016
SCODE: an efficient regulatory network inference algorithm from single-cell RNA-Seq during differentiation journal April 2017
YeTFaSCo: a database of evaluated yeast transcription factor sequence specificities journal November 2011
Systematic discovery and characterization of regulatory motifs in ENCODE TF binding experiments journal December 2013
A prior-based integrative framework for functional transcriptional regulatory network inference journal November 2016
JASPAR 2018: update of the open-access database of transcription factor binding profiles and its web framework journal November 2017
Defining an essential transcription factor program for naive pluripotency journal June 2014
Conditional density-based analysis of T cell signaling in single-cell data journal November 2014
GATA-3 regulates hematopoietic stem cell maintenance and cell-cycle entry journal March 2012
TIGRESS: Trustful Inference of Gene REgulation using Stability Selection journal January 2012
Curated collection of yeast transcription factor DNA binding specificity data reveals novel structural and gene regulatory insights journal January 2011
BTR: training asynchronous Boolean models using single-cell expression data journal September 2016
Evaluating methods of inferring gene regulatory networks highlights their lack of performance for single cell gene expression data journal June 2018
Metacells untangle large and complex single-cell transcriptome networks journal August 2022
Mapping human pluripotent stem cell differentiation pathways using high throughput single-cell RNA-sequencing journal April 2018
MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions journal October 2019
A systematic evaluation of single-cell RNA-sequencing imputation methods journal August 2020
Graphical models for zero-inflated single cell gene expression journal June 2019
Cdx2 is required for correct cell fate specification and differentiation of trophectoderm in the mouse blastocyst journal May 2005
Single-cell RNA sequencing reveals intrinsic and extrinsic regulatory heterogeneity in yeast responding to stress journal December 2017
Integrated Module and Gene-Specific Regulatory Inference Implicates Upstream Signaling Networks journal October 2013
Construction and Validation of a Regulatory Network for Pluripotency and Self-Renewal of Mouse Embryonic Stem Cells journal August 2014
Sharing and Specificity of Co-expression Networks across 35 Human Tissues journal May 2015
SILGGM: An extensive R package for efficient statistical inference in large-scale gene networks journal August 2018
Multi-study inference of regulatory networks for more accurate models of gene regulation journal January 2019
Inferring Regulatory Networks from Expression Data Using Tree-Based Methods journal September 2010
Identifying strengths and weaknesses of methods for computational network inference from single cell RNA-seq data dataset January 2022
BTR: training asynchronous Boolean models using single-cell expression data collection January 2016
Mapping human pluripotent stem cell differentiation pathways using high throughput single-cell RNA-sequencing collection January 2018
Evaluating methods of inferring gene regulatory networks highlights their lack of performance for single cell gene expression data collection January 2018
MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions collection January 2019
A systematic evaluation of single-cell RNA-sequencing imputation methods collection January 2020
Gene regulatory network reconstruction using single-cell RNA sequencing of barcoded genotypes in diverse environments journal January 2020

Similar Records

Inference of cell type-specific gene regulatory networks on cell lineages from single cell omic datasets
Journal Article · Fri May 26 20:00:00 EDT 2023 · Nature Communications · OSTI ID:1975415
Deep structural clustering for single-cell RNA-seq data jointly through autoencoder and graph neural network
Journal Article · Tue Feb 15 19:00:00 EST 2022 · Briefings in Bioinformatics · OSTI ID:1845371

Related Subjects

59 BASIC BIOLOGICAL SCIENCES
Genetics & Heredity
benchmarking
gene regulatory networks
network inference algorithms
prior knowledge
single-cell RNA-seq