Parallel String Graph Construction and Transitive Reduction for De Novo Genome Assembly

Guidi, Giulia; Selvitopi, Oguz; Ellis, Marquita; Oliker, Leonid; Yelick, Katherine; Buluc, Aydin

doi:10.1109/ipdps49936.2021.00060

Parallel String Graph Construction and Transitive Reduction for De Novo Genome Assembly

Journal Article · Sat May 01 00:00:00 EDT 2021 · Proceedings - IEEE International Parallel and Distributed Processing Symposium (IPDPS)

DOI:https://doi.org/10.1109/ipdps49936.2021.00060· OSTI ID:1818231

Guidi, Giulia ^[1]; Selvitopi, Oguz ^[2]; Ellis, Marquita ^[1]; Oliker, Leonid ^[2]; Yelick, Katherine ^[2]; Buluc, Aydin ^[1]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

One of the most computationally intensive tasks in computational biology is de novo genome assembly, the decoding of the sequence of an unknown genome from redundant and erroneous short sequences. A common assembly paradigm identifies overlapping sequences, simplifies their layout, and creates consensus. Despite many algorithms developed in the literature, the efficient assembly of large genomes is still an open problem. In this work, we introduce new distributed-memory parallel algorithms for overlap detection and layout simplification steps of de novo genome assembly, and implement them in the diBELLA 2D pipeline. Our distributed memory algorithms for both overlap detection and layout simplification are based on linear-algebra operations over semirings using 2D distributed sparse matrices. Our layout step consists of performing a transitive reduction from the overlap graph to a string graph. We provide a detailed communication analysis of the main stages of our new algorithms. diBELLA 2D achieves near linear scaling with over 80% parallel efficiency for the human genome, reducing the runtime for overlap detection by 1.2-1.3× for the human genome and 1.5-1.9× for C.elegans compared to the state-of-the-art. Our transitive reduction algorithm outperforms an existing distributed-memory implementation by 10.5-13.3× for the human genome and 18-29× for the C. elegans. Our work paves the way for efficient de novo assembly of large genomes using long reads in distributed memory.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1818231

Journal Information:: Proceedings - IEEE International Parallel and Distributed Processing Symposium (IPDPS), Journal Name: Proceedings - IEEE International Parallel and Distributed Processing Symposium (IPDPS); ISSN 1530-2075

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (31)

PacBio Sequencing and Its Applications Rhoads, Anthony; Au, Kin Fai Genomics, Proteomics & Bioinformatics, Vol. 13, Issue 5 https://doi.org/10.1016/j.gpb.2015.08.002	journal	October 2015
FSG: Fast String Graph Construction for De Novo Assembly Bonizzoni, Paola; Vedova, Gianluca Della; Pirola, Yuri Journal of Computational Biology, Vol. 24, Issue 10 https://doi.org/10.1089/cmb.2017.0089	journal	October 2017
Efficient counting of k-mers in DNA sequences using a bloom filter Melsted, Páll; Pritchard, Jonathan K. BMC Bioinformatics, Vol. 12, Issue 1 https://doi.org/10.1186/1471-2105-12-333	journal	August 2011
SeqAn An efficient, generic C++ library for sequence analysis Döring, Andreas; Weese, David; Rausch, Tobias BMC Bioinformatics, Vol. 9, Issue 1 https://doi.org/10.1186/1471-2105-9-11	journal	January 2008
Accurate circular consensus long-read sequencing improves variant detection and assembly of a human genome Wenger, Aaron M.; Peluso, Paul; Rowell, William J. Universität des Saarlandes https://doi.org/10.22028/d291-28704	collection	January 2019
An External-Memory Algorithm for String Graph Construction Bonizzoni, Paola; Della Vedova, Gianluca; Pirola, Yuri Algorithmica, Vol. 78, Issue 2 https://doi.org/10.1007/s00453-016-0165-4	journal	May 2016
Assembling large genomes with single-molecule sequencing and locality-sensitive hashing Berlin, Konstantin; Koren, Sergey; Chin, Chen-Shan Nature Biotechnology, Vol. 33, Issue 6 https://doi.org/10.1038/nbt.3238	journal	May 2015
Genetic variation and the de novo assembly of human genomes Chaisson, Mark J. P.; Wilson, Richard K.; Eichler, Evan E. Nature Reviews Genetics, Vol. 16, Issue 11 https://doi.org/10.1038/nrg3933	journal	October 2015
Accurate circular consensus long-read sequencing improves variant detection and assembly of a human genome Wenger, Aaron M.; Peluso, Paul; Rowell, William J. Nature Biotechnology, Vol. 37, Issue 10 https://doi.org/10.1038/s41587-019-0217-9	journal	August 2019
Parametric Complexity of Sequence Assembly: Theory and Applications to Next Generation Sequencing Nagarajan, Niranjan; Pop, Mihai Journal of Computational Biology, Vol. 16, Issue 7 https://doi.org/10.1089/cmb.2009.0005	journal	July 2009
The fragment assembly string graph Myers, E. W. Bioinformatics, Vol. 21, Issue Suppl 2 https://doi.org/10.1093/bioinformatics/bti1114	journal	September 2005
Minimap and miniasm: fast mapping and de novo assembly for noisy long sequences Li, Heng Bioinformatics, Vol. 32, Issue 14 https://doi.org/10.1093/bioinformatics/btw152	journal	March 2016
Minimap2: pairwise alignment for nucleotide sequences Li, Heng Bioinformatics, Vol. 34, Issue 18 https://doi.org/10.1093/bioinformatics/bty191	journal	May 2018
Efficient de novo assembly of large genomes using compressed data structures Simpson, J. T.; Durbin, R. Genome Research, Vol. 22, Issue 3 https://doi.org/10.1101/gr.126953.111	journal	December 2011
Oxford Nanopore sequencing, hybrid error correction, and de novo assembly of a eukaryotic genome Goodwin, Sara; Gurtowski, James; Ethe-Sayers, Scott Genome Research, Vol. 25, Issue 11 https://doi.org/10.1101/gr.191395.115	journal	October 2015
Canu: scalable and accurate long-read assembly via adaptive k -mer weighting and repeat separation Koren, Sergey; Walenz, Brian P.; Berlin, Konstantin Genome Research, Vol. 27, Issue 5 https://doi.org/10.1101/gr.215087.116	journal	March 2017
SORA: Scalable Overlap-graph Reduction Algorithms for Genome Assembly using Apache Spark in the Cloud Paul, Alexander J.; Lawrence, Dylan; Song, Myoungkyu 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) https://doi.org/10.1109/BIBM.2018.8621546	conference	December 2018
Challenges and Advances in Parallel Sparse Matrix-Matrix Multiplication Buluc, Aydin; Gilbert, John R. 2008 37th International Conference on Parallel Processing (ICPP) https://doi.org/10.1109/ICPP.2008.45	conference	September 2008
Parallel Construction of Bidirected String Graphs for Genome Assembly Jackson, Benjamin G.; Aluru, Srinivas 2008 37th International Conference on Parallel Processing https://doi.org/10.1109/ICPP.2008.70	conference	September 2008
Communication-Efficient Jaccard similarity for High-Performance Distributed Genome Comparisons Besta, Maciej; Kanakagiri, Raghavendra; Mustafa, Harun 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS47924.2020.00118	conference	May 2020
Distributed Many-to-Many Protein Sequence Alignment using Sparse Matrices Selvitopi, Oguz; Ekanayake, Saliya; Guidi, Giulia SC20: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC41405.2020.00079	conference	November 2020
Real-Time DNA Sequencing from Single Polymerase Molecules Eid, John; Fehr, Adrian; Gray, Jeremy Science, Vol. 323, Issue 5910, p. 133-138 https://doi.org/10.1126/science.1162986	journal	January 2009
Indexing compressed text Ferragina, Paolo; Manzini, Giovanni Journal of the ACM, Vol. 52, Issue 4 https://doi.org/10.1145/1082036.1082039	journal	July 2005
Communication optimal parallel multiplication of sparse random matrices Ballard, Grey; Buluc, Aydin; Demmel, James Proceedings of the 25th ACM symposium on Parallelism in algorithms and architectures - SPAA '13 https://doi.org/10.1145/2486159.2486196	conference	January 2013
HipMer: an extreme-scale de novo genome assembler Georganas, Evangelos; Buluç, Aydın; Chapman, Jarrod Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15 https://doi.org/10.1145/2807591.2807664	conference	January 2015
Apache Spark: a unified engine for big data processing Zaharia, Matei; Franklin, Michael J.; Ghodsi, Ali Communications of the ACM, Vol. 59, Issue 11 https://doi.org/10.1145/2934664	journal	October 2016
diBELLA: Distributed Long Read to Long Read Alignment Ellis, Marquita; Guidi, Giulia; Buluç, Aydın Proceedings of the 48th International Conference on Parallel Processing - ICPP 2019 https://doi.org/10.1145/3337821.3337919	conference	January 2019
The Combinatorial BLAS: design, implementation, and applications Buluç, Aydın; Gilbert, John R. The International Journal of High Performance Computing Applications, Vol. 25, Issue 4 https://doi.org/10.1177/1094342011403516	journal	May 2011
Genome assembly forensics: finding the elusive mis-assembly Phillippy, Adam M.; Schatz, Michael C.; Pop, Mihai Genome Biology, Vol. 9, Issue 3 https://doi.org/10.1186/gb-2008-9-3-r55	journal	January 2008
The Oxford Nanopore MinION: delivery of nanopore sequencing to the genomics community Jain, Miten; Olsen, Hugh E.; Paten, Benedict Genome Biology, Vol. 17, Issue 1 https://doi.org/10.1186/s13059-016-1103-0	journal	November 2016
A Practical Comparison of De Novo Genome Assembly Software Tools for Next-Generation Sequencing Technologies Zhang, Wenyu; Chen, Jiajia; Yang, Yang PLoS ONE, Vol. 6, Issue 3 https://doi.org/10.1371/journal.pone.0017915	journal	March 2011

Similar Records

diBELLA 2D: Parallel String Graph Construction and Transitive Reduction for De Novo Assembly (diBELLA 2D) v1.0

Software · Mon Feb 08 19:00:00 EST 2021 · OSTI ID:code-52577

Extreme-Scale De Novo Genome Assembly

Journal Article · Tue Sep 26 00:00:00 EDT 2017 · OSTI ID:1398520

SORA: Scalable Overlap-graph Reduction Algorithms for Genome Assembly using Apache Spark in the Cloud

Conference · Fri Nov 30 23:00:00 EST 2018 · OSTI ID:1557475

Related Subjects

59 BASIC BIOLOGICAL SCIENCES

Parallel String Graph Construction and Transitive Reduction for De Novo Genome Assembly

Citation Formats

References (31)

Similar Records

Related Subjects