Extreme-Scale De Novo Genome Assembly

doi:10.1201/b21930

Title: Extreme-Scale De Novo Genome Assembly

Full Record
Other Related Research

Abstract

De novo whole genome assembly reconstructs genomic sequence from short, overlapping, and potentially erroneous DNA segments and is one of the most important computations in modern genomics. This work presents HipMER, a high-quality end-to-end de novo assembler designed for extreme scale analysis, via efficient parallelization of the Meraculous code. Genome assembly software has many components, each of which stresses different components of a computer system. This chapter explains the computational challenges involved in each step of the HipMer pipeline, the key distributed data structures, and communication costs in detail. We present performance results of assembling the human genome and the large hexaploid wheat genome on large supercomputers up to tens of thousands of cores.

Authors:

Georganas, Evangelos ^[1]; Hofmeyr, Steven ^[2]; Egan, Rob ^[3]; Buluc, Aydin ^[2]; Oliker, Leonid ^[2]; Rokhsar, Daniel ^[3]; Yelick, Katherine ^[2]

Intel Corporation, Santa Clara, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Joint Genome Inst.
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division

Publication Date:: 2017-09-26

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

OSTI Identifier:: 1398520

Grant/Contract Number:: [AC02-05CH11231]

Resource Type:: Accepted Manuscript

Resource Relation:: [Related Information: Chapter in Exascale Scientific Applications: Programming Approaches for Scalability, Performance, and Portability, Straatsma, Antypas, Williams (eds.)]

Country of Publication:: United States

Language:: English

Subject:: 60 APPLIED LIFE SCIENCES; 59 BASIC BIOLOGICAL SCIENCES

Citation Formats


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, and Yelick, Katherine. Extreme-Scale De Novo Genome Assembly.  United States: N. p., 2017. 
        Web.  doi:10.1201/b21930.

Copy to clipboard


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, & Yelick, Katherine. Extreme-Scale De Novo Genome Assembly.  United States.  doi:10.1201/b21930.

Copy to clipboard


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, and Yelick, Katherine. Tue .  
        "Extreme-Scale De Novo Genome Assembly".  United States.  doi:10.1201/b21930.  https://www.osti.gov/servlets/purl/1398520.

Copy to clipboard


                    
@article{osti_1398520,

  title        = {Extreme-Scale De Novo Genome Assembly},

  author       = {Georganas, Evangelos and Hofmeyr, Steven and Egan, Rob and Buluc, Aydin and Oliker, Leonid and Rokhsar, Daniel and Yelick, Katherine},

  abstractNote = {De novo whole genome assembly reconstructs genomic sequence from short, overlapping, and potentially erroneous DNA segments and is one of the most important computations in modern genomics. This work presents HipMER, a high-quality end-to-end de novo assembler designed for extreme scale analysis, via efficient parallelization of the Meraculous code. Genome assembly software has many components, each of which stresses different components of a computer system. This chapter explains the computational challenges involved in each step of the HipMer pipeline, the key distributed data structures, and communication costs in detail. We present performance results of assembling the human genome and the large hexaploid wheat genome on large supercomputers up to tens of thousands of cores.},

  doi          = {10.1201/b21930},

  journal      = {},

  number       = ,

  volume       = ,

  place        = {United States},

  year         = {2017},

  month        = {9}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

DOI: 10.1201/b21930

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Optimizing de novo genome assembly from PCR-amplified metagenomes

Journal Article Roux, Simon ; Trubl, Gareth ; Goudeau, Danielle ; ... - PeerJ

Background. Metagenomics has transformed our understanding of microbial diversity across ecosystems, with recent advances enabling de novo assembly of genomes from metagenomes. These metagenome-assembled genomes are critical to provide ecological, evolutionary, and metabolic context for all the microbes and viruses yet to be cultivated. Metagenomes can now be generated from nanogram to subnanogram amounts of DNA. However, these libraries require several rounds of PCR amplification before sequencing, and recent data suggest these typically yield smaller and more fragmented assemblies than regular metagenomes. Methods. Here we evaluate de novo assembly methods of 169 PCR-amplified metagenomes, including 25 for which an unamplifiedmore »« less
Cited by 3
DOI: 10.7717/peerj.6902

Full Text Available
in silico Whole Genome Sequencer & Analyzer (iWGS): A Computational Pipeline to Guide the Design and Analysis of de novo Genome Sequencing Studies

Journal Article Zhou, Xiaofan ; Peris, David ; Kominek, Jacek ; ... - G3

The availability of genomes across the tree of life is highly biased toward vertebrates, pathogens, human disease models, and organisms with relatively small and simple genomes. Recent progress in genomics has enabled the de novo decoding of the genome of virtually any organism, greatly expanding its potential for understanding the biology and evolution of the full spectrum of biodiversity. The increasing diversity of sequencing technologies, assays, and de novo assembly algorithms have augmented the complexity of de novo genome sequencing projects in nonmodel organisms. To reduce the costs and challenges in de novo genome sequencing projects and streamline their experimentalmore »« less
Cited by 7
DOI: 10.1534/g3.116.034249

Full Text Available
De novo Nanopore read quality improvement using deep learning

Journal Article LaPierre, Nathan ; Egan, Rob ; Wang, Wei ; ... - BMC Bioinformatics

BACKGROUND: Long read sequencing technologies such as Oxford Nanopore can greatly decrease the complexity of de novo genome assembly and large structural variation identification. Currently Nanopore reads have high error rates, and the errors often cluster into low-quality segments within the reads. The limited sensitivity of existing read-based error correction methods can cause large-scale mis-assemblies in the assembled genomes, motivating further innovation in this area. RESULTS: Here we developed a Convolutional Neural Network (CNN) based method, called MiniScrub, for identification and subsequent "scrubbing" (removal) of low-quality Nanopore read segments to minimize their interference in downstream assembly process. MiniScrub first generatesmore »« less
DOI: 10.1186/s12859-019-3103-z

Full Text Available
NovoGraph: Human genome graph construction from multiple long-read de novo assemblies

Journal Article Biederstedt, Evan ; Oliver, Jeffrey C. ; Hansen, Nancy F. ; ... - F1000Research

Genome graphs are emerging as an important novel approach to the analysis of high-throughput human sequencing data. By explicitly representing genetic variants and alternative haplotypes in a mappable data structure, they can enable the improved analysis of structurally variable and hyperpolymorphic regions of the genome. In most existing approaches, graphs are constructed from variant call sets derived from short-read sequencing. As long-read sequencing becomes more cost-effective and enablesde novoassembly for increasing numbers of whole genomes, a method for the direct construction of a genome graph from sets of assembled human genomes would be desirable. Such assembly-based genome graphs would encompassmore »« less
DOI: 10.12688/f1000research.15895.2

Full Text Available
A whole-genome shotgun approach for assembling and anchoring the hexaploid bread wheat genome

Journal Article Chapman, Jarrod A. ; Mascher, Martin ; Buluc, Aydin ; ... - GenomeBiology.com

We report that polyploid species have long been thought to be recalcitrant to whole-genome assembly. By combining high-throughput sequencing, recent developments in parallel computing, and genetic mapping, we derive, de novo, a sequence assembly representing 9.1 Gbp of the highly repetitive 16 Gbp genome of hexaploid wheat, Triticum aestivum, and assign 7.1 Gb of this assembly to chromosomal locations. The genome representation and accuracy of our assembly is comparable or even exceeds that of a chromosome-by-chromosome shotgun assembly. Our assembly and mapping strategy uses only short read sequencing technology and is applicable to any species where it is possible tomore » construct a mapping population.« less
Cited by 74
DOI: 10.1186/s13059-015-0582-8

Full Text Available

Similar Records