Extreme-Scale De Novo Genome Assembly

Georganas, Evangelos; Hofmeyr, Steven; Egan, Rob; Buluc, Aydin; Oliker, Leonid; Rokhsar, Daniel; Yelick, Katherine

doi:10.1201/b21930

Title: Extreme-Scale De Novo Genome Assembly

Full Record
Other Related Research

Abstract

De novo whole genome assembly reconstructs genomic sequence from short, overlapping, and potentially erroneous DNA segments and is one of the most important computations in modern genomics. This work presents HipMER, a high-quality end-to-end de novo assembler designed for extreme scale analysis, via efficient parallelization of the Meraculous code. Genome assembly software has many components, each of which stresses different components of a computer system. This chapter explains the computational challenges involved in each step of the HipMer pipeline, the key distributed data structures, and communication costs in detail. We present performance results of assembling the human genome and the large hexaploid wheat genome on large supercomputers up to tens of thousands of cores.

Authors:

Georganas, Evangelos ^[1]; Hofmeyr, Steven ^[2]; Egan, Rob ^[3]; Buluc, Aydin ^[2]; Oliker, Leonid ^[2]; Rokhsar, Daniel ^[3]; Yelick, Katherine ^[2]

Intel Corporation, Santa Clara, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Joint Genome Inst.
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division

Publication Date:: 2017-09-26

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

OSTI Identifier:: 1398520

Grant/Contract Number:: AC02-05CH11231

Resource Type:: Accepted Manuscript

Resource Relation:: Related Information: Chapter in Exascale Scientific Applications: Programming Approaches for Scalability, Performance, and Portability, Straatsma, Antypas, Williams (eds.)

Country of Publication:: United States

Language:: English

Subject:: 60 APPLIED LIFE SCIENCES; 59 BASIC BIOLOGICAL SCIENCES

Citation Formats


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, and Yelick, Katherine. Extreme-Scale De Novo Genome Assembly.  United States: N. p., 2017. 
        Web.  doi:10.1201/b21930.

Copy to clipboard


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, & Yelick, Katherine. Extreme-Scale De Novo Genome Assembly.  United States.  doi:https://doi.org/10.1201/b21930

Copy to clipboard


                    Georganas, Evangelos, Hofmeyr, Steven, Egan, Rob, Buluc, Aydin, Oliker, Leonid, Rokhsar, Daniel, and Yelick, Katherine. Tue .  
        "Extreme-Scale De Novo Genome Assembly".  United States.  doi:https://doi.org/10.1201/b21930.  https://www.osti.gov/servlets/purl/1398520.

Copy to clipboard


                    
@article{osti_1398520,

  title        = {Extreme-Scale De Novo Genome Assembly},

  author       = {Georganas, Evangelos and Hofmeyr, Steven and Egan, Rob and Buluc, Aydin and Oliker, Leonid and Rokhsar, Daniel and Yelick, Katherine},

  abstractNote = {De novo whole genome assembly reconstructs genomic sequence from short, overlapping, and potentially erroneous DNA segments and is one of the most important computations in modern genomics. This work presents HipMER, a high-quality end-to-end de novo assembler designed for extreme scale analysis, via efficient parallelization of the Meraculous code. Genome assembly software has many components, each of which stresses different components of a computer system. This chapter explains the computational challenges involved in each step of the HipMer pipeline, the key distributed data structures, and communication costs in detail. We present performance results of assembling the human genome and the large hexaploid wheat genome on large supercomputers up to tens of thousands of cores.},

  doi          = {10.1201/b21930},

  journal      = {},

  number       = ,

  volume       = ,

  place        = {United States},

  year         = {2017},

  month        = {9}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

DOI: https://doi.org/10.1201/b21930

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Extreme Scale De Novo Metagenome Assembly

Conference Georganas, Evangelos ; Egan, Rob ; Hofmeyr, Steven ; ...

Metagenome assembly is the process of transforming a set of short, overlapping, and potentially erroneous DNA segments from environmental samples into the accurate representation of the underlying microbiomes's genomes. State-of-the-art tools require big shared memory machines and cannot handle contemporary metagenome datasets that exceed Terabytes in size. In this paper, we introduce the MetaHipMer pipeline, a high-quality and high-performance metagenome assembler that employs an iterative de Bruijn graph approach. MetaHipMer leverages a specialized scaffolding algorithm that produces long scaffolds and accommodates the idiosyncrasies of metagenomes. MetaHipMer is end-to-end parallelized using the Unified Parallel C language and therefore can run seamlesslymore »« less
DOI: https://doi.org/10.1109/sc.2018.00013
Optimizing de novo genome assembly from PCR-amplified metagenomes

Journal Article Roux, Simon ; Trubl, Gareth ; Goudeau, Danielle ; ...

Background. Metagenomics has transformed our understanding of microbial diversity across ecosystems, with recent advances enabling de novo assembly of genomes from metagenomes. These metagenome-assembled genomes are critical to provide ecological, evolutionary, and metabolic context for all the microbes and viruses yet to be cultivated. Metagenomes can now be generated from nanogram to subnanogram amounts of DNA. However, these libraries require several rounds of PCR amplification before sequencing, and recent data suggest these typically yield smaller and more fragmented assemblies than regular metagenomes. Methods. Here we evaluate de novo assembly methods of 169 PCR-amplified metagenomes, including 25 for which an unamplifiedmore »« less
DOI: https://doi.org/10.7287/peerj.preprints.27453
Optimizing de novo genome assembly from PCR-amplified metagenomes

Journal Article Roux, Simon ; Trubl, Gareth ; Goudeau, Danielle ; ...

Background: Metagenomics has transformed our understanding of microbial diversity across ecosystems, with recent advances enabling de novo assembly of genomes from metagenomes. These metagenome-assembled genomes are critical to provide ecological, evolutionary, and metabolic context for all the microbes and viruses yet to be cultivated. Metagenomes can now be generated from nanogram to subnanogram amounts of DNA. However, these libraries require several rounds of PCR amplification before sequencing, and recent data suggest these typically yield smaller and more fragmented assemblies than regular metagenomes. Methods: Here we evaluate de novo assembly methods of 169 PCR-amplified metagenomes, including 25 for which an unamplifiedmore »« less
DOI: https://doi.org/10.7287/peerj.preprints.27453v1
Optimizing de novo genome assembly from PCR-amplified metagenomes

Journal Article Roux, Simon ; Trubl, Gareth ; Goudeau, Danielle ; ... - PeerJ

Background. Metagenomics has transformed our understanding of microbial diversity across ecosystems, with recent advances enabling de novo assembly of genomes from metagenomes. These metagenome-assembled genomes are critical to provide ecological, evolutionary, and metabolic context for all the microbes and viruses yet to be cultivated. Metagenomes can now be generated from nanogram to subnanogram amounts of DNA. However, these libraries require several rounds of PCR amplification before sequencing, and recent data suggest these typically yield smaller and more fragmented assemblies than regular metagenomes. Methods. Here we evaluate de novo assembly methods of 169 PCR-amplified metagenomes, including 25 for which an unamplifiedmore »« less
Cited by 5
DOI: https://doi.org/10.7717/peerj.6902

Full Text Available
De novo Assembly of a 40 Mb Eukaryotic Genome from Short Sequence Reads: Sordaria macrospora, a Model Organism for Fungal Morphogenesis

Journal Article Nowrousian, Minou ; Stajich, Jason E. ; Chu, Meiling ; ... - PLoS Genetics

Filamentous fungi are of great importance in ecology, agriculture, medicine, and biotechnology. Thus, it is not surprising that genomes for more than 100 filamentous fungi have been sequenced, most of them by Sanger sequencing. While nextgeneration sequencing techniques have revolutionized genome resequencing, e.g. for strain comparisons, genetic mapping, or transcriptome and ChIP analyses, de novo assembly of eukaryotic genomes still presents significant hurdles, because of their large size and stretches of repetitive sequences. Filamentous fungi contain few repetitive regions in their 30–90 Mb genomes and thus are suitable candidates to test de novo genome assembly from short sequence reads. Here,more »« less
DOI: https://doi.org/10.1371/journal.pgen.1000891

Full Text Available

Similar Records