Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Yim, Won Cheol; Cushman, John C.

doi:10.7717/peerj.3486

Title: Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Journal Article · Sat Jul 22 00:00:00 EDT 2017 · PeerJ

DOI: https://doi.org/10.7717/peerj.3486 · OSTI ID:1423937

Yim, Won Cheol ^[1]; Cushman, John C. ^[2]

Univ. of Nevada Reno, Reno NV (United States). Dept. of Biochemistry and Molecular Biology; University of Nevada Reno
Univ. of Nevada Reno, Reno NV (United States). Dept. of Biochemistry and Molecular Biology

Bioinformatics is currently faced with very large-scale data sets that lead to computational jobs, especially sequence similarity searches, that can take absurdly long times to run. For example, the National Center for Biotechnology Information (NCBI) Basic Local Alignment Search Tool (BLAST and BLAST+) suite, which is by far the most widely used tool for rapid similarity searching among nucleic acid or amino acid sequences, is highly central processing unit (CPU) intensive. While the BLAST suite of programs perform searches very rapidly, they have the potential to be accelerated. In recent years, distributed computing environments have become more widely accessible and used due to the increasing availability of high-performance computing (HPC) systems. Therefore, simple solutions for data parallelization are needed to expedite BLAST and other sequence analysis tools. However, existing software for parallel sequence similarity searches often requires extensive computational experience and skill on the part of the user. In order to accelerate BLAST and other sequence analysis tools, Divide and Conquer BLAST (DCBLAST) was developed to perform NCBI BLAST searches within a cluster, grid, or HPC environment by using a query sequence distribution approach. Scaling from one (1) to 256 CPU cores resulted in significant improvements in processing speed. Thus, DCBLAST dramatically accelerates the execution of BLAST searches using a simple, accessible, robust, and parallel approach. DCBLAST works across multiple nodes automatically and it overcomes the speed limitation of single-node BLAST programs. DCBLAST can be used on any HPC system, can take advantage of hundreds of nodes, and has no output limitations. Thus, this freely available tool simplifies distributed computation pipelines to facilitate the rapid discovery of sequence similarities between very large data sets.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Univ. of Nevada Reno, Reno NV (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23). Biological Systems Science Division

Grant/Contract Number:: SC0008834

OSTI ID:: 1423937

Journal Information:: PeerJ, Journal Name: PeerJ Vol. 5; ISSN 2167-8359

Publisher:: PeerJ Inc.Copyright Statement

Country of Publication:: United States

Language:: English

References (43)

GridBLAST: a Globus-based high-throughput implementation of BLAST in a Grid computing framework Krishnan, Arun Concurrency and Computation: Practice and Experience, Vol. 17, Issue 13 https://doi.org/10.1002/cpe.906	journal	January 2005
Design and implementation of a CUDA-compatible GPU-based core for gapped BLAST algorithm Ling, Cheng; Benkrid, Khaled Procedia Computer Science, Vol. 1, Issue 1 https://doi.org/10.1016/j.procs.2010.04.053	journal	May 2010
Basic local alignment search tool Altschul, Stephen F.; Gish, Warren; Miller, Webb Journal of Molecular Biology, Vol. 215, Issue 3, p. 403-410 https://doi.org/10.1016/s0022-2836(05)80360-2	journal	October 1990
Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes. Karlin, S.; Altschul, S. F. Proceedings of the National Academy of Sciences, Vol. 87, Issue 6 https://doi.org/10.1073/pnas.87.6.2264	journal	March 1990
GPU-BLAST: using graphics processors to accelerate protein sequence alignment Vouzis, P. D.; Sahinidis, N. V. Bioinformatics, Vol. 27, Issue 2 https://doi.org/10.1093/bioinformatics/btq644	journal	November 2010
ScalaBLAST 2.0: rapid and robust BLAST calculations on multiprocessor systems Oehmen, C. S.; Baxter, D. J. Bioinformatics, Vol. 29, Issue 6 https://doi.org/10.1093/bioinformatics/btt013	journal	January 2013
SOAPdenovo-Trans: de novo transcriptome assembly with short RNA-Seq reads Xie, Y.; Wu, G.; Tang, J. Bioinformatics, Vol. 30, Issue 12 https://doi.org/10.1093/bioinformatics/btu077	journal	February 2014
The Arabidopsis Information Resource (TAIR): improved gene annotation and new tools Lamesch, Philippe; Berardini, Tanya Z.; Li, Donghui Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr1090	journal	December 2011
Scientific workflow applications on Amazon EC2 Juve, Gideon; Deelman, Ewa; Vahi, Karan 2009 5th IEEE International Conference On E-Science Workshops https://doi.org/10.1109/esciw.2009.5408002	conference	December 2009
Compiler transformations for high-performance computing Bacon, David F.; Graham, Susan L.; Sharp, Oliver J. ACM Computing Surveys, Vol. 26, Issue 4 https://doi.org/10.1145/197405.197406	journal	December 1994
HPC-BLAST: distributed BLAST for xeon phi clusters Sawyer, Shane E.; Rekepalli, Bhanu; Horton, Mitchel D. Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics - BCB '15 https://doi.org/10.1145/2808719.2811435	conference	January 2015
SCBI_MapReduce, a New Ruby Task-Farm Skeleton for Automated Parallelisation and Distribution in Chunks of Sequences: The Implementation of a Boosted Blast+ Guerrero-Fernández, Darío; Falgueras, Juan; Claros, M. Gonzalo Computational Biology Journal, Vol. 2013 https://doi.org/10.1155/2013/707540	journal	January 2013
BLAST+: architecture and applications Camacho, Christiam; Coulouris, George; Avagyan, Vahram BMC Bioinformatics, Vol. 10, Issue 1 https://doi.org/10.1186/1471-2105-10-421	journal	January 2009
Accumulation of DNA damage alters microRNA gene transcription in Arabidopsis thaliana Du, Juan; Liu, Yang; Lu, Lu BMC Plant Biology, Vol. 22, Issue 1 https://doi.org/10.1186/s12870-022-03951-9	journal	December 2022
The Grid: A New Infrastructure for 21st Century Science Foster, Ian Grid Computing: Making the Global Infrastructure a Reality https://doi.org/10.1002/0470867167.ch2	book	January 2003
GridBLAST: a Globus-based high-throughput implementation of BLAST in a Grid computing framework Krishnan, Arun Concurrency and Computation: Practice and Experience, Vol. 17, Issue 13 https://doi.org/10.1002/cpe.906	journal	January 2005
Basic local alignment search tool Altschul, Stephen F.; Gish, Warren; Miller, Webb Journal of Molecular Biology, Vol. 215, Issue 3, p. 403-410 https://doi.org/10.1016/S0022-2836(05)80360-2	journal	October 1990
A high-performance MPI implementation on a shared-memory vector supercomputer Gropp, William; Lusk, Ewing Parallel Computing, Vol. 22, Issue 11 https://doi.org/10.1016/S0167-8191(96)00062-2	journal	January 1997
Design and implementation of a CUDA-compatible GPU-based core for gapped BLAST algorithm Ling, Cheng; Benkrid, Khaled Procedia Computer Science, Vol. 1, Issue 1 https://doi.org/10.1016/j.procs.2010.04.053	journal	May 2010
De novo transcript sequence reconstruction from RNA-seq using the Trinity platform for reference generation and analysis Haas, Brian J.; Papanicolaou, Alexie; Yassour, Moran Nature Protocols, Vol. 8, Issue 8 https://doi.org/10.1038/nprot.2013.084	journal	July 2013
Coriander Genomics Database: a genomic, transcriptomic, and metabolic database for coriander Song, Xiaoming; Nie, Fulei; Chen, Wei Horticulture Research, Vol. 7, Issue 1 https://doi.org/10.1038/s41438-020-0261-0	journal	April 2020
The Grid: A New Infrastructure for 21st Century Science Foster, Ian Physics Today, Vol. 55, Issue 2 https://doi.org/10.1063/1.1461327	journal	February 2002
Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes. Karlin, S.; Altschul, S. F. Proceedings of the National Academy of Sciences, Vol. 87, Issue 6 https://doi.org/10.1073/pnas.87.6.2264	journal	March 1990
SPAdes: A New Genome Assembly Algorithm and Its Applications to Single-Cell Sequencing Bankevich, Anton; Nurk, Sergey; Antipov, Dmitry Journal of Computational Biology, Vol. 19, Issue 5 https://doi.org/10.1089/cmb.2012.0021	journal	May 2012
GPU-BLAST: using graphics processors to accelerate protein sequence alignment Vouzis, P. D.; Sahinidis, N. V. Bioinformatics, Vol. 27, Issue 2 https://doi.org/10.1093/bioinformatics/btq644	journal	November 2010
ScalaBLAST 2.0: rapid and robust BLAST calculations on multiprocessor systems Oehmen, C. S.; Baxter, D. J. Bioinformatics, Vol. 29, Issue 6 https://doi.org/10.1093/bioinformatics/btt013	journal	January 2013
SOAPdenovo-Trans: de novo transcriptome assembly with short RNA-Seq reads Xie, Y.; Wu, G.; Tang, J. Bioinformatics, Vol. 30, Issue 12 https://doi.org/10.1093/bioinformatics/btu077	journal	February 2014
The Arabidopsis Information Resource (TAIR): improved gene annotation and new tools Lamesch, Philippe; Berardini, Tanya Z.; Li, Donghui Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr1090	journal	December 2011
BLAT---The BLAST-Like Alignment Tool Kent, W. J. Genome Research, Vol. 12, Issue 4 https://doi.org/10.1101/gr.229202	journal	March 2002
OpenMP: an industry standard API for shared-memory programming Dagum, L.; Menon, R. IEEE Computational Science and Engineering, Vol. 5, Issue 1 https://doi.org/10.1109/99.660313	journal	January 1998
Cloud Computing and Grid Computing 360-Degree Compared Foster, Ian; Zhao, Yong; Raicu, Ioan 2008 Grid Computing Environments Workshop https://doi.org/10.1109/GCE.2008.4738445	conference	November 2008
GPU Computing Owens, J. D.; Houston, M.; Luebke, D. Proceedings of the IEEE, Vol. 96, Issue 5 https://doi.org/10.1109/JPROC.2008.917757	journal	May 2008
Coordinating Computation and I/O in Massively Parallel Sequence Search Lin, Heshan; Ma, Xiaosong; Feng, Wuchun IEEE Transactions on Parallel and Distributed Systems, Vol. 22, Issue 4 https://doi.org/10.1109/TPDS.2010.101	journal	April 2011
MapReduce: simplified data processing on large clusters Dean, Jeffrey; Ghemawat, Sanjay; Mehta, Brijesh Communications of the ACM, Vol. 51, Issue 1 https://doi.org/10.1145/1327452.1327492	journal	January 2008
Compiler transformations for high-performance computing Bacon, David F.; Graham, Susan L.; Sharp, Oliver J. ACM Computing Surveys, Vol. 26, Issue 4 https://doi.org/10.1145/197405.197406	journal	December 1994
HPC-BLAST: distributed BLAST for xeon phi clusters Sawyer, Shane E.; Rekepalli, Bhanu; Horton, Mitchel D. Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics - BCB '15 https://doi.org/10.1145/2808719.2811435	conference	January 2015
SCBI_MapReduce, a New Ruby Task-Farm Skeleton for Automated Parallelisation and Distribution in Chunks of Sequences: The Implementation of a Boosted Blast+ Guerrero-Fernández, Darío; Falgueras, Juan; Claros, M. Gonzalo Computational Biology Journal, Vol. 2013 https://doi.org/10.1155/2013/707540	journal	January 2013
PLAST: parallel local alignment search tool for database comparison Van Nguyen, Hoa; Lavenier, Dominique BMC Bioinformatics, Vol. 10, Issue 1 https://doi.org/10.1186/1471-2105-10-329	journal	October 2009
BLAST+: architecture and applications Camacho, Christiam; Coulouris, George; Avagyan, Vahram BMC Bioinformatics, Vol. 10, Issue 1 https://doi.org/10.1186/1471-2105-10-421	journal	January 2009
Comparison of 15 dinoflagellate genomes reveals extensive sequence and structural divergence in family Symbiodiniaceae and genus Symbiodinium González-Pech, Raúl A.; Stephens, Timothy G.; Chen, Yibi BMC Biology, Vol. 19, Issue 1 https://doi.org/10.1186/s12915-021-00994-6	journal	April 2021
Accelerated Profile HMM Searches Eddy, Sean R. PLoS Computational Biology, Vol. 7, Issue 10 https://doi.org/10.1371/journal.pcbi.1002195	journal	October 2011
Cloud Computing and Grid Computing 360-Degree Compared Foster, Ian; Zhao, Yong; Raicu, Ioan arXiv https://doi.org/10.48550/arxiv.0901.0131	text	January 2009
SOAPdenovo-Trans: De novo transcriptome assembly with short RNA-Seq reads Xie, Yinlong; Wu, Gengxiong; Tang, Jingbo arXiv https://doi.org/10.48550/arxiv.1305.6760	preprint	January 2013

Cited By (1)

MorphoCatcher: a multiple-alignment based web tool for target selection and designing taxon-specific primers in the loop-mediated isothermal amplification method Shirshikov, Fedor V.; Pekov, Yuri A.; Miroshnikov, Konstantin A. PeerJ, Vol. 7 https://doi.org/10.7717/peerj.6801	journal	April 2019

Similar Records

iBLAST: Incremental BLAST of new sequences via automated e-value correction

Journal Article · Thu Apr 22 00:00:00 EDT 2021 · PLoS ONE · OSTI ID:1814308

Dash, Sajal; Rahman, Sarthok Rasique; M. hines, Heather; +1 more

Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters

Journal Article · Thu Dec 15 23:00:00 EST 2022 · BMC Bioinformatics · OSTI ID:1905340

Cheng, Trinity; Chin, Pei-Ju; Cha, Kenny; +2 more

Supporting divide-and-conquer algorithms for image processing

Journal Article · Sat Jan 31 23:00:00 EST 1987 · J. Parallel Distrib. Comput.; (United States) · OSTI ID:6984858

Stout, Q F

Related Subjects

59 BASIC BIOLOGICAL SCIENCES
Agricultural Science
BLAST
Bioinformatics
Computational Biology
Distributed computing
Environment
HPC
Parallel processing
Plant Science
Sequence similarity

Title: Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Citation Formats

References (43)

Cited By (1)

Similar Records

Related Subjects