skip to main content

SciTech ConnectSciTech Connect

Title: A work stealing based approach for enabling scalable optimal sequence homology detection

Sequence homology detection is central to a number of bioinformatics applications including genome sequencing and protein family characterization. Given millions of sequences, the goal is to identify all pairs of sequences that are highly similar (or “homologous”) on the basis of alignment criteria. While there are optimal alignment algorithms to compute pairwise homology, their deployment for large-scale is currently not feasible; instead, heuristic methods are used at the expense of quality. Here, we present the design and evaluation of a parallel implementation for conducting optimal homology detection on distributed memory supercomputers. Our approach uses a combination of techniques from asynchronous load balancing (viz. work stealing, dynamic task counters), data replication, and exact-matching filters to achieve homology detection at scale. Results for 2.56M sequences on up to 8K cores show parallel efficiencies of ~ 75-100%, a time-to-solution of 33s, and a rate of ~ 2.0M alignments per second.
 [1] ;  [2] ;  [1] ;  [1]
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  2. Washington State Univ., Pullman, WA (United States)
Publication Date:
OSTI Identifier:
Report Number(s):
Journal ID: ISSN 0743-7315; KJ0402000
DOE Contract Number:
Resource Type:
Journal Article
Resource Relation:
Journal Name: Journal of Parallel and Distributed Computing; Journal Volume: 79-80; Journal Issue: C
Research Org:
Pacific Northwest National Laboratory (PNNL), Richland, WA (US)
Sponsoring Org:
Country of Publication:
United States
sequence homology; homolgy detection; pairwise sequence alignment; protein family identification; dynamic load balancing; work stealing; distributed task counters; parallel suffix tree construction