skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Towards Flexible Exascale Stream Processing System Simulation

Abstract

Stream processing is an important emerging computational model for performing complex operations on and across multi-source, high-volume, unpredictable dataflows. We present Flow, a platform for parallel and distributed stream processing system simulation that provides a flexible modeling environment for analyzing stream processing applications. The Flow stream processing system simulator is a high-performance, scalable simulator that automatically parallelizes chunks of the model space and incurs near-zero synchronization overhead for acyclic stream application graphs. We show promising parallel and distributed event rates exceeding 149 million events per second on a cluster with 512 processor cores.

Authors:
 [1];  [1];  [2];  [3];  [3];  [1]
  1. IBM T. J. Watson Research Center
  2. IBM Research, Japan
  3. IBM Corporation, Haifa Research Center
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences (NCCS)
Sponsoring Org.:
USDOE
OSTI Identifier:
1047643
DOE Contract Number:  
DE-AC05-00OR22725
Resource Type:
Journal Article
Journal Name:
Simulation
Additional Journal Information:
Journal Volume: 88; Journal Issue: 7; Journal ID: ISSN 0037-5497
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; PROCESSING; SIMULATION; SIMULATORS; SYNCHRONIZATION

Citation Formats

Li, Cheng-Hong, Nair, Ravi, Ohba, Noboyuki, Shvadron, Uzi, Zaks, Ayal, and Schenfeld, Eugen. Towards Flexible Exascale Stream Processing System Simulation. United States: N. p., 2012. Web.
Li, Cheng-Hong, Nair, Ravi, Ohba, Noboyuki, Shvadron, Uzi, Zaks, Ayal, & Schenfeld, Eugen. Towards Flexible Exascale Stream Processing System Simulation. United States.
Li, Cheng-Hong, Nair, Ravi, Ohba, Noboyuki, Shvadron, Uzi, Zaks, Ayal, and Schenfeld, Eugen. 2012. "Towards Flexible Exascale Stream Processing System Simulation". United States.
@article{osti_1047643,
title = {Towards Flexible Exascale Stream Processing System Simulation},
author = {Li, Cheng-Hong and Nair, Ravi and Ohba, Noboyuki and Shvadron, Uzi and Zaks, Ayal and Schenfeld, Eugen},
abstractNote = {Stream processing is an important emerging computational model for performing complex operations on and across multi-source, high-volume, unpredictable dataflows. We present Flow, a platform for parallel and distributed stream processing system simulation that provides a flexible modeling environment for analyzing stream processing applications. The Flow stream processing system simulator is a high-performance, scalable simulator that automatically parallelizes chunks of the model space and incurs near-zero synchronization overhead for acyclic stream application graphs. We show promising parallel and distributed event rates exceeding 149 million events per second on a cluster with 512 processor cores.},
doi = {},
url = {https://www.osti.gov/biblio/1047643}, journal = {Simulation},
issn = {0037-5497},
number = 7,
volume = 88,
place = {United States},
year = {Sun Jan 01 00:00:00 EST 2012},
month = {Sun Jan 01 00:00:00 EST 2012}
}