Towards Flexible Exascale Stream Processing System Simulation

Journal Article · Sun Jan 01 00:00:00 EST 2012 · Simulation

OSTI ID:1047643

Li, Cheng-Hong ^[1]; Nair, Ravi ^[1]; Ohba, Noboyuki ^[2]; Shvadron, Uzi ^[3]; Zaks, Ayal ^[3]; Schenfeld, Eugen ^[1]

---

1. IBM T. J. Watson Research Center
2. IBM Research, Japan
3. IBM Corporation, Haifa Research Center

Stream processing is an important emerging computational model for performing complex operations on and across multi-source, high-volume, unpredictable dataflows. We present Flow, a platform for parallel and distributed stream processing system simulation that provides a flexible modeling environment for analyzing stream processing applications. The Flow stream processing system simulator is a high-performance, scalable simulator that automatically parallelizes chunks of the model space and incurs near-zero synchronization overhead for acyclic stream application graphs. We show promising parallel and distributed event rates exceeding 149 million events per second on a cluster with 512 processor cores.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences (NCCS)
• Sponsoring Organization: USDOE
• DOE Contract Number: DE-AC05-00OR22725
• OSTI ID: 1047643
• Journal Information: Simulation, Vol. 88, Issue 7; ISSN 0037-5497
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE
PROCESSING
SIMULATION
SIMULATORS
SYNCHRONIZATION