Towards Hybrid Online On-Demand Querying of Realtime Data with Stateful Complex Event Processing

Conference · Wed Oct 09 04:00:00 EDT 2013

DOI:https://doi.org/10.1109/BigData.2013.6691575· OSTI ID:1332549

Zhou, Qunzhi ^[1]; Simmhan, Yogesh; Prasanna, Viktor K.

---

1. Power System Information & Advanced Technologies LADWP Power System Engineering Division

Emerging Big Data applications in areas like e-commerce and energy industry require both online and on-demand queries to be performed over vast and fast data arriving as streams. These present novel challenges to Big Data management systems. Complex Event Processing (CEP) is recognized as a high performance online query scheme which in particular deals with the velocity aspect of the 3-V’s of Big Data. However, traditional CEP systems do not consider data variety and lack the capability to embed ad hoc queries over the volume of data streams. In this paper, we propose H2O, a stateful complex event processing framework, to support hybrid online and on-demand queries over realtime data. We propose a semantically enriched event and query model to address data variety. A formal query algebra is developed to precisely capture the stateful and containment semantics of online and on-demand queries. We describe techniques to achieve the interactive query processing over realtime data featured by efficient online querying, dynamic stream data persistence and on-demand access. The system architecture is presented and the current implementation status reported.

• Research Organization: City of Los Angeles Department
• Sponsoring Organization: USDOE Office of Electricity Delivery and Energy Reliability (OE)
• DOE Contract Number: OE0000192
• OSTI ID: 1332549
• Report Number(s): DOE-USC-00192-97
• Country of Publication: United States
• Language: English

Related Subjects

big data
complex event processing
inmemory database
semantic web
stream processing