The MeDICi Integration Framework: A Platform for High Performance Data Streaming Applications
Building high performance analytical applications for data streams generated from sensors is a challenging software engineering problem. Such applications typically comprise a complex pipeline of processing components that capture, transform and analyze the incoming data stream. In addition, applications must provide high throughput, be scalable, and easily modifiable so that new analytical components can be added with minimum effort. In this paper we describe the MeDICi Integration Framework (MIF), which is a middleware platform we have created to address these challenges. The MIF extends an open source messaging platform with a component-based API for integrating components into analytical pipelines. We describe the features and capabilities of the MIF, and show how it has been used to build a production analytical application for detecting cyber security attacks. The application was composed from multiple independently developed components using several different programming languages. The resulting application was able to process network sensor traffic in real time and provide insightful feedback to network analysts as soon as potential attacks were recognized.
- Research Organization:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (US)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 925493
- Report Number(s):
- PNNL-SA-57277
- Country of Publication:
- United States
- Language:
- English
Similar Records
Middleware Case Study: MeDICi
A Flexible, High Performance Service-Oriented Architecture for Detecting Cyber Attacks