Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective

Conference · Tue Feb 16 00:00:00 EST 2021 · 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)

DOI:https://doi.org/10.1109/isgt49243.2021.9372203· OSTI ID:1971198

Kumar, Vijay S.; Wang, Tianyi; Aggour, Kareem S.; Wang, Pengyuan; Hart, Philip J.; Yan, Weizhong

The discovery of `event signatures' and useful insights from very large historical Phasor Measurement Unit (PMU) datasets is predicated on offline Big Data analysis approaches that rely on the generation of predictive features on a massive scale. This paper presents lessons learned from a data platform perspective towards reducing barriers to adoption of Big Data analytics against a real dataset of almost half a trillion data points drawn from over 400 PMUs distributed across the North American power grid. We demonstrate software abstractions and targeted performance optimizations that can lead to significant productivity gains for power systems researchers seeking to perform offline exploratory temporal analysis and modeling tasks, with a focus on feature generation. We describe how our optimized approach goes beyond a naive application of mainstream Big Data technologies, enabling feature generation tasks, that previously took days or even weeks, to now be completed in just a few hours.

• Research Organization: GE Global Research
• Sponsoring Organization: USDOE
• DOE Contract Number: OE0000915
• OSTI ID: 1971198
• Report Number(s): DOE-GE-0000915
• Journal Information: 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Conference: 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA,16-18 February 2021
• Country of Publication: United States
• Language: English