Selective Sampling for Sensor Type Classification in Buildings

Ma, Jing; Hong, Dezhi; Wang, Hongning

doi:10.1109/ipsn48710.2020.00028

Selective Sampling for Sensor Type Classification in Buildings

Conference · Tue Jun 09 04:00:00 EDT 2020 · 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)

DOI:https://doi.org/10.1109/ipsn48710.2020.00028· OSTI ID:1822657

Ma, Jing ^[1]; Hong, Dezhi; Wang, Hongning

University of Virginia

A key barrier to applying any smart technology to a building is the requirement of locating and connecting to the necessary resources among the thousands of sensing and control points, i.e., the metadata mapping problem. Existing solutions depend on exhaustive manual annotation of sensor metadata --- a laborious, costly, and hardly scalable process. To reduce the amount of manual effort required, this paper presents a multi-oracle selective sampling framework to leverage noisy labels from information sources with unknown reliability such as existing buildings, which we refer to as weak oracles, for metadata mapping. This framework involves an interactive process, where a small set of sensor instances are progressively selected and labeled for it to learn how to aggregate the noisy labels as well as to predict sensor types. Two key challenges arise in designing the framework, namely, weak oracle reliability estimation and instance selection for querying. To address the first challenge, we develop a clustering-based approach for weak oracle reliability estimation to capitalize on the observation that weak oracles perform differently in different groups of instances. For the second challenge, we propose a disagreement-based query selection strategy to combine the potential effect of a labeled instance on both reducing classifier uncertainty and improving the quality of label aggregation. We evaluate our solution on a large collection of real-world building sensor data from 5 buildings with more than 11,000 sensors of 18 different types. The experiment results validate the effectiveness of our solution, which outperforms a set of state-of-the-art baselines.

View Conference

Research Organization:: University of Virginia

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office. Building Technologies Office

DOE Contract Number:: EE0008227

OSTI ID:: 1822657

Report Number(s):: DOE-UVA-0008227-7

Conference Information:: Journal Name: 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)

Country of Publication:: United States

Language:: English

References (32)

On Information and Sufficiency Kullback, S.; Leibler, R. A. The Annals of Mathematical Statistics, Vol. 22, Issue 1 https://doi.org/10.1214/aoms/1177729694	journal	March 1951
Towards Automatic Spatial Verification of Sensor Placement in Buildings Hong, Dezhi; Ortiz, Jorge; Whitehouse, Kamin Proceedings of the 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings https://doi.org/10.1145/2528282.2528302	conference	November 2013
Scrabble: transferrable semi-automated semantic metadata normalization using intermediate representation Koh, Jason; Balaji, Bharathan; Sengupta, Dhiman BuildSys '18: The 5th ACM International Conference on Systems for Built Environments, Proceedings of the 5th Conference on Systems for Built Environments https://doi.org/10.1145/3276774.3276795	conference	November 2018
Short Paper Pritoni, Marco; Bhattacharya, Arka A.; Culler, David Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments https://doi.org/10.1145/2821650.2821677	conference	November 2015
A Bayesian Analysis of Some Nonparametric Problems Ferguson, Thomas S. The Annals of Statistics, Vol. 1, Issue 2 https://doi.org/10.1214/aos/1176342360	journal	March 1973
Brick: Towards a Unified Metadata Schema For Buildings Balaji, Bharathan; Agarwal, Yuvraj; Berges, Mario Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments - BuildSys '16 https://doi.org/10.1145/2993422.2993577	conference	January 2016
Active learning with confidence-based answers for crowdsourcing labeling tasks Song, Jinhua; Wang, Hao; Gao, Yang Knowledge-Based Systems, Vol. 159 https://doi.org/10.1016/j.knosys.2018.07.010	journal	November 2018
Strip, bind, and search: a method for identifying abnormal energy consumption in buildings Fontugne, Romain; Ortiz, Jorge; Tremblay, Nicolas Proceedings of the 12th international conference on Information processing in sensor networks - IPSN '13 https://doi.org/10.1145/2461381.2461399	conference	January 2013
Efficient crowdsourcing of unknown experts using bounded multi-armed bandits Tran-Thanh, Long; Stein, Sebastian; Rogers, Alex Artificial Intelligence, Vol. 214 https://doi.org/10.1016/j.artint.2014.04.005	journal	September 2014
Proactive learning Donmez, Pinar; Carbonell, Jaime G. Proceeding of the 17th ACM conference on Information and knowledge mining - CIKM '08 https://doi.org/10.1145/1458082.1458165	conference	January 2008
POEM: power-efficient occupancy-based energy management system Erickson, Varick L.; Achleitner, Stefan; Cerpa, Alberto E. Proceedings of the 12th international conference on Information processing in sensor networks - IPSN '13 https://doi.org/10.1145/2461381.2461407	conference	January 2013
Automated Metadata Construction to Support Portable Building Applications Bhattacharya, Arka A.; Hong, Dezhi; Culler, David Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments - BuildSys '15 https://doi.org/10.1145/2821650.2821667	conference	January 2015
Comparison of linear correlation and a statistical dependency measure for inferring spatial relation of temperature sensors in buildings Koc, Merthan; Akinci, Burcu; Bergés, Mario Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings https://doi.org/10.1145/2674061.2674075	conference	November 2014
Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm Dawid, A. P.; Skene, A. M. Applied Statistics, Vol. 28, Issue 1 https://doi.org/10.2307/2346806	journal	January 1979
DeviceMien Ortiz, Jorge; Crawford, Catherine; Le, Franck Proceedings of the International Conference on Internet of Things Design and Implementation https://doi.org/10.1145/3302505.3310073	conference	April 2019
Towards automating the deployment of energy saving approaches in buildings Schumann, Anika; Ploennigs, Joern; Gorman, Bernard Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings https://doi.org/10.1145/2674061.2674081	conference	November 2014
Truth inference in crowdsourcing Zheng, Yudian; Li, Guoliang; Li, Yuanbing Proceedings of the VLDB Endowment, Vol. 10, Issue 5 https://doi.org/10.14778/3055540.3055547	journal	January 2017
Combining Crowd and Expert Labels Using Decision Theoretic Active Learning Nguyen, An; Wallace, Byron; Lease, Matthew Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 3 https://doi.org/10.1609/hcomp.v3i1.13225	journal	September 2015
Clustering-based Active Learning on Sensor Type Classification in Buildings Hong, Dezhi; Wang, Hongning; Whitehouse, Kamin Proceedings of the 24th ACM International on Conference on Information and Knowledge Management https://doi.org/10.1145/2806416.2806574	conference	October 2015
Mismatch string kernels for discriminative protein classification Leslie, C. S.; Eskin, E.; Cohen, A. Bioinformatics, Vol. 20, Issue 4 https://doi.org/10.1093/bioinformatics/btg431	journal	January 2004
Automated point mapping for building control systems: Recent advances and future research needs Wang, Weimin; Brambley, Michael R.; Kim, Woohyun Automation in Construction, Vol. 85 https://doi.org/10.1016/j.autcon.2017.09.013	journal	January 2018
A Bayesian Framework for Modeling Human Evaluations Lakkaraju, Himabindu; Leskovec, Jure; Kleinberg, Jon Proceedings of the 2015 SIAM International Conference on Data Mining https://doi.org/10.1137/1.9781611974010.21	conference	June 2015
Zodiac Balaji, Bharathan; Verma, Chetan; Narayanaswamy, Balakrishnan Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments https://doi.org/10.1145/2821650.2821674	conference	November 2015
Efficiently learning the accuracy of labeling sources for selective sampling Donmez, Pinar; Carbonell, Jaime G.; Schneider, Jeff Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '09 https://doi.org/10.1145/1557019.1557053	conference	January 2009
The Building Adapter: Towards Quickly Applying Building Analytics at Scale Hong, Dezhi; Wang, Hongning; Ortiz, Jorge Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments - BuildSys '15 https://doi.org/10.1145/2821650.2821657	conference	January 2015
A study of the effect of different types of noise on the precision of supervised learning techniques Nettleton, David F.; Orriols-Puig, Albert; Fornells, Albert Artificial Intelligence Review, Vol. 33, Issue 4 https://doi.org/10.1007/s10462-010-9156-z	journal	January 2010
Plaster: an integration, benchmark, and development framework for metadata normalization methods Koh, Jason; Hong, Dezhi; Gupta, Rajesh BuildSys '18: The 5th ACM International Conference on Systems for Built Environments, Proceedings of the 5th Conference on Systems for Built Environments https://doi.org/10.1145/3276774.3276794	conference	November 2018
The wisdom of minority Li, Hongwei; Zhao, Bo; Fuxman, Ariel Proceedings of the 23rd international conference on World wide web - WWW '14 https://doi.org/10.1145/2566486.2568033	conference	January 2014
Active Learning with Statistical Models Cohn, D. A.; Ghahramani, Z.; Jordan, M. I. Journal of Artificial Intelligence Research, Vol. 4 https://doi.org/10.1613/jair.295	journal	January 1996
Sentinel: occupancy based HVAC actuation using existing WiFi infrastructure within commercial buildings Balaji, Bharathan; Xu, Jian; Nwokafor, Anthony Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems - SenSys '13 https://doi.org/10.1145/2517351.2517370	conference	January 2013
A Data-driven Meta-data Inference Framework for Building Automation Systems Gao, Jingkun; Ploennigs, Joern; Berges, Mario Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments - BuildSys '15 https://doi.org/10.1145/2821650.2821670	conference	January 2015
Online crowdsourcing: Rating annotators and obtaining cost-effective labels Welinder, Peter; Perona, Pietro 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops https://doi.org/10.1109/CVPRW.2010.5543189	conference	June 2010

Similar Records

Sequential Learning with Active Partial Labeling for Building Metadata

Conference · Tue Nov 12 23:00:00 EST 2019 · OSTI ID:1567729

The Building Adapter: Automatic Mapping of Commercial Buildings for Scalable Building Analytics

Technical Report · Tue Sep 28 00:00:00 EDT 2021 · OSTI ID:1822357

Towards Semantic Search in Building Sensor Data

Conference · Wed Sep 29 00:00:00 EDT 2021 · OSTI ID:1822654

Related Subjects

96 KNOWLEDGE MANAGEMENT AND PRESERVATION
Selective sampling
label aggregation
sensor type classification
smart buildings

Selective Sampling for Sensor Type Classification in Buildings

Citation Formats

References (32)

Similar Records

Related Subjects