Non-invasive (non-contact) measurements of human thermal physiology signals and thermal comfort/discomfort poses -A review

Yang, Bin; Li, Xiaojing; Hou, Yingzhen; Meier, Alan; Cheng, Xiaogang; Choi, Joon-Ho; Wang, Faming; Wang, Huan; Wagner, Andreas; Yan, Da; Li, Angui; Olofsson, Thomas; Li, Haibo

doi:10.1016/j.enbuild.2020.110261

Title: Non-invasive (non-contact) measurements of human thermal physiology signals and thermal comfort/discomfort poses -A review

Journal Article · Sun Jul 05 00:00:00 EDT 2020 · Energy and Buildings

DOI:https://doi.org/10.1016/j.enbuild.2020.110261· OSTI ID:1670422

Yang, Bin ^[1]; Li, Xiaojing ^[2]; Hou, Yingzhen ^[2]; Meier, Alan ^[3]; Cheng, Xiaogang ^[4]; Choi, Joon-Ho ^[5]; Wang, Faming ^[6]; Wang, Huan ^[7]; Wagner, Andreas ^[8]; Yan, Da ^[7]; Li, Angui ^[1]; Olofsson, Thomas ^[9]; Li, Haibo ^[10]

Xi'an Univ. of Architecture and Technology (China); Umea Univ. (Sweden)
Xi'an Univ. of Architecture and Technology (China)
Univ. of California, Davis, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Umea Univ. (Sweden); Nanjing Univ. of Posts and Telecommunications (China)
Univ. of Southern California, Los Angeles, CA (United States)
Central South Univ., Changsha (China)
Tsinghua Univ., Beijing (China)
Karlsruhe Inst. of Technology (KIT) (Germany)
Umea Univ. (Sweden)
KTH Royal Inst. of Technology, Stockholm (Sweden)

Heating, ventilation and air-conditioning (HVAC) systems have been adopted to create comfortable, healthy and safe indoor environments. In the control loop, the technical feature of the human demand-oriented supply can help operate HVAC effectively. Among many technical options, real time monitoring based on feedback signals from end users has been frequently reported as a critical technology to confirm optimizing building performance. Recent studies have incorporated human thermal physiology signals and thermal comfort/discomfort status as real-time feedback signals. A series of human subject experiments used to be conducted by primarily adopting subjective questionnaire surveys in a lab-setting study, which is limited in the application for reality. With the help of advanced technologies, physiological signals have been detected, measured and processed by using multiple technical formats, such as wearable sensors. Nevertheless, they mostly require physical contacts with the skin surface in spite of the small physical dimension and compatibility with other wearable accessories, such as goggles, and intelligent bracelets. Most recently, a low cost small infrared camera has been adopted for monitoring human facial images, which could detect the facial skin temperature and blood perfusion in a contactless way. Also, according to latest pilot studies, a conventional digital camera can generate infrared images with the help of new methods, such as the Euler video magnification technology. Human thermal comfort/discomfort poses can also be detected by video methods without contacting human bodies and be analyzed by the skeleton keypoints model. Here, new sensing technologies were summarized, their cons and pros were discussed, and extended applications for the demand-oriented ventilation were also reviewed as potential development and applications.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office. Building Technologies Office

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1670422

Journal Information:: Energy and Buildings, Vol. 224; ISSN 0378-7788

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (62)

Predicting individual thermal comfort using machine learning algorithms Farhan, Asma Ahmad; Pattipati, Krishna; Wang, Bing 2015 IEEE International Conference on Automation Science and Engineering (CASE) https://doi.org/10.1109/CoASE.2015.7294164	conference	August 2015
A comparative study of predicting individual thermal sensation and satisfaction using wrist-worn temperature sensor, thermal camera and ambient temperature sensor Aryal, Ashrant; Becerik-Gerber, Burcin Building and Environment, Vol. 160 https://doi.org/10.1016/j.buildenv.2019.106223	journal	August 2019
Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment Aryal, Ashrant; Becerik-Gerber, Burcin BuildSys '19: The 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation https://doi.org/10.1145/3360322.3360848	conference	November 2019
Human emotion recognition from facial thermal image based on fused statistical feature and multi-class SVM Basu, Anushree; Routray, Aurobinda; Shit, Suprosanna 2015 Annual IEEE India Conference (INDICON) https://doi.org/10.1109/INDICON.2015.7443712	conference	December 2015
Machine learning method for real-time non-invasive prediction of individual thermal preference in transient conditions Cosma, Andrei Claudiu; Simha, Rahul Building and Environment, Vol. 148 https://doi.org/10.1016/j.buildenv.2018.11.017	journal	January 2019
Thermal comfort modeling in transient conditions using real-time local body temperature extraction with a thermographic camera Cosma, Andrei Claudiu; Simha, Rahul Building and Environment, Vol. 143 https://doi.org/10.1016/j.buildenv.2018.06.052	journal	October 2018
Using the contrast within a single face heat map to assess personal thermal comfort Cosma, Andrei Claudiu; Simha, Rahul Building and Environment, Vol. 160 https://doi.org/10.1016/j.buildenv.2019.106163	journal	August 2019
An online learning approach for quantifying personalized thermal comfort via adaptive stochastic modeling Ghahramani, Ali; Tang, Chao; Becerik-Gerber, Burcin Building and Environment, Vol. 92 https://doi.org/10.1016/j.buildenv.2015.04.017	journal	October 2015
Infrared thermography of human face for monitoring thermoregulation performance and estimating personal thermal comfort Ghahramani, Ali; Castro, Guillermo; Becerik-Gerber, Burcin Building and Environment, Vol. 109 https://doi.org/10.1016/j.buildenv.2016.09.005	journal	November 2016
Towards unsupervised learning of thermal comfort using infrared thermography Ghahramani, Ali; Castro, Guillermo; Karvigh, Simin Ahmadi Applied Energy, Vol. 211 https://doi.org/10.1016/j.apenergy.2017.11.021	journal	February 2018
Upscaling participatory thermal sensing: Lessons from an interdisciplinary case study at University of California for improving campus efficiency and comfort Sanguinetti, Angela; Pritoni, Marco; Salmon, Kiernan Energy Research & Social Science, Vol. 32 https://doi.org/10.1016/j.erss.2017.05.026	journal	October 2017
DeepPose: Human Pose Estimation via Deep Neural Networks Toshev, Alexander; Szegedy, Christian 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2014.214	conference	June 2014
Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease Galna, Brook; Barry, Gillian; Jackson, Dan Gait & Posture, Vol. 39, Issue 4 https://doi.org/10.1016/j.gaitpost.2014.01.008	journal	April 2014
Real-Time Monitoring of Occupants’ Thermal Comfort through Infrared Imaging: A Preliminary Study Pavlin, Boris; Pernigotto, Giovanni; Cappelletti, Francesca Buildings, Vol. 7, Issue 4 https://doi.org/10.3390/buildings7010010	journal	February 2017
Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings Yang, Bin; Cheng, Xiaogang; Dai, Dengxin Building and Environment, Vol. 162 https://doi.org/10.1016/j.buildenv.2019.106284	journal	September 2019
Skin and core temperature response to partial- and whole-body heating and cooling Huizenga, Charlie; Zhang, Hui; Arens, Edward Journal of Thermal Biology, Vol. 29, Issue 7-8 https://doi.org/10.1016/j.jtherbio.2004.08.024	journal	October 2004
Motion magnification Liu, Ce; Torralba, Antonio; Freeman, William T. ACM Transactions on Graphics, Vol. 24, Issue 3 https://doi.org/10.1145/1073204.1073223	journal	July 2005
StressCam: non-contact measurement of users' emotional states through thermal imaging Puri, Colin; Olson, Leslie; Pavlidis, Ioannis CHI '05 extended abstracts on Human factors in computing systems - CHI '05 https://doi.org/10.1145/1056808.1057007	conference	January 2005
Non-intrusive interpretation of human thermal comfort through analysis of facial infrared thermography Li, Da; Menassa, Carol C.; Kamat, Vineet R. Energy and Buildings, Vol. 176 https://doi.org/10.1016/j.enbuild.2018.07.025	journal	October 2018
Robust non-intrusive interpretation of occupant thermal comfort in built environments with low-cost networked thermal cameras Li, Da; Menassa, Carol C.; Kamat, Vineet R. Applied Energy, Vol. 251 https://doi.org/10.1016/j.apenergy.2019.113336	journal	October 2019
Measuring indoor occupancy in intelligent buildings using the fusion of vision sensors Liu, Dixin; Guan, Xiaohong; Du, Youtian Measurement Science and Technology, Vol. 24, Issue 7 https://doi.org/10.1088/0957-0233/24/7/074023	journal	June 2013
The mental and subjective skin: Emotion, empathy, feelings and thermography Salazar-López, E.; Domínguez, E.; Juárez Ramos, V. Consciousness and Cognition, Vol. 34 https://doi.org/10.1016/j.concog.2015.04.003	journal	July 2015
Infrared Imaging Analysis for Thermal Comfort Assessment De Oliveira, Fabrice; Moreau, Sophie; Gehin, Claudine 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society https://doi.org/10.1109/IEMBS.2007.4353054	conference	August 2007
Personalized thermal comfort inference using RGB video images for distributed HVAC control Jazizadeh, Farrokh; Jung, Wooyoung Applied Energy, Vol. 220 https://doi.org/10.1016/j.apenergy.2018.02.049	journal	June 2018
Smart control of indoor thermal environment based on online learned thermal comfort model using infrared thermal imaging Wang, Fulin; Zhu, Binruo; Li, Rui 2017 13th IEEE Conference on Automation Science and Engineering (CASE 2017), 2017 13th IEEE Conference on Automation Science and Engineering (CASE) https://doi.org/10.1109/COASE.2017.8256221	conference	August 2017
Skin temperature measurements by infrared thermography during running exercise Tanda, Giovanni Experimental Thermal and Fluid Science, Vol. 71 https://doi.org/10.1016/j.expthermflusci.2015.10.006	journal	February 2016
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies Joo, Hanbyul; Simon, Tomas; Sheikh, Yaser 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00868	conference	June 2018
Real-time human skin temperature analysis using thermal image recognition for thermal comfort assessment Metzmacher, Henning; Wölki, Daniel; Schmidt, Carolin Energy and Buildings, Vol. 158 https://doi.org/10.1016/j.enbuild.2017.09.032	journal	January 2018
Development of a human metabolic rate prediction model based on the use of Kinect-camera generated visual data-driven approaches Na, HooSeung; Choi, Joon-Ho; Kim, HoSeong Building and Environment, Vol. 160 https://doi.org/10.1016/j.buildenv.2019.106216	journal	August 2019
ThermalWrist: Smartphone Thermal Camera Correction Using a Wristband Sensor † Yoshikawa, Hiroki; Uchiyama, Akira; Higashino, Teruo Sensors, Vol. 19, Issue 18 https://doi.org/10.3390/s19183826	journal	September 2019
Eulerian video magnification for revealing subtle changes in the world Wu, Hao-Yu; Rubinstein, Michael; Shih, Eugene ACM Transactions on Graphics, Vol. 31, Issue 4 https://doi.org/10.1145/2185520.2185561	journal	August 2012
Development of the data-driven thermal satisfaction prediction model as a function of human physiological responses in a built environment Choi, Joon-Ho; Yeom, Dongwoo Building and Environment, Vol. 150 https://doi.org/10.1016/j.buildenv.2019.01.007	journal	March 2019
Study of data-driven thermal sensation prediction model as a function of local body skin temperatures in a built environment Choi, Joon-Ho; Yeom, Dongwoo Building and Environment, Vol. 121 https://doi.org/10.1016/j.buildenv.2017.05.004	journal	August 2017
Infrared Thermography Sensor for Disease Activity Detection in Rheumatoid Arthritis Patients Pauk, Jolanta; Wasilewska, Agnieszka; Ihnatouski, Mikhail Sensors, Vol. 19, Issue 16 https://doi.org/10.3390/s19163444	journal	August 2019
ThermalSense: determining dynamic thermal comfort preferences using thermographic imaging Ranjan, Juhi; Scott, James UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing https://doi.org/10.1145/2971648.2971659	conference	September 2016
Occupancy detection in the office by analyzing surveillance videos and its application to building energy conservation Zou, Jianhong; Zhao, Qianchuan; Yang, Wen Energy and Buildings, Vol. 152 https://doi.org/10.1016/j.enbuild.2017.07.064	journal	October 2017
Multibiometric identification by using ear, face, and thermal face Bayram, Kadir Sercan; Bolat, Bülent EURASIP Journal on Image and Video Processing, Vol. 2018, Issue 1 https://doi.org/10.1186/s13640-018-0274-x	journal	May 2018
Fever Screening and Detection of Febrile Arrivals at an International Airport in Korea: Association among Self-reported Fever, Infrared Thermal Camera Scanning, and Tympanic Temperature Cho, Kyung Sook; Yoon, Jangho Epidemiology and Health, Vol. 36 https://doi.org/10.4178/epih/e2014004	journal	January 2014
Hierarchical Classification of Low Resolution Thermal Images for Occupancy Estimation Walmsley-Eyre, Liam; Cardell-Oliver, Rachel 2017 IEEE 42nd Conference on Local Computer Networks: Workshops (LCN Workshops), 2017 IEEE 42nd Conference on Local Computer Networks Workshops (LCN Workshops) https://doi.org/10.1109/LCN.Workshops.2017.59	conference	October 2017
Listening to the occupants: a Web-based indoor environmental quality survey Zagreus, Leah; Huizenga, Charlie; Arens, Edward Indoor Air, Vol. 14, Issue s8 https://doi.org/10.1111/j.1600-0668.2004.00301.x	journal	December 2004
Using Infrared Thermography and Biosensors to Detect Thermal Discomfort in a Building’s Inhabitants Burzo, Mihai; Abouelenien, Mohamed; Pérez-Rosas, Verónica ASME 2014 International Mechanical Engineering Congress and Exposition, Volume 6B: Energy https://doi.org/10.1115/IMECE2014-40269	conference	March 2015
Thermal Discomfort Detection Using Thermal Imaging Burzo, Mihai; Abouelenien, Mohamed; Van Alstine, David ASME 2017 International Mechanical Engineering Congress and Exposition, Volume 6: Energy https://doi.org/10.1115/IMECE2017-72162	conference	January 2018
A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings Kopaczka, Marcin; Breuer, Lukas; Schock, Justus Sensors, Vol. 19, Issue 19 https://doi.org/10.3390/s19194135	journal	September 2019
A feature-based approach to people re-identification using skeleton keypoints Munaro, Matteo; Ghidoni, Stefano; Dizmen, Deniz Tartaro 2014 IEEE International Conference on Robotics and Automation (ICRA) https://doi.org/10.1109/ICRA.2014.6907689	conference	May 2014
Visualization of body thermoregulation by infrared imaging Bouzida, Nabila; Bendada, Abdelhakim; Maldague, Xavier P. Journal of Thermal Biology, Vol. 34, Issue 3 https://doi.org/10.1016/j.jtherbio.2008.11.008	journal	April 2009
Single Sample Face Recognition in the Last Decade: A Survey Kumar, Nitin; Garg, Vaani International Journal of Pattern Recognition and Artificial Intelligence, Vol. 33, Issue 13 https://doi.org/10.1142/S0218001419560093	journal	December 2019
A comparative study of face landmarking techniques Çeliktutan, Oya; Ulukaya, Sezer; Sankur, Bülent EURASIP Journal on Image and Video Processing, Vol. 2013, Issue 1 https://doi.org/10.1186/1687-5281-2013-13	journal	March 2013
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group Vemulapalli, Raviteja; Arrate, Felipe; Chellappa, Rama 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2014.82	conference	June 2014
Convolutional Pose Machines Wei, Shih-En; Ramakrishna, Varun; Kanade, Takeo 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.511	conference	June 2016
Near infrared face recognition: A literature survey Farokhi, Sajad; Flusser, Jan; Ullah Sheikh, Usman Computer Science Review, Vol. 21 https://doi.org/10.1016/j.cosrev.2016.05.003	journal	August 2016
A multi-viewpoint feature-based re-identification system driven by skeleton keypoints Ghidoni, Stefano; Munaro, Matteo Robotics and Autonomous Systems, Vol. 90 https://doi.org/10.1016/j.robot.2016.10.006	journal	April 2017
Thermal infrared imaging in psychophysiology: Potentialities and limits Ioannou, Stephanos; Gallese, Vittorio; Merla, Arcangelo Psychophysiology, Vol. 51, Issue 10 https://doi.org/10.1111/psyp.12243	journal	June 2014
Prediction of whole-body thermal sensation in the non-steady state based on skin temperature Takada, Satoru; Matsumoto, Sho; Matsushita, Takayuki Building and Environment, Vol. 68 https://doi.org/10.1016/j.buildenv.2013.06.004	journal	October 2013
Estimation of Thermal Sensation Based on Wrist Skin Temperatures Sim, Soo; Koh, Myung; Joo, Kwang Sensors, Vol. 16, Issue 4 https://doi.org/10.3390/s16040420	journal	March 2016
Machine learning based prediction of thermal comfort in buildings of equatorial Singapore Chaudhuri, Tanaya; Soh, Yeng Chai; Li, Hua 2017 IEEE International Conference on Smart Grid and Smart Cities (ICSGSC) https://doi.org/10.1109/ICSGSC.2017.8038552	conference	July 2017
Experimental research of online monitoring and evaluation method of human thermal sensation in different active states based on wristband device Li, Wei; Zhang, Jili; Zhao, Tianyi Energy and Buildings, Vol. 173 https://doi.org/10.1016/j.enbuild.2018.05.056	journal	August 2018
NIDL: A pilot study of contactless measurement of skin temperature for intelligent building Cheng, Xiaogang; Yang, Bin; Hedman, Anders Energy and Buildings, Vol. 198 https://doi.org/10.1016/j.enbuild.2019.06.007	journal	September 2019
A Contactless Measuring Method of Skin Temperature based on the Skin Sensitivity Index and Deep Learning Cheng, Xiaogang; Yang, Bin; Tan, Kaige Applied Sciences, Vol. 9, Issue 7 https://doi.org/10.3390/app9071375	journal	April 2019
A pilot study of online non-invasive measuring technology based on video magnification to determine skin temperature Cheng, Xiaogang; Yang, Bin; Olofsson, Thomas Building and Environment, Vol. 121 https://doi.org/10.1016/j.buildenv.2017.05.021	journal	August 2017
Breathable and Stretchable Temperature Sensors Inspired by Skin Chen, Ying; Lu, Bingwei; Chen, Yihao Scientific Reports, Vol. 5, Issue 1 https://doi.org/10.1038/srep11505	journal	June 2015
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields Cao, Zhe; Simon, Tomas; Wei, Shih-En 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.143	conference	July 2017
Forehead Thermal Signature Extraction in Lie Detection Zhu, Zhen; Tsiamyrtzis, Panagiotis; Pavlidis, Ioannis 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society https://doi.org/10.1109/IEMBS.2007.4352269	conference	August 2007

Similar Records

Parameters of human discomfort in warm environments

Conference · Wed Jan 01 00:00:00 EST 1986 · ASHRAE Trans.; (United States) · OSTI ID:1670422

Berglund, L G; Cunningham, D J

Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls

Journal Article · Mon Aug 15 00:00:00 EDT 2022 · Buildings · OSTI ID:1670422

Ghahramani, Ali; Xu, Qian; Min, Syung; +5 more

Human-in-the-loop Sensing and Control for Commercial Building Energy Efficiency and Occupant Comfort

Technical Report · Sun Aug 15 00:00:00 EDT 2021 · OSTI ID:1670422

Bergés, Mario; Munir, Sirajum; Shelton, Charles

Related Subjects

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION
non-invasive (non-contact) measurements
thermal physiology signals
thermal comfort/discomfort poses
Euler video magnification
skeleton keypoints

Title: Non-invasive (non-contact) measurements of human thermal physiology signals and thermal comfort/discomfort poses -A review

Citation Formats

References (62)

Similar Records

Related Subjects