Human-Machine Shared Control for Path Following Considering Driver Fatigue Characteristics

Fang, Zhenwu; Wang, Jinxiang; Wang, Zejiang; Chen, Jinxin; Yin, Guodong; Zhang, Hui

doi:10.1109/tits.2023.3347439

Title: Human-Machine Shared Control for Path Following Considering Driver Fatigue Characteristics

Journal Article · 16 January 2024 · IEEE Transactions on Intelligent Transportation Systems

DOI: https://doi.org/10.1109/tits.2023.3347439 · OSTI ID:2282937

^[1];

^[1]; Wang, Zejiang ^[2];

^[1];

^[3]

Southeast University, Nanjing (China)
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Beihang University, Beijing (China)

Fatigue driving has been regarded as one of the most important factors that cause traffic accidents. This paper proposes a robust human-machine shared control strategy to improve the vehicle performance for different driver fatigue states. Firstly, the time-varying driver steering model is proposed to address the model mismatch caused by fatigue driving. And the driver fatigue evaluation system is established based on facial features to quantify driver fatigue levels. Based on the quantified fatigue levels, a novel strategy for allocating authorities of the driver and controller is developed for building the driver-vehicle interaction system. Then, to weaken the influence of parameter perturbations caused by the time-varying driver states, we design a fatigue-based shared controller through state feedback. The actuator saturation and system constraints are considered in the controller design through the robust set-invariance property to improve vehicle safety and driving comfort. The driver-in-the-loop platform is conducted to validate the effectiveness of the proposed shared steering controller. In conclusion, the experimental results show that the proposed strategy can adaptively optimize the human-machine authorities according to fatigue states and comprehensively improve vehicle performance.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; National Natural Science Foundation of China (NSFC); China Scholarship Council (CSC)

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 2282937

Journal Information:: IEEE Transactions on Intelligent Transportation Systems, Journal Name: IEEE Transactions on Intelligent Transportation Systems Journal Issue: 1 Vol. 1; ISSN 1524-9050

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (45)

Driver state detection for driver-automation shared control with fuzzy logic Zhou, Shaodong; Ju, Zhiyang; Liu, Yue Control Engineering Practice, Vol. 127 https://doi.org/10.1016/j.conengprac.2022.105294	journal	October 2022
Driver fatigue detection based on deeply-learned facial expression representation Liu, Zhongmin; Peng, Yuxi; Hu, Wenjin Journal of Visual Communication and Image Representation, Vol. 71 https://doi.org/10.1016/j.jvcir.2019.102723	journal	August 2020
Effects of driver work-rest patterns, lifestyle and payment incentives on long-haul truck driver sleepiness Mahajan, Kirti; Velaga, Nagendra R.; Kumar, Akhilesh Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 60 https://doi.org/10.1016/j.trf.2018.10.028	journal	January 2019
Human-like driving behaviour emerges from a risk-based driver model Kolekar, Sarvesh; de Winter, Joost; Abbink, David Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-18353-4	journal	September 2020
A predictive control framework for torque-based steering assistance to improve safety in highway driving Ercan, Ziya; Carvalho, Ashwin; Tseng, H. Eric Vehicle System Dynamics, Vol. 56, Issue 5 https://doi.org/10.1080/00423114.2017.1337915	journal	June 2017
Identification and validation of a driver steering control model incorporating human sensory dynamics Nash, C. J.; Cole, D. J. Vehicle System Dynamics, Vol. 58, Issue 4 https://doi.org/10.1080/00423114.2019.1589536	journal	March 2019
Driver assistant yaw stability control via integration of AFS and DYC Ahmadian, Narjes; Khosravi, Alireza; Sarhadi, Pouria Vehicle System Dynamics, Vol. 60, Issue 5 https://doi.org/10.1080/00423114.2021.1879390	journal	January 2021
Decision-making in driver-automation shared control: A review and perspectives Wang, Wenshuo; Na, Xiaoxiang; Cao, Dongpu IEEE/CAA Journal of Automatica Sinica, Vol. 7, Issue 5 https://doi.org/10.1109/JAS.2020.1003294	journal	September 2020
Human–Machine Adaptive Shared Control for Safe Driving Under Automation Degradation Huang, Chao; Lv, Chen; Hang, Peng IEEE Intelligent Transportation Systems Magazine, Vol. 14, Issue 2 https://doi.org/10.1109/MITS.2021.3065382	journal	March 2022
Modeling, Identification, and Predictive Control of a Driver Steering Assistance System Ercan, Ziya; Carvalho, Ashwin; Gokasan, Metin IEEE Transactions on Human-Machine Systems, Vol. 47, Issue 5 https://doi.org/10.1109/THMS.2017.2717881	journal	October 2017
A Review of Shared Control for Automated Vehicles: Theory and Applications Marcano, Mauricio; Diaz, Sergio; Perez, Joshue IEEE Transactions on Human-Machine Systems, Vol. 50, Issue 6 https://doi.org/10.1109/THMS.2020.3017748	journal	December 2020
Human–Machine Cooperative Steering Control Considering Mitigating Human–Machine Conflict Based on Driver Trust Shi, Zhuqing; Chen, Hong; Qu, Ting IEEE Transactions on Human-Machine Systems, Vol. 52, Issue 5 https://doi.org/10.1109/THMS.2022.3190683	journal	October 2022
Driver-Automation Cooperative Approach for Shared Steering Control Under Multiple System Constraints: Design and Experiments Nguyen, Anh-Tu; Sentouh, Chouki; Popieul, Jean-Christophe IEEE Transactions on Industrial Electronics, Vol. 64, Issue 5 https://doi.org/10.1109/TIE.2016.2645146	journal	May 2017
Cooperative Trajectory Planning for Haptic Shared Control Between Driver and Automation in Highway Driving Benloucif, Amir; Nguyen, Anh-Tu; Sentouh, Chouki IEEE Transactions on Industrial Electronics, Vol. 66, Issue 12 https://doi.org/10.1109/TIE.2019.2893864	journal	December 2019
Driver Fatigue Detection Systems: A Review Sikander, Gulbadan; Anwar, Shahzad IEEE Transactions on Intelligent Transportation Systems, Vol. 20, Issue 6 https://doi.org/10.1109/TITS.2018.2868499	journal	June 2019
A Novel Machine Vision-Based 3D Facial Action Unit Identification for Fatigue Detection Sikander, Gulbadan; Anwar, Shahzad IEEE Transactions on Intelligent Transportation Systems, Vol. 22, Issue 5 https://doi.org/10.1109/TITS.2020.2974263	journal	May 2021
Multimodal Features for Detection of Driver Stress and Fatigue: Review Nemcova, Andrea; Svozilova, Veronika; Bucsuhazy, Katerina IEEE Transactions on Intelligent Transportation Systems, Vol. 22, Issue 6 https://doi.org/10.1109/TITS.2020.2977762	journal	June 2021
Vision-Based Fatigue Driving Recognition Method Integrating Heart Rate and Facial Features Du, Guanglong; Li, Tao; Li, Chunquan IEEE Transactions on Intelligent Transportation Systems, Vol. 22, Issue 5 https://doi.org/10.1109/TITS.2020.2979527	journal	May 2021
Indirect Shared Control for Cooperative Driving Between Driver and Automation in Steer-by-Wire Vehicles Li, Renjie; Li, Yanan; Li, Shengbo Eben IEEE Transactions on Intelligent Transportation Systems, Vol. 22, Issue 12 https://doi.org/10.1109/TITS.2020.3010620	journal	December 2021
RF-DCM: Multi-Granularity Deep Convolutional Model Based on Feature Recalibration and Fusion for Driver Fatigue Detection Huang, Rui; Wang, Yan; Li, Zijian IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 1 https://doi.org/10.1109/TITS.2020.3017513	journal	January 2022
A Two-Layer Potential-Field-Driven Model Predictive Shared Control Towards Driver-Automation Cooperation Li, Mingjun; Cao, Haotian; Li, Guofa IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 5 https://doi.org/10.1109/TITS.2020.3044666	journal	May 2022
Adaptive Steering Torque Coupling Framework Considering Conflict Resolution for Human-Machine Shared Driving Han, Jiayi; Zhao, Jian; Zhu, Bing IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 8 https://doi.org/10.1109/TITS.2021.3098466	journal	August 2022
Intelligent Driver Drowsiness Detection for Traffic Safety Based on Multi CNN Deep Model and Facial Subsampling Ahmed, Muneeb; Masood, Sarfaraz; Ahmad, Musheer IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 10 https://doi.org/10.1109/TITS.2021.3134222	journal	October 2022
Trajectory Tracking of Autonomous Vehicle Based on Model Predictive Control With PID Feedback Chu, Duanfeng; Li, Haoran; Zhao, Chenyang IEEE Transactions on Intelligent Transportation Systems https://doi.org/10.1109/TITS.2022.3150365	journal	January 2022
Integrated Longitudinal and Lateral Vehicle Stability Control for Extreme Conditions With Safety Dynamic Requirements Analysis Li, Zihan; Chen, Hong; Liu, Hanghang IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 10 https://doi.org/10.1109/TITS.2022.3152485	journal	October 2022
Coordination Control Strategy for Human-Machine Cooperative Steering of Intelligent Vehicles: A Reinforcement Learning Approach Xie, Ju; Xu, Xing; Wang, Feng IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 11 https://doi.org/10.1109/TITS.2022.3187016	journal	November 2022
Driver’s Individual Risk Perception-Based Trajectory Planning: A Human-Like Method Yan, Yongjun; Wang, Jinxiang; Zhang, Kuoran IEEE Transactions on Intelligent Transportation Systems, Vol. 23, Issue 11 https://doi.org/10.1109/TITS.2022.3190521	journal	November 2022
Sensor Reduction for Driver-Automation Shared Steering Control via an Adaptive Authority Allocation Strategy Nguyen, Anh-Tu; Sentouh, Chouki; Popieul, Jean-Christophe IEEE/ASME Transactions on Mechatronics, Vol. 23, Issue 1 https://doi.org/10.1109/TMECH.2017.2698216	journal	February 2018
Takagi–Sugeno Fuzzy Unknown Input Observers to Estimate Nonlinear Dynamics of Autonomous Ground Vehicles: Theory and Real-Time Verification Nguyen, Anh-Tu; Dinh, Truong Quang; Guerra, Thierry-Marie IEEE/ASME Transactions on Mechatronics, Vol. 26, Issue 3 https://doi.org/10.1109/TMECH.2020.3049070	journal	June 2021
Functional Connectivity Analysis of Mental Fatigue Reveals Different Network Topological Alterations Between Driving and Vigilance Tasks Dimitrakopoulos, Georgios N.; Kakkos, Ioannis; Dai, Zhongxiang IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 26, Issue 4 https://doi.org/10.1109/TNSRE.2018.2791936	journal	April 2018
Dynamic Reorganization of Functional Connectivity Unmasks Fatigue Related Performance Declines in Simulated Driving Wang, Hongtao; Liu, Xucheng; Hu, Hongying IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 28, Issue 8 https://doi.org/10.1109/TNSRE.2020.2999599	journal	August 2020
Driver-Assistance Lateral Motion Control for In-Wheel-Motor-Driven Electric Ground Vehicles Subject to Small Torque Variation Chen, Yimin; Stout, Corwin; Joshi, Adit IEEE Transactions on Vehicular Technology, Vol. 67, Issue 8 https://doi.org/10.1109/TVT.2018.2817514	journal	August 2018
Fault-Tolerant Control for Electric Vehicles With Independently Driven in-Wheel Motors Considering Individual Driver Steering Characteristics Zhang, Han; Zhao, Wanzhong; Wang, Junmin IEEE Transactions on Vehicular Technology, Vol. 68, Issue 5 https://doi.org/10.1109/TVT.2019.2904698	journal	May 2019
Driver Activity Recognition for Intelligent Vehicles: A Deep Learning Approach Xing, Yang; Lv, Chen; Wang, Huaji IEEE Transactions on Vehicular Technology, Vol. 68, Issue 6 https://doi.org/10.1109/TVT.2019.2908425	journal	June 2019
Adaptive Steering Feedback Torque Design and Control for Driver–Vehicle System Considering Driver Handling Properties Jiang, Yuyao; Deng, Weiwen; Wu, Jian IEEE Transactions on Vehicular Technology, Vol. 68, Issue 6 https://doi.org/10.1109/TVT.2019.2908987	journal	June 2019
A Distributed Integrated Control Architecture of AFS and DYC Based on MAS for Distributed Drive Electric Vehicles Liang, Jinhao; Lu, Yanbo; Yin, Guodong IEEE Transactions on Vehicular Technology, Vol. 70, Issue 6 https://doi.org/10.1109/TVT.2021.3076105	journal	June 2021
A Novel Learning Model of Driver Fatigue Features Representation for Steering Wheel Angle Li, Zuojin; Chen, Liukui; Nie, Ling IEEE Transactions on Vehicular Technology, Vol. 71, Issue 1 https://doi.org/10.1109/TVT.2021.3130152	journal	January 2022
Shared Control for Intelligent Vehicle Based on Handling Inverse Dynamics and Driving Intention Deng, Huifan; Zhao, Youqun; Feng, Shilin IEEE Transactions on Vehicular Technology, Vol. 71, Issue 3 https://doi.org/10.1109/TVT.2022.3143665	journal	March 2022
Highway Decision-Making and Motion Planning for Autonomous Driving via Soft Actor-Critic Tang, Xiaolin; Huang, Bing; Liu, Teng IEEE Transactions on Vehicular Technology, Vol. 71, Issue 5 https://doi.org/10.1109/TVT.2022.3151651	journal	May 2022
Real-Time Driver Behavior Detection Based on Deep Deformable Inverted Residual Network With an Attention Mechanism for Human-Vehicle Co-Driving System Huang, Tao; Fu, Rui; Chen, Yunxing IEEE Transactions on Vehicular Technology, Vol. 71, Issue 12 https://doi.org/10.1109/TVT.2022.3195230	journal	December 2022
Regional brain wave activity changes associated with fatigue Craig, Ashley; Tran, Yvonne; Wijesuriya, Nirupama Psychophysiology, Vol. 49, Issue 4 https://doi.org/10.1111/j.1469-8986.2011.01329.x	journal	February 2012
Motor learning affects car-to-driver handover in automated vehicles Russell, Holly E. B.; Harbott, Lene K.; Nisky, Ilana Science Robotics, Vol. 1, Issue 1 https://doi.org/10.1126/scirobotics.aah5682	journal	December 2016
Robust steering assistance control for tracking large-curvature path considering uncertainties of driver’s steering behavior Wang, Jinxiang; Fang, Zhenwu; Dai, Mengmeng Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, Vol. 235, Issue 7 https://doi.org/10.1177/0954407020976827	journal	January 2021
Automatic Detection of Driver Fatigue Using Driving Operation Information for Transportation Safety Li, Zuojin; Chen, Liukui; Peng, Jun Sensors, Vol. 17, Issue 6 https://doi.org/10.3390/s17061212	journal	May 2017
Effects of Human Adaptation and Trust on Shared Control for Driver-Automation Cooperative Driving Li, Renjie; Li, Shengbo; Gao, Hongbo SAE Technical Paper Series https://doi.org/10.4271/2017-01-1987	conference	September 2017

Similar Records

A Human-Machine Shared Control Framework Considering Time-Varying Driver Characteristics

Journal Article · 2023 · IEEE Transactions on Intelligent Vehicles · OSTI ID:1972558

Fang, Zhenwu; Wang, Jinxiang; Wang, Zejiang; +3 more

Driver Distraction Behavior Detection using a Vision Transformer Model based on Transfer Learning Strategy

Conference · 2022 · OSTI ID:1961946

Fang, Zhenwu; Chen, Jinxin; Wang, Jinxiang; +3 more

Vehicle-to-Grid Automatic Load Sharing with Driver Preference in Micro-Grids

Conference · 2014 · OSTI ID:1332690

Wang, Yubo; Nazaripouya, Hamidreza; Chu, Chi-Cheng; +2 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Human-machine shared control
control authority
driver model
fatigue driving

Title: Human-Machine Shared Control for Path Following Considering Driver Fatigue Characteristics

Citation Formats

References (45)

Similar Records

Related Subjects