[1]张奇良,杨坤华,曲行达,等.基于多模态生理信号的驾驶人脑力负荷评估[J].深圳大学学报理工版,2022,39(3):278-286.[doi:10.3724/SP.J.1249.2022.03278]
 ZHANG Qiliang,YANG Kunhua,QU Xingda,et al.Evaluation of drivers’ mental workload based on multi-modal physiological signals[J].Journal of Shenzhen University Science and Engineering,2022,39(3):278-286.[doi:10.3724/SP.J.1249.2022.03278]
点击复制

基于多模态生理信号的驾驶人脑力负荷评估()
分享到:

《深圳大学学报理工版》[ISSN:1000-2618/CN:44-1401/N]

卷:
第39卷
期数:
2022年第3期
页码:
278-286
栏目:
交通物流
出版日期:
2022-05-16

文章信息/Info

Title:
Evaluation of drivers’ mental workload based on multi-modal physiological signals
文章编号:
202203006
作者:
张奇良 杨坤华 曲行达 陶达
深圳大学人因工程研究所,广东深圳 518060
Author(s):
ZHANG Qiliang YANG Kunhua QU Xingda and TAO Da
Institute of Human Factors and Ergonomics, Shenzhen University, Shenzhen 518060, Guangdong Province, P. R. China
关键词:
交通信息工程及控制脑力负荷机器学习多模态脑电信号心电信号皮电信号
Keywords:
traffic information engineering and control mental workload machine learning multi-modal electroencephalographic signal electrocardiographic signal electrodermal activity signal
分类号:
U491;TN911.72
DOI:
10.3724/SP.J.1249.2022.03278
文献标志码:
A
摘要:
准确评估驾驶人脑力负荷状态对降低因驾驶人脑力负荷过载导致的交通事故具有重要意义.基于典型驾驶场景,结合N-back认知负荷次任务,设计不同难度的驾驶任务实验,研究驾驶人脑力负荷.实验收集驾驶人在任务完成过程中的多种模态生理信号(脑电、心电和皮电信号)及美国航空航天局任务负荷指数量表主观脑力负荷数据,提出基于多模态生理信号特征分析和模式识别的驾驶人脑力负荷分类模型,并比较不同模态生理信号及其组合在3种典型机器学习算法(随机森林、决策树和k最近邻模型)中的脑力负荷分类识别效果.研究表明,基于不同模态生理信号组合的脑力负荷分类模型具有不同的分类准确率.单一模态生理信号的分类模型中,基于皮电、心电和脑电信号的分类模型准确率依次增加;基于多模态生理信号的分类模型准确率普遍优于单一模态分类模型;基于脑电、心电及皮电3模态生理信号的随机森林分类模型具有最高的分类准确率.
Abstract:
Accurately assessing the driver’s mental workload is of great significance to reduce the traffic accidents caused by the driver’s mental overload. This study aims to evaluate drivers’ mental workload in simulated typical driving scenarios, with N-back cognitive tasks used to manipulate varied levels of task difficulty. We collect data on multi-modal physiological signals including electroencephalogram (EEG), electrocardiogram (ECG), and electrodermal activity (EDA) signals, and subjective mental load of the National Aeronautics and Space Administration task load index (NASA_TLX) during the task completion process of the driver in the experiment, and propose a series of mental workload classification models based on feature analysis and pattern recognition of the multi-modal physiological signals. These classification models are verified by machine learning algorithms of random forest, decision tree and k-nearest neighbor models. The results show that the accuracy of classification models varies with different modalities of physiological signals. EEG-based classification models yield the highest accuracy among single-modal classification models, followed by EDA-based and ECG-based models. Multi-modal-based classification models generally perform better than single-modal classification models. The random forest classification algorithm based on three-modal physiological signals of EEG, ECG and EDA has the highest classification accuracy.

参考文献/References:

[1] 郑玲,乔旭强,倪涛,等. 基于多维信息特征分析的驾驶人认知负荷研究[J]. 中国公路学报,2021,34(4):240-250. ZHENG Ling, QIAO Xuqiang, NI Tao, et al. Driver congnitive loads based on multi-dimensional information feature analysis [J]. China Journal of Highway and Transport, 2021, 34(4): 240-250. (in Chinese) [2] National Safety Council. Understanding the distracted brain: why driving while using hands-free cell phones is risky behavior [R]. Spring Lake: National Safety Council, 2012. [3] GALY E. Consideration of several mental workload categories: perspectives for elaboration of new ergonomic recommendations concerning shiftwork [J]. Theoretical Issues in Ergonomics Science, 2018, 19(4): 483-497. [4] DONG Yanchao, HU Zhencheng, UCHIMURA K, et al. Driver inattention monitoring system for intelligent vehicles: a review [J]. IEEE Transactions on Intelligent Transportation Systems, 2010, 12(2): 596-614. [5] DI FLUMERI G, BORGHINI G, ARIC? P, et al. EEG-based mental workload neurometric to evaluate the impact of different traffic and road conditions in real driving settings [J]. Frontiers in Human Neuroscience, 2018, 12: 509. [6] JIMENEZ-MOLINA A, RETAMAL C, LIRA H. Using psychophysiological sensors to assess mental workload in web browsing [J]. Sensors, 2017, 18(2): 458. [7] OMURTAG A, ROY R N, DEHAIS F, et al. Tracking team mental workload by multimodal measurements in the operating room [M/OL]. Neuroergonomics-The Brain at Work and in Everyday Life. New York, USA: 2019: 99-103. [8] KOSTI M A, GEORGIADIS K, ADAMOS D A, et al. Towards an affordable brain computer interface for the assessment of programmers’ mental workload [J]. International Journal of Human Computer Studies, 2018, 115: 52-66. [9] DIAZ-PIEDRA C, SEBASTI?N M V, DI STASI L L. EEG theta power activity reflects workload among army combat drivers: an experimental study [J]. Brain Sciences, 2020, 10(4): 199. [10] DE RIVECOURT M, KUPERUS M, POST W, et al. Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight [J]. Ergonomics, 2008, 51(9): 1295-1319. [11] TAO Da, TAN Haibo, WANG Hailiang, et al. A systematic review of physiological measures of mental workload [J]. International Journal of Environmental Research and Public Health, 2019, 16(15): 2716. [12] HWANG S L, YAU Y J, LIN Y T, et al. Predicting work performance in nuclear power plants [J]. Safety Science, 2008, 46(7): 1115-1124. [13] FOY H J, CHAPMAN P J A E. Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation [J]. Applied Ergonomics, 2018, 73: 90-99. [14] DING Yi, CAO Yaqin, DUFFY V G, et al. Measurement and identification of mental workload during simulated computer tasks with multimodal methods and machine learning [J]. Ergonomics, 2020, 63(7): 896-908. [15] DEBIE E, ROJAS R F, FIDOCK J, et al. Multimodal fusion for objective assessment of cognitive workload: a review [J]. IEEE Transactions on Cybernetics, 2019, 51(3): 1542-1555. [16] CINAZ B, ARNRICH B, LA MARCA R, et al. Monitoring of mental workload levels during an everyday life office-work scenario [J]. Personal Ubiquitous Computing, 2013, 17(2): 229-239. [17] MATTHEWS G, REINERMAN-JONES L E, BARBER D J, et al. The psychometrics of mental workload: multiple measures are sensitive but divergent [J]. Human Factors, 2015, 57(1): 125-143. [18] ZHAO Guozhen, LIU Yongjin, SHI Yuanchun. Real-time assessment of the cross-task mental workload using physiological measures during anomaly detection [J]. IEEE Transactions on Human-Machine Systems, 2018, 48(2): 149-160. [19] HOLZINGER A, KIESEBERG P, WEIPPL E, et al. Current advances, trends and challenges of machine learning and knowledge extraction: from machine learning to explainable AI [C]// International Cross-Domain Conference for Machine Learning and Knowledge Extraction. Hamburg, Germany: Springer, 2018: 1-8. [20] KOENIG A, REHG T, RASSHOFER R. Statistical sensor fusion of ECG data using automotive-grade sensors [J]. Advances in Radio Science, 2015, 13: 197-202. [21] YIN Zhong, ZHANG Jianhua. Operator functional state classification using least-square support vector machine based recursive feature elimination technique [J]. Computer Methods & Programs in Biomedicine, 2014, 113(1): 101-115. [22] YAN Shengyuan, TRAN C C, WEI Yingying, et al. Driver’s mental workload prediction model based on physiological indices [J]. International Journal of Occupational Safety & Ergonomics, 2017, 25(2): 1-37. [23] DAMOS D. Multiple-task performance [M]. Hants, UK: Taylor and Francis, 1991. [24] HEINE T, LENIS G, REICHENSPERGER P, et al. Electrocardiographic features for the measurement of drivers’ mental workload [J]. Applied Ergonomics, 2017, 61: 31-43. [25] FENG Chuanyan, WANYAN Xiaoru, YANG Kun, et al. A comprehensive prediction and evaluation method of pilot workload [J]. Technology and Health Care, 2018, 26(Suppl.1): 65-78. [26] BIAU G, SCORNET E. A random forest guided tour [J]. Test, 2016, 25(2): 197-227. [27] SO W K, WONG S W, MAK J N, et al. An evaluation of mental workload with frontal EEG [J]. PloS One, 2017, 12(4): e0174949. [28] ZHANG Minling, ZHOU Zhihua. ML-KNN: a lazy learning approach to multi-label learning [J]. Pattern Recognition, 2007, 40(7): 2038-2048. [29] FAN Xiaoli, ZHAO Chaoyi, ZHANG Xin, et al. Assessment of mental workload based on multi-physiological signals [J]. Technology Health Care, 2020, 28(Suppl.1): 67-80. [30] ISLAM M R, BARUA S, AHMED M U, et al. A novel mutual information based feature set for drivers’ mental workload evaluation using machine learning [J]. Brain Sciences, 2020, 10(8): 551.

备注/Memo

备注/Memo:
Received: 2021-06-15; Revised: 2021-11-31; Accepted: 2021-12-30; Online (CNKI): 2022-03-01 Foundation: Shenzhen Basic Research Foundation for General Projects (20200813225029002); National Natural Science Foundation of China (72101161) Corresponding author: Assistant professor TAO Da. E-mail: taoda@szu.edu.cn Citation: ZHANG Qiliang, YANG Kunhua, QU Xingda, et al. Evaluation of drivers’ mental workload based on multi-modal physiological signals [J]. Journal of Shenzhen University Science and Engineering, 2022, 39(3): 278-286.(in Chinese) 基金项目:深圳市基础研究面上计划资助项目(2020081322502 9002);国家自然科学基金资助项目(72101161) 作者简介:张奇良(1996—),深圳大学硕士研究生.研究方向:人因工程.E-mail: 747826028@qq.com 引 文:引用格式:张奇良,杨坤华,曲行达,等.基于多模态生理信号的驾驶人脑力负荷评估[J].深圳大学学报理工版,2022,39(3):278-286.
更新日期/Last Update: 2022-05-30