近红外光谱技术在航空心理学研究中的应用与展望 &#x0002A;

刘煜; 潘盈朵; 李萌; 李晨麟; 王新野; 游旭群

doi:10.16719/j.cnki.1671-6981.20230631

PDF(515 KB)

心理科学 ›› 2023, Vol. 46 ›› Issue (6) : 1518-1528. DOI: 10.16719/j.cnki.1671-6981.20230631

理论与史

近红外光谱技术在航空心理学研究中的应用与展望 ^*

刘煜^1,2, 潘盈朵^1,2, 李萌^1,2, 李晨麟^1,2, 王新野^**1,2, 游旭群^**1,2

作者信息 +

The Application and Prospect of Functional Near-Infrared Spectroscopy in Aviation Psychology

Liu Yu^1,2, Pan Yingduo^1,2, Li Meng^1,2, Li Chenlin^1,2, Wang Xinye^1,2, You Xuqun^1,2

Author information +

文章历史 +

摘要

功能性近红外光谱技术（functional near-infrared spectroscopy,fNIRS）可以连续、无创、便携地检测与人脑功能相关的血液动力学变化,使研究具有较好的生态效度,研究者利用血液动力学变化等信息推断飞行员在飞行场景下的认知状态。本文对相关研究的研究目的、研究任务、所用滤波和脑机接口的分类算法进行了系统性综述,在此基础上提出fNIRS应用于飞行员选拔与训练以及机组资源管理的方向性建议,以此推广fNIRS在航空心理学研究中的应用,最终直接或间接地服务于航空安全。

Abstract

As the operational subject of aviation activities, the cognitive activity of pilots has a significant impact on flight safety. In decades, researchers are unable to obtain information about the neural activity of pilots during simulated or real flights efficiently because of the limitation of technology, which greatly hinders the understanding of the mental processing processes and the way pilots work. The development and application of functional near-infrared spectroscopy (fNIRS) provides a solution to this problem. It allows continuous, non-invasive, portable monitoring of hemodynamic changes associated with human brain function. Compared to the physical limitations of functional magnetic resonance imaging (fMRI) and the low spatial resolution of the electroencephalogram (EEG), the portability and hypersensitivity to the body movement of fNIRS makes it more suitable for detecting cortical hemodynamics during exercise tasks. The brain functional activation and functional connectivity states of pilots in different cognitive task demand states can be revealed through fNIRS to provide a realistic basis for neuroergonomics studies of flight safety, and provide valid physiological neurological indicators for identifying different cognitive states. To this end, this paper reviewed studies in aviation psychology applying fNIRS, and summarized some general findings that can provide reference and guidance for maintaining aviation safety.

The aeropsychological studies applying fNIRS can be divided into three themes: (1) The usability studies of fNIRS in the assessment of brain activation patterns, which focus on mental load assessment and skill training assessment. The assessment of mental load refers to the detection of brain activation patterns under different task difficulties (i.e., the examination of the similarities and differences of brain activation patterns under different mental loads), aiming to confirm the reliability of fNIRS in assessing mental load. The assessment of skill training refers to the use of fNIRS to measure the changes of brain activation patterns of individuals during the same training task and the differences of brain activation patterns of different individuals on the same training task. (2) The accuracy studies of fNIRS in brain activation pattern recognition. That is, using the neural signals evoked by the task as brain activation indicators, combined with specific statistical methods (e.g., machine learning algorithms) to perform discriminative analysis at the individual level to correctly classify different mental loads. (3) The applicability study of fNIRS in special events, emphasizing the use of fNIRS to detect brain tissue blood oxygenation in pilots experiencing special events (e.g., loss of consciousness under high +Gz) to reveal the neurophysiological mechanisms.

With the widespread use of fNIRS, more relevant research can be conducted around pilots. It can be used in pilot selection and training, flight cockpit design, and crew cooperation. In addition, fNIRS also can be used to assess pilots'cognitive load states and mental fatigue as a staffing tool. When focusing on the impact of workload on individual performance, a pilot's emotional perceptions or preferences should be seriously considered. Typically, we monitor individual pilots'workload to measure the blood oxygern concentration changes in the prefrontal cortex, which is an important region that facilitates emotion regulation, in individual pilots. Functional neuroimaging of the prefrontal cortex has been applied to study the neural correlates of emotional processing. In terms of fNIRS device configuration, the addition of an emotional computing interface to the workload interface needs to be considered, so that the cognitive state of pilots can be monitored and identified in terms of both workload and emotional experience. In addition, passive brain-computer interface (pBCI) technology and multimodal technology can be applied in the future in real environments to assess the mental state of pilots and team resources in real flight and to dynamically plan programming pilots and automated interactions.

Finally, in the future, fNIRS research should also focus on methodological advances (e.g., signal quality control, motion artifact correction strategies, etc.), and standardization of the analysis process. On the one hand, it can provide guidance for future fNIRS studies on pilots; on the other hand, it can help promote the application of fNIRS in aviation psychology research, which ultimately serves aviation safety directly or indirectly.

导出引用

刘煜, 潘盈朵, 李萌, 李晨麟, 王新野, 游旭群. 近红外光谱技术在航空心理学研究中的应用与展望 ^*[J]. 心理科学. 2023, 46(6): 1518-1528 https://doi.org/10.16719/j.cnki.1671-6981.20230631

Liu Yu, Pan Yingduo, Li Meng, Li Chenlin, Wang Xinye, You Xuqun. The Application and Prospect of Functional Near-Infrared Spectroscopy in Aviation Psychology[J]. Journal of Psychological Science. 2023, 46(6): 1518-1528 https://doi.org/10.16719/j.cnki.1671-6981.20230631

参考文献

[1] 白学军, 章鹏, 张琪涵, 宋璐, 杨宇. (2019). 功能性近红外光谱技术在说谎研究中的应用. 心理科学进展, 27(1), 160-170.
[2] 包晓宁, 赵培林, 张保中, 胡志文, 兰于清, 薛飞. (2020). 先进战斗机生命保障系统. 航空学报, 41(6), 523464.
[3] 金朝, 曹征涛, 王海霞, 张立辉, 李毅峰, 周玉斌, 耿喜臣. (2020). 慢长率模式+Gz加速度暴露对局部脑血氧饱和度的影响. 中国应用生理学杂志, 36(2), 130-133.
[4] 叶佩霞, 朱睿达, 唐红红, 买晓琴, 刘超. (2017). 近红外光学成像在社会认知神经科学中的应用. 心理科学进展, 25(5), 731-741.
[5] 游旭群. (2017). 航空心理学: 理论、实践与应用. 杭州:浙江教育出版社.
[6] Ahlstrom, U. (2015). Weather display symbology affects pilot behavior and decision-making. International Journal of Industrial Ergonomics, 50, 73-96.
[7] Ahlstrom U., Ohneiser O., & Caddigan E. (2016). Portable weather applications for general aviation pilots. Human Factors, 58(6), 864-885.
[8] Aricò P., Borghini G., Di Flumeri G., Sciaraffa N., & Babiloni F. (2018). Passive BCI beyond the lab: Current trends and future directions. Physiological Measurement, 39(8), Article 08TR02
[9] Ayaz H., Shewokis P. A., Bunce S., Izzetoglu K., Willems B., & Onaral B. (2012). Optical brain monitoring for operator training and mental workload assessment. NeuroImage, 59(1), 36-47.
[10] Bennett, S. A. (2019). The training and practice of crew resource management: Recommendations from an inductive in vivo study of the flight deck. Ergonomics, 62(2), 219-232.
[11] Borghini G., Astolfi L., Vecchiato G., Mattia D., & Babiloni F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience and Biobehavioral Reviews, 44, 58-75.
[12] Bouak F., Vartanian O., & Hofer K. (2019). Performance and health effects of mild hypoxic hypoxia in simulated 6-hour exposures between 2,438 and 3,048 metres. Journal of Military, Veteran and Family Health, 5(2), 40-49.
[13] Bouak F., Vartanian O., Hofer K., & Cheung B. (2018). Acute mild hypoxic hypoxia effects on cognitive and simulated aircraft pilot performance. Aerospace Medicine and Human Performance, 89(6), 526-535.
[14] Brigadoi S., Ceccherini L., Cutini S., Scarpa F., Scatturin P., Selb J., & Cooper R. J. (2014). Motion artifacts in functional near-infrared spectroscopy: A comparison of motion correction techniques applied to real cognitive data. NeuroImage, 85, 181-191.
[15] Çakır M. P., Vural M., Koç S. Ö., & Toktaş A. (2016). Real-time monitoring of cognitive workload of airline pilots in a flight simulator with fNIR optical brain imaging technology. In D. Schmorrow & C. Fidopiastis (Eds.), Foundations of augmented cognition: Neuroergonomics and operational neuroscience (pp. 147-158). Springer.
[16] Causse M., Chua Z., Peysakhovich V., Del Campo N., & Matton N. (2018). Author Correction: Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS (vol 7, 5222, 2017). Scientific Reports, 8, Article 7184.
[17] Causse M., Chua Z. K., & Rémy F. (2019). Influences of age, mental workload, and flight experience on cognitive performance and prefrontal activity in private pilots: A fNIRS study. Scientific Reports, 9, Article 7688.
[18] Causse M., Dehais F., Péran P., Sabatini U., & Pastor J. (2013). The effects of emotion on pilot decision-making: A neuroergonomic approach to aviation safety. Transportation Research Part C: Emerging Technologies, 33, 272-281.
[19] Causse M., Peysakhovich V., & Mandrick K. (2017). Eliciting sustained mental effort using the toulouse N-back task: Prefrontal cortex and pupillary responses. In K. Hale & K. Stanney (Eds.), Advances in neuroergonomics and cognitive engineering(pp. 185-193). Springer.
[20] Choe J., Coffman B. A., Bergstedt D. T., Ziegler M. D., & Phillips M. E. (2016). Transcranial direct current stimulation modulates neuronal activity and learning in pilot training. Frontiers in Human Neuroscience, 10, Article 34.
[21] Cooper R. J., Selb J., Gagnon L., Phillip D., Schytz H. W., Iversen H. K., & Boas D. A. (2012). A systematic comparison of motion artifact correction techniques for functional near-infrared spectroscopy. Frontiers in Neuroscience, 6, Article 147.
[22] Cui X., Bryant D. M., & Reiss A. L. (2012). Nirs-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation. NeuroImage, 59(3), 2430-2437.
[23] Dehais F., Dupres A., Di Flumeri G., Verdiere K., Borghini G., Babiloni F., & Roy R. (2018). Monitoring pilot's cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fnirs-eeg passive BCI. Paper presented at the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan.
[24] Doi H., Nishitani S., & Shinohara K. (2013). NIRS as a tool for assaying emotional function in the prefrontal cortex. Frontiers in Human Neuroscience, 7, Article 770.
[25] Durantin G., Gagnon J. F., Tremblay S., & Dehais F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behavioural Brain Research, 259, 16-23.
[26] Durantin G., Scannella S., Gateau T., Delorme A., & Dehais F. (2016). Processing functional near infrared spectroscopy signal with a kalman filter to assess working memory during simulated flight. Frontiers in Human Neuroscience, 9, Article 707.
[27] Fresnel E., Dray G., Pla S., Jean P., Belda G., & Perrey S. (2021). Cerebral oxygenation responses to aerobatic flight. Aerospace Medicine and Human Performance, 92(10), 838-842.
[28] Gateau T., Ayaz H., & Dehais F. (2018). In silico vs. over the clouds: On-the-fly mental state estimation of aircraft pilots, using a functional near infrared spectroscopy based passive-BCI. Frontiers in Human Neuroscience, 12, Article 187.
[29] Gateau T., Durantin G., Lancelot F., Scannella S., & Dehais F. (2015). Real-time state estimation in a flight simulator using fNIRS. PLoS ONE, 10(3), Article e0121279.
[30] Gerega A., Wojtkiewicz S., Sawosz P., Kacprzak M., Toczylowska B., Bejm K., & Liebert A. (2020). Assessment of the brain ischemia during orthostatic stress and lower body negative pressure in air force pilots by near-infrared spectroscopy. Biomedical Optics Express, 11(2), 1043-1060.
[31] Grabner R. H., Neubauer A. C., & Stern E. (2006). Superior performance and neural efficiency: The impact of intelligence and expertise. Brain Research Bulletin, 69(4), 422-439.
[32] Hamann, A., & Carstengerdes, N. (2022). Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Scientific Reports, 12(1), Article 6449.
[33] Hasson, U., & Frith, C. D. (2016). Mirroring and beyond: Coupled dynamics as a generalized framework for modelling social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1693), Article 20150366.
[34] Hernandez-Meza G., Slason L., Ayaz H., Craven P., & Izzetoglu K. (2015). Investigation of functional near infrared spectroscopy in evaluation of pilot expertise acquisition. In D. D. Schmorrow & C. M. Fidopiastis (Eds.), Foundations of augmented cognition (pp. 232-243). Springer.
[35] Hu X., Zhuang C., Wang F., Liu Y. J., Im C. H., & Zhang D. (2019). fNIRS evidence for recognizably different positive emotions. Frontiers in Human Neuroscience, 13, Article 120.
[36] Izzetoglu M., Izzetoglu K., Bunce S., Ayaz H., Devaraj A., Onaral B., & Pourrezaei K. (2005). Functional near-infrared neuroimaging. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13(2), 153-159.
[37] Kelly, A. M. C., & Garavan, H. (2005). Human functional neuroimaging of brain changes associated with practice. Cerebral Cortex, 15(8), 1089-1102.
[38] Kikukawa A., Kobayashi A., & Miyamoto Y. (2008). Monitoring of pre-frontal oxygen status in helicopter pilots using near-infrared spectrophotometers. Dynamic Medicine, 7, Article 10.
[39] Kobayashi A., Kikukawa A., Kimura M., Inui T., & Miyamoto Y. (2012). Cerebral near-infrared spectroscopy to evaluate anti-G straining maneuvers in centrifuge training. Aviation, Space, and Environtal Medicine, 83(8), 790-794.
[40] Kurihara K., Kikukawa A., Kobayashi A., & Nakadate T. (2007). Frontal cortical oxygenation changes during gravity-induced loss of consciousness in humans: A near-infrared spatially resolved spectroscopic study. Journal of Applied Physiology, 103(4), 1326-1331.
[41] Li G. H., Baker S. P., Grabowski J. G., & Rebok G. W. (2001). Factors associated with pilot error in aviation crashes. Aviation, Space, and Environmental Medicine, 72(1), 52-58.
[42] Li W. B., Li R., Xie X. P., & Chang Y. M. (2022). Evaluating mental workload during multitasking in simulated flight. Brain and Behavior, 12(4), Article e2489.
[43] Mandrick K., Derosiere G., Dray G., Coulon D., Micallef J. P., & Perrey S. (2013). Prefrontal cortex activity during motor tasks with additional mental load requiring attentional demand: A near-infrared spectroscopy study. Neuroscience Research, 76(3), 156-162.
[44] McKendrick R., Ayaz H., Olmstead R., & Parasuraman R. (2014). Enhancing dual-task performance with verbal and spatial working memory training: Continuous monitoring of cerebral hemodynamics with NIRS. NeuroImage, 85, 1014-1026.
[45] Menda J., Hing J. T., Ayaz H., Shewokis P. A., Izzetoglu K., Onaral B., & Oh P. (2011). Optical brain imaging to enhance UAV operator training, evaluation, and interface development. Journal of Intelligent and Robotic Systems, 61(1-4), 423-443.
[46] Murai K., Hayashi Y., Okazaki T., Stone L. C., & Mitomo N. (2008). Evaluation of ship navigator's mental workload using nasal temperature and heart rate variability. Paper presented at the 2008 IEEE International Conference on Systems, Man and Cybernetics, Singapore.
[47] Neubauer, A. C., & Fink, A. (2009). Intelligence and neural efficiency. Neuroscience and Biobehavioral Reviews, 33(7), 1004-1023.
[48] Nozawa T., Sasaki Y., Sakaki K., Yokoyama R., & Kawashima R. (2016). Interpersonal frontopolar neural synchronization in group communication: An exploration toward fNIRS hyperscanning of natural interactions. NeuroImage, 133, 484-497.
[49] Paas F., Tuovinen J. E., Tabbers H., & van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63-71.
[50] Pan T., Wang H. B., Si H. Q., Liu H. B., & Xu M. Y. (2022). Research on the identification of pilots' fatigue status based on functional near-infrared spectroscopy. Aerospace, 9(3), Article 173.
[51] Pan Y. F., Borragán G., & Peigneux P. (2019). Applications of functional near-infrared spectroscopy in fatigue, sleep deprivation, and social cognition. Brain Topography, 32(6), 998-1012.
[52] Parasuraman, R. (2011). Neuroergonomics. Current Directions in Psychological Science, 20(3), 181-186.
[53] Phillips J. B., Hørning D., & Funke M. E. (2015). Cognitive and perceptual deficits of normobaric hypoxia and the time course to performance recovery. Aerospace Medicine And Human Performance, 86(4), 357-365.
[54] Pinti P., Aichelburg C., Gilbert S., Hamilton A., Hirsch J., Burgess P., & Tachtsidis I. (2018). A review on the use of wearable functional near-infrared spectroscopy in naturalistic environments. Japanese Psychological Research, 60(4), 347-373.
[55] Reddy P., Shewokis P. A., & Izzetoglu K. (2022). Individual differences in skill acquisition and transfer assessed by dual task training performance and brain activity. Brain Informatics, 9(1), Article 9.
[56] Ryoo H. C., Sun H. H., Shender B. S., & Hrebien L. (2004). Consciousness monitoring using near-infrared spectroscopy (NIRS) during high + Gz exposures. Medical Engineering and Physics, 26(9), 745-753.
[57] Scarapicchia V., Brown C., Mayo C., & Gawryluk J. R. (2017). Functional magnetic resonance imaging and functional near-infrared spectroscopy: Insights from combined recording studies. Frontiers in Human Neuroscience, 11, Article 419.
[58] Sun J. C., Cheng S., Ma J., Xiong K. W., Su M., & Hu W. D. (2019). Assessment of the static upright balance index and brain blood oxygen levels as parameters to evaluate pilot workload. PLoS ONE, 14(3), Article e0214277.
[59] Szufladowicz E., Maniewski R., Kozluk E., Zbiec A., Nosek A., & Walczak F. (2004). Near-infrared spectroscopy in evaluation of cerebral oxygenation during vasovagal syncope. Physiological Measurement, 25(4), 823-836.
[60] Takeuchi, Y. (2000). Change in blood volume in the brain during a simulated aircraft landing task. Journal of Occupational Health, 42(2), 60-65.
[61] Tang H. H., Mai X., Wang S., Zhu C. Z., Krueger F., & Liu C. (2016). Interpersonal brain synchronization in the right temporo-parietal junction during face-to-face economic exchange. Social Cognitive and Affective Neuroscience, 11(1), 23-32.
[62] Toni I., Krams M., Turner R., & Passingham R. E. (1998). The time course of changes during motor sequence learning: A whole-brain fMRI study. NeuroImage, 8(1), 50-61.
[63] Toppi J., Borghini G., Petti M., He E. J., De Giusti V., He B., & Babiloni F. (2016). Investigating cooperative behavior in ecological settings: An EEG hyperscanning study. PLoS ONE, 11(4), Article e0154236.
[64] Tripp L. D., Warm J. S., Matthews G., Chiu P. Y., & Bracken R. B. (2009). On tracking the course of cerebral oxygen saturation and pilot performance during gravity-induced loss of consciousness. Human Factors, 51(6), 775-784.
[65] van den Heuvel, M. P., & Sporns, O. (2013). Network hubs in the human brain. Trends in Cognitive Sciences, 17(12), 683-696.
[66] Verdière K. J., Roy R. N., & Dehais F. (2018). Detecting pilot's engagement using fNIRS connectivity features in an automated vs. manual landing scenario. Frontiers in Human Neuroscience, 12, Article 6.
[67] Wang M. Y., Luan P., Zhang J., Xiang Y. T., Niu H. J., & Yuan Z. (2018). Concurrent mapping of brain activation from multiple subjects during social interaction by hyperscanning: A mini-review. Quantitative Imaging in Medicine and Surgery, 8(8), 819-837.
[68] Young M. S., Brookhuis K. A., Wickens C. D., & Hancock P. A. (2015). State of science: Mental workload in ergonomics. Ergonomics, 58(1), 1-17.
[69] Young, M. S., & Stanton, N. A. (2004). Taking the load off: Investigations of how adaptive cruise control affects mental workload. Ergonomics, 47(9), 1014-1035.
[70] Zhu Y. B., Rodriguez-Paras C., Rhee J., & Mehta R. K. (2020). Methodological approaches and recommendations for functional near-infrared spectroscopy applications in HF/E research. Human Factors, 62(4), 613-642.