基于跨模态生理信号融合的飞行员认知状态检测
DOI:
CSTR:
作者:
作者单位:

1.中国民用航空飞行学院空中交通管理学院 广汉 618307; 2.中国民用航空飞行学院民航飞行技术与飞行安全工程技术重点实验室 广汉 618307

作者简介:

通讯作者:

中图分类号:

TN911.7

基金项目:

民航飞行技术与飞行安全重点实验室自主研究项目(FZ2022ZZ02)、四川省民航飞行技术与飞行安全工程技术研究中心项目(GY2024-64E)资助


Detection of pilots′ cognitive states based on cross-modal physiological signal fusion
Author:
Affiliation:

1.College of Air Traffic Management, Civil Aviation Flight University of China,Guanghan 618307, China; 2.Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Flight University of China, Guanghan 618307, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    飞行员认知状态的精准评估对保障飞行安全至关重要,然而现有方法在多模态生理信号融合上存在局限性。为此,本文提出一种基于双向跨模态注意力的双流深度学习网络。该模型采用并行双分支架构:脑电(EEG)分支通过相位锁定值(PLV)量化大脑功能连接,并采用SE模块增强的密集连接网络进行深度特征提取;心电(ECG)分支则提取心率变异性(HRV)及波形特征,经残差连接多层感知机处理以表征自主神经系统活动。在此基础上,通过创新设计的双向跨模态注意力模块,动态加权融合双路深度特征,实现对注意力集中、分散和惊吓/惊奇3种状态的精准分类。在NASA公开数据集上的实验表明,模型总体识别准确率达97.44%。消融与对比分析证实,该融合策略显著优于单模态分析和简单的特征拼接方法。研究表明,通过注意力机制深度融合EEG功能连接与ECG生理信息,可有效提升认知状态识别性能,该方法为开发客观、高效的飞行员状态监测系统提供了可靠的技术支持,对提升飞行安全具有重要的应用价值。

    Abstract:

    Accurate assessment of pilot cognitive states is critical for ensuring flight safety, yet existing methods exhibit limitations in fusing multimodal physiological signals. To address this, this paper proposes a dual-stream deep learning network based on bidirectional cross-modal attention. The model adopts a parallel dual-branch architecture: The electroencephalography (EEG) branch quantifies brain functional connectivity through phase locking value (PLV) features and employs a densely connected network enhanced with squeeze-and-excitation (SE) modules for deep feature extraction; the electrocardiogram (ECG) branch extracts heart rate variability (HRV) and waveform features, processed by a residual-connected multilayer perceptron to characterize autonomic nervous system activity. Building upon this, an innovatively designed bidirectional cross-modal attention module dynamically weights and fuses the dual-path deep features to achieve precise classification of three states—concentrated attention, distracted attention, and startle/surprise. Experimental results on the NASA public dataset demonstrate an overall recognition accuracy of 97.44%. Ablation and comparative analyses confirm that the fusion strategy significantly outperforms single-modality analysis and simple feature concatenation methods. The study reveals that deep integration of EEG functional connectivity and ECG physiological information via attention mechanisms effectively enhances cognitive state recognition performance. This approach provides reliable technical support for developing objective and efficient pilot state monitoring systems, holding significant application value for improving flight safety.

    参考文献
    相似文献
    引证文献
引用本文

张文泽,王在俊,蒋宇恒,杨睿哲.基于跨模态生理信号融合的飞行员认知状态检测[J].电子测量技术,2026,49(6):146-155

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2026-05-13
  • 出版日期:
文章二维码

重要通知公告

①《电子测量技术》期刊收款账户变更公告