Research on Measuring Pilot Fatigue Data through Multimodal Data Fusion Based on SPO Mode

Abstract

Pilot fatigue detection is critical for aviation safety, as fatigue impairs pilots’ cognitive functions, reaction speeds, and decision-making abilities, posing severe threats to flights and potentially leading to heavy property damage or even casualties. This paper focuses on pilots under the single-pilot operation (SPO) mode—where individual pilots take on all flight tasks (from navigation to system monitoring), bearing greater physical and mental workload and facing higher fatigue risks than in traditional multi-pilot settings—and adopts a fatigue decision analysis method based on multimodal data fusion. It comprehensively collects four key types of data: electroencephalogram (EEG) signals reflecting real-time brain activity, electrocardiogram (ECG) signals related to autonomic nervous system changes, electromyogram (EMG) signals capturing muscle tension (e.g., around the eyes and jaw), and partial facial features (like eyelid closure duration or blink frequency) that visually indicate fatigue. By integrating and analyzing these multi-dimensional data, the paper reviews the latest research progress in pilot fatigue detection, identifies shortcomings of existing methods (such as single-modal approaches being easily disturbed by environmental factors), and explores future research directions, aiming to provide targeted technical support for ensuring SPO-mode flight safety.