Pilot Operation Action and Attitude Recognition Technology Based on Multi-View Visual Image Sequence

Abstract

Pilot maneuver and posture recognition serves as a key technical cornerstone for supporting quantitative evaluation of flight training and cockpit safety early warning. Its recognition accuracy directly determines the objectivity of training effect scoring, while its response speed is linked to the timeliness of cockpit risk early warning, making it of great significance for improving the quality of aviation training and flight safety standards. Multi-view vision technology, leveraging the unique advantages of noncontact measurement and accurate 3D scene reconstruction, has gradually replaced traditional solutions such as single-view observation and sensor-based contact monitoring, becoming the core technical support in this field. It consists of three core modules: image preprocessing, key point detection, and temporal modeling. Image preprocessing can eliminate light and clutter interference in the cockpit; key point detection is able to locate critical movement parts like hands, arms, and torso; and temporal modeling connects continuous operations to form an action logic chain. Relying on these modules, it can real-time correct non-standard operations such as push rod force deviation and lever position offset during flight training, and quickly identify dangerous postures like excessive forward leaning and delayed hand response in safety monitoring. Meanwhile, this technology needs to address severe occlusion caused by control levers and instrument panels in the cockpit, as well as the strict real-time requirement of millisecond-level response, while lightweight algorithm optimization and multi-modal data fusion provide a clear path for its subsequent development.