CN112257190B - Method for monitoring abnormal people in ring flight simulation environment - Google Patents
Method for monitoring abnormal people in ring flight simulation environment Download PDFInfo
- Publication number
- CN112257190B CN112257190B CN202011301339.7A CN202011301339A CN112257190B CN 112257190 B CN112257190 B CN 112257190B CN 202011301339 A CN202011301339 A CN 202011301339A CN 112257190 B CN112257190 B CN 112257190B
- Authority
- CN
- China
- Prior art keywords
- flight simulation
- monitoring
- human
- abnormal
- simulation environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/15—Vehicle, aircraft or watercraft design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Abstract
The invention provides a method for monitoring abnormal state people in a ring flight simulation environment, which not only can well simulate the human in-ring flight simulation environment, but also can realize the monitoring and evaluation of mental load of tested personnel through abnormal state injection, and can be sequentially carried out according to the following five stages: step one, building a human-in-loop flight simulation environment and a monitoring platform; designing the injection type of the abnormal state and the alarm mode thereof; measuring the simulation index parameters of the abnormal people in the ring flight; constructing an abnormal human in-loop flight simulation monitoring model; and fifthly, evaluating the abnormal human in-loop flight simulation monitoring model. The invention completes the construction of the abnormal human in-loop flight simulation environment and the monitoring platform, and realizes the monitoring and evaluation of the abnormal human in-loop flight simulation environment by using the information fusion technology; the method is scientific, has good manufacturability and has wide popularization and application values.
Description
The technical field is as follows:
the invention provides a method for monitoring abnormal persons in a ring flight simulation environment, which is an abnormal person ring flight simulation environment and a monitoring platform thereof, wherein the abnormal person ring flight simulation environment is built based on professional simulated flight (namely X-Plane) flight simulation software, and the abnormal person ring flight simulation environment is characterized in that a specific abnormal state is injected, the eye movement data of a tested person and the parameter change of a physiological index are measured, and then the monitoring and evaluation of the mental load are realized by using an Error Back Propagation (BP) neural network, so that a basis can be better provided for the task allocation of a human-computer system, the task performance level of the human-computer system is improved, the safety and reliability level of the human-computer system is ensured, and the method belongs to the technical field of the safety design and analysis of the human-computer system.
Background art:
the aircraft system is a highly automated and complicated human-computer interaction system, and the fine operation process and the complicated electromechanical equipment of the aircraft system require that a pilot must be skilled in mastering the use method of relevant equipment of the aircraft and the display mode of relevant information to ensure the normal completion of a flight task. However, during actual flight, some sudden abnormal states including equipment failure, human error, environmental disturbance, and combinations thereof are inevitably encountered. The abnormal state is characterized by urgent task, short time and quick treatment, so that the pilot not only needs to monitor the abnormal state in time, but also needs to quickly respond correctly to overcome the abnormal state.
Because the cost for training pilots on real aircrafts is overlarge and is limited by flight fields and environmental factors, people adopt flight simulation equipment to train pilots for years, and the setting of the in-loop flight simulation environment of people is easier to realize. In terms of engineering, the common point of the existing flight simulation equipment is that the existing flight simulation equipment only focuses on the normal human-computer interaction process, and aims to research the human-computer interaction influence in the normal human-computer interaction state, and the considered conditions are not systematic and incomplete; technically, most flight simulation equipment is not added with an aircraft simulation fault injection module, so that the flight simulation equipment does not have flight task simulation conditions under the conditions of executing mechanical faults and environmental disturbance; in terms of application, most of the physiological data collected by the flight simulator are only used for physiological observation of personnel and are not combined with the cognitive characteristics of the personnel for human-computer interaction experiments. Based on the consideration, a human-in-loop flight simulation environment is constructed and configured, and meanwhile, the function of the flight simulation system is more perfect due to the introduction of the fault injection module, so that the human-in-loop flight simulation environment is more real. In addition, the physiological monitoring module is introduced, so that the physiological and psychological states of the tested person can be better monitored, and the relationship between the physiological state parameters and typical man-machine interaction faults can be explored.
The assessment or measurement of Mental Workload (MWL) is not only used throughout the design lifecycle to improve the design of systems and tasks, but also applied to the MWL assessment of existing running systems and programs and Workload assessment during the running of existing systems. The physiological indexes of the tested person are measured in real time, so that the mental load level of the tested person can be measured and evaluated, and the commonly used physiological indexes comprise eye movement data, electrocardiogram data, electroencephalogram data and the like. For eye movement data, pupil size, eyelid openness, and blink frequency can be used to assess mental load, where pupil size and eyelid openness both show a trend of increasing and decreasing with increasing load level, and the difference is significant at different load levels. For electrocardiographic data, common indicators include Heart Rate (HR), heart Rate Variability (HRV), wherein the HR indicator does not change significantly with the increase of mental load, and HRV is a relatively reliable indicator, and Standard deviation of the RR interval (SDNN) of all sinus heartbeat interval cycles of the HRV temporal indicator in a simulated flight environment significantly decreases with the increase of mental load. For electroencephalogram data, when a specific part of the brain feels some stimulation, the Potential of the part changes, and the change is called Event Related Potential (ERP). ERP has many components, and more commonly N1 (N1 refers to a Negative component recorded about 100ms after stimulation), negative change of relevance (CNV appears about 1000ms after stimulation), and P300 (P300 refers to the positive deflection of the electroencephalogram voltage that appears about 300ms after stimulation presentation). The traditional mental load assessment is only used for assessing the mental load level in the normal human-computer interaction state, and the mental load assessment for researching the abnormal state human in the circular flight simulation environment is relatively less. In addition, the traditional mental load assessment collects single-channel physiological data of a tested person and only uses the single-channel physiological data for observation, and the multi-channel index data is not well fused to monitor and assess the mental load level of the tested person in an abnormal state.
Therefore, the invention provides a method for monitoring abnormal people in a ring flight simulation environment.
The invention content is as follows:
(1) The purpose is as follows:
the invention provides a method for monitoring abnormal persons in a ring flight simulation environment, which is an abnormal person ring flight simulation environment and a monitoring platform thereof which are built based on X-Plane flight simulation software. The purpose of monitoring is to evaluate the tested personnel in real time, so that the task allocation of the human-computer system is better performed, and the overall safety and reliability of the human-computer system are improved.
(2) The technical scheme is as follows:
the invention relates to a method for monitoring abnormal state people in a ring flight simulation environment, which not only can well simulate the human in-ring flight simulation environment, but also can realize the monitoring and evaluation of mental load of tested personnel through abnormal state injection, and can be sequentially carried out according to the following five stages:
step one, building a human-in-loop flight simulation environment and a monitoring platform;
designing the injection type of the abnormal state and the alarm mode thereof;
measuring the simulation index parameters of the abnormal people in the ring flight;
constructing an abnormal human in-loop flight simulation monitoring model;
and fifthly, evaluating the abnormal human in-loop flight simulation monitoring model.
Wherein, stage one "build people and fly simulated environment and monitoring platform in the ring", its purpose is: on one hand, a corresponding software and hardware platform basis is provided for the subsequent development of the in-loop flight simulation environment, and on the other hand, a virtual and relatively real flight simulation environment is realized as far as possible; the specific implementation steps can be further subdivided as follows:
step 1: construction of software platform for simulating environment of man-in-the-loop flight
Selecting X-Plane as flight simulation software, and rearranging a dial plate of a flight simulation system by using a Panel maker (namely Panel Builder) to complete the construction of a software platform for simulating the environment of the man-in-the-loop flight;
step 2: construction of hardware platform for human-in-the-loop flight simulation environment
Adopting a surrounding layout method, using three curved surface display screens as simple layout of the aircraft cockpit, and simultaneously matching hardware facilities such as a handle, a keyboard, a mouse and the like to finish the construction of a hardware platform of the human in-the-loop flight simulation environment;
and step 3: construction of human-in-loop flight simulation environment monitoring platform
And the construction of the in-loop flight simulation monitoring platform is completed by combining the functional design requirements of monitoring and evaluation and using a measurement and analysis module of corresponding physiological indexes and eye movement data in a matched manner.
Wherein, the stage two mentioned "design abnormal state injection type and alarm mode" aims at: realizing fault injection of abnormal states and designing a corresponding alarm mode;
the types of the abnormal states specifically comprise three aspects of human errors, functional faults and environmental disturbance, flight mission faults are divided into six fault categories which are respectively external faults, aircraft faults, instrument faults, engine faults, wing faults and navigation aid equipment faults, and each fault category comprises multiple conditions; the types of fault injection include both system level and device meter level; either randomly or at fixed points in time or events as required.
Wherein, the 'measuring abnormal state human in-loop flight simulation index parameters' in the third stage aims to: by designing an abnormal human in-loop flight simulation environment and an experimental scheme for monitoring the abnormal human in-loop flight simulation environment, relevant index parameters for measuring tested personnel are collected, data support is provided for the subsequent construction of an abnormal human in-loop flight simulation monitoring model, and the specific implementation steps can be subdivided as follows:
step 1: designing an experimental scheme flow;
step 2: setting the difficulty of an experiment task;
and step 3: generating an experiment task event;
and 4, step 4: and measuring experimental index parameters.
Wherein, the step four is to construct an abnormal human in-loop flight simulation monitoring model, and the purpose is to: on one hand, the analysis processing of experimental monitoring data is completed, and on the other hand, the model construction of multi-channel monitoring data is realized; the evaluation index and monitoring framework for developing mental load is shown in fig. 4, and the specific implementation steps can be subdivided as follows:
step 1: developing significance test, and constructing a single-channel multiple linear regression model;
step 2: using principal component analysis to realize multi-channel experimental data dimension reduction analysis;
and step 3: constructing a multi-channel data fusion multi-linear regression model by adopting a multi-linear regression method;
and 4, step 4: and (4) completing the construction of a multi-channel data fusion model by adopting a BP neural network.
Wherein, the "abnormal human in-loop flight simulation monitoring model for evaluation" in stage five aims to: on one hand, the real-time monitoring of the abnormal state person in the ring flight simulation environment is carried out, on the other hand, the rationality of a monitoring model of the abnormal state person in the ring flight simulation environment is evaluated, and the specific implementation steps can be subdivided as follows:
step 1: real-time monitoring of a BP neural network monitoring model;
through multiple times of simulation, the prediction results of the training set and the test set are respectively obtained, and the prediction capability of the BP neural network monitoring model is tested;
step 2: carrying out comparative evaluation on the BP neural network monitoring model;
through comparative analysis, from the three aspects of correct rate, mean square error and correlation coefficient, the multichannel index data parameters are synthesized to obtain a monitoring model with optimal performance.
(3) Efficacy and advantages of the invention
The invention provides a method for monitoring abnormal people in a ring flight simulation environment, which completes the construction of the abnormal people in the ring flight simulation environment and a monitoring platform, and realizes the monitoring and evaluation of the abnormal people in the ring flight simulation environment by using an information fusion technology. The efficacy is mainly embodied in the following aspects:
1. a fault injection module is introduced, so that the function of the flight simulation system is more complete, and the environment simulated by the flight of people around is more real;
2. by adopting a principal component analysis method, the difficulty that the BP neural network cannot be well played due to relatively few experimenters and experimental times because of various kinds of index data monitored by experiments is avoided;
3. by using the BP neural network model, parameters such as eye movement data, physiological indexes and the like are comprehensively considered, fusion analysis of the data is realized, and construction and evaluation of the abnormal human in-loop flight simulation environment monitoring model are completed;
4. the method of the invention is scientific, has good manufacturability and has wide popularization and application value.
Description of the drawings:
FIG. 1 is a flow chart of a method for monitoring an abnormal state human under an airplane loop flight simulation environment.
FIG. 2 is a flow chart of an abnormal state man-machine ring flight simulation environment and a monitoring platform thereof in the invention.
FIG. 3 is a diagram of a situational awareness model in an abnormal state in the present invention.
FIG. 4 is a flowchart of an experiment for assessing mental load in an abnormal state in the present invention.
Fig. 5 is a block diagram of mental load evaluation index and monitoring in the present invention.
FIG. 6 is a flowchart of training BP neural network model in the present invention.
FIG. 7 is a schematic diagram of a prediction result of a BP neural network training set model in the present invention.
FIG. 8 is a diagram of the prediction results of the BP neural network test set model in the present invention.
The specific implementation mode is as follows:
the invention relates to a method for monitoring abnormal state people in a ring flight simulation environment, which can not only well simulate the human in the ring flight simulation environment, but also realize the monitoring and evaluation of mental load of a tested person through abnormal state injection, and the specific implementation steps are as follows:
stage one: a human-in-the-loop flight simulation environment and a monitoring platform are built, the platform layout is shown in figure 2, and the specific implementation steps can be subdivided as follows:
step 1: construction of software platform for simulating environment of man-in-the-loop flight
The X-Plane is selected as the kernel of flight simulation software, and the X-Plane mainly plays two functions in the system, namely flight simulation calculation and visual display, and the visual display is the visual display of the X-Plane. In addition, the dial plate of the flight simulation system is rearranged by utilizing the Plane Builder, so that the more real reflection of the aircraft cockpit is realized, wherein: 1) The most important is the instrument indicating attitude, which is generally placed in a central location; 2) The instrument indicating airspeed must be placed to the left of the instrument indicating attitude; 3) The height indicating gauge must be placed to the right of the attitude indicating gauge; 4) The instrument indicating the heading must be placed under the instrument indicating the attitude; 5) For other instruments, the regulation is loose, and the instruments can be placed according to the experimental requirements.
Step 2: construction of hardware platform for human-in-loop flight simulation environment
The method of surrounding layout is adopted, three curved surface display screens are used as the simple layout of the aircraft cockpit, and meanwhile, hardware facilities such as a handle, a keyboard, a mouse and the like are matched to finish the construction of a hardware platform in the ring flight simulation environment. In addition, each display screen can display specific parameter information, wherein the left display screen is used for displaying an airplane flight map and airplane alarm state information, the middle display screen is used for displaying a flight view and airplane basic flight parameter information, and the right display screen is used for displaying parameter information such as an engine system and a fuel system, so that a testee can acquire a parameter area related to flight through the transfer of a fixation point.
And step 3: construction of human-in-loop flight simulation environment monitoring platform
In combination with the functional design requirements of monitoring and evaluation, the physiological indexes and the eye movement data need to be measured, wherein the main measurement equipment of the physiological indexes is a physiological measurement module of the Yangtze Send company, and the measured main parameters are electrocardio (EEG), electromyogram (EMG) and photoplethysmography (PPG), so that the muscle activity condition, the heart beating condition and the subcutaneous tube flow condition of the tested person during the experiment can be measured; the corresponding analysis software is an ErgoLAB man-machine loop synchronous experiment platform, and can be used for objectively observing and analyzing the physiological data transmitted by the physiological measurement equipment. The main measuring equipment of the eye movement data is a Tobii Glasses Pro2 type eye movement instrument of Tobii company, and can measure the fixation track of eyeball sight, sight coordinates and blink frequency; the corresponding analysis software is Tobii Lab Viewer and Tobii Pro Lab, the former is used for recording the relevant measurement data of the eye tracker, and the latter can analyze the data recorded by the former.
And a second stage: designing the injection type of the abnormal state and the alarm mode thereof, wherein the specific implementation steps are as follows:
the abnormal state specifically comprises three aspects of human error, functional failure and environmental disturbance. The situational awareness modeling under abnormal conditions is shown in fig. 3, in which human errors can be analyzed by combining Cognitive Reliability and Error Analysis Method (CREAM); functional faults are mainly analyzed from the perspective of a human-computer interaction process which needs to be provided in the functional unit, and mainly comprise faults, false alarms, missed alarms and the like; the environmental conditions are relatively fixed, and environmental influences, such as flight tasks, mainly have weather influences on the flight tasks, including rain, snow, fog, visibility, cloud cover thickness and the like, can be classified according to different task scenes.
The functional state of the system can be divided into failure, performance reduction and normal operation, and the alarm signal of the equipment can display the working state of the system as normal operation and alarm. The faults are divided into six fault categories, namely, an external cause fault, an aircraft fault, an instrument fault, an engine fault, a wing fault and a navigation aid equipment fault, and each fault category comprises multiple conditions (as shown in table 1). The types of fault injection include both system level and device meter level; injection can be either random or on demand at a fixed point in time or event.
Table 1 fault injection major classification table
And a third stage: the method comprises the following specific implementation steps of measuring abnormal human in-loop flight simulation index parameters:
step 1: design of experimental protocol flow
The flow chart for the set of experiments is shown in FIG. 4. First, the experimental staff recruited in this group were all in-school students with certain aviation knowledge background and english hearing level, the dominant hand was the right hand, no achromatopsia, and the corrected eyesight was all above 1.0. Second, the same pre-experiment training and practice was performed on all subjects prior to the start of the experiment, so that each subject was at the same level of familiarity with the experiment prior to the experiment. Thirdly, testing the experimental tasks of each tested person to ensure that each tested person is familiar with mastering the group of experimental tasks and taking the test result as the reference level of the mental load level of the tested person. And fourthly, before the formal experiment is started, each tested person wears and calibrates the equipment to ensure that the measured data in the experiment process is real and effective. And fifthly, randomly distributed experiment tasks are adopted for each tested person, so that the interference influence caused by the sequence of the experiment tasks is avoided. And sixthly, carrying out data analysis on the acquired experimental data, and completing the construction and verification of the mental load assessment model.
Step 2: setting the difficulty of the experiment task
The aircraft flight task mainly relates to four subtasks, namely a control task, a system monitoring task, a voice communication task and a resource management task. Considering that the complexity of each subtask is different, and simultaneously, in order to better objectively quantify the mental load in the experimental task process, the shannon information entropy is referred to herein to convert the stimulation information content in the task execution process into the bit rate size, and the bit rate size is used as an objective reference value (unit: bit value per second) for evaluating the mental load level.
For each subtask, the set of experiments assumed equal likelihood for all decision alternatives, and the stimulation information could be represented as H s (i)＝log 2 [j(i)]Meanwhile, the information bit rate calculation formula of each subtask is as follows:
wherein: i represents the subtask type, j represents the total number of types that the subtask generates stimulation, H s (i) Represents the entropy of the information of the ith subtask, and δ T (i) represents the time interval period of the ith subtask (the time interval in which the same stimulation information starts to occur twice).
For the manipulation task, the information bit rate is determined by the effective width (W) of the cursor and the moving speed of the cursor during the operation, and the calculation formula of the bit rate is as follows:
in the group of experiments, the operation task is used as a main task, and the other subtasks are used as secondary tasks, so that five difficulty-level experiment tasks are compiled based on the operation task and the secondary tasks.
TABLE 2 Experimental task difficulty ratings
And 3, step 3: generating experimental task events
Uniformly distributed pseudo-random integers are generated herein by means of a Randi function in MATLAB, where the Randi function is called in the format: r = randi (imax, [ m, n ]), indicating that a pseudo-random number matrix of m × n in the range of [1, imax ] is returned. On one hand, the distribution condition and the type of each abnormal state are further determined by defining the task number of different subtasks when the abnormal state occurs in advance; on the other hand, the occurrence time of each abnormal state is further determined through the difficulty level division in different task events, and the conversion formula of the occurrence time is as follows:
t i ＝n i +Δt j * (i-1) or t i ＝n i +Δt j *i (3)
Wherein: n is i Represents the ith pseudo-random number generated by Randi; Δ t j Time intervals representing exceptional events at the j difficulty level; t is t i Representing the occurrence time of the ith abnormal event, and the calculation formula is determined by the task type and the abnormal times.
And 4, step 4: measuring experimental index parameters
At the beginning of the formal experiment, the relevant indexes of the tested person need to be measured in real time, so as to monitor and evaluate the mental load of the tested person, and the evaluation indexes and the measurement method for evaluating the mental load are shown in fig. 5.
And a fourth stage: the method comprises the following steps of constructing an abnormal human in-loop flight simulation monitoring model, and subdividing the specific implementation steps as follows:
step 1: developing significance test and constructing single-channel multiple linear regression model
For evaluation index data obtained by experimental data, initial calculation and analysis are firstly carried out on original data, and then further significance test is carried out on the collected experimental data. Only by the significance test can the experimental data prove useful. And finally, sequentially constructing a single-channel Multiple Linear Regression (MLR) model by adopting a gradual entry analysis method, wherein the Regression prediction results of the models are shown.
TABLE 3 MLR model summary of Single channel data types
| Performance level | Performance data MLR | Eye movement data MLR | Electrocardio data MLR | NASA-TLX data MLR |
| Accuracy rate | 26.47％ | 16.18％ | 22.06％ | 29.41％ |
| Mean square error | 0.15987 | 0.16751 | 0.10508 | 0.13959 |
| Coefficient of correlation | 0.200 | 0.122 | 0.311 | 0.390 |
Step 2: using principal component analysis, multi-channel experimental data dimension reduction analysis is achieved
Because the variety of the index data obtained by experiment monitoring is various, and the number of experimenters and the number of experiments are relatively less compared with the number of the monitored index data, if the collected index data are directly subjected to fusion analysis, the number of experiments is often insufficient, so that the advantages of the BP neural network cannot be well exerted. Therefore, the monitored index data is subjected to Principal Component Analysis (PCA), so that a large number of original index variables can be replaced by a small number of comprehensive variables, and the original data variables can be more completely reserved.
KMO sampling fitness test and Bartlett spherical test are sequentially carried out on each channel data to obtain a principal component coefficient matrix, and finally, the 14 performance data, 13 eye movement data, 13 electrocardio data and 7 subjective data are reduced to 4, 5, 3 and 2 total 14 principal component comprehensive variables, so that the difficulty of problem analysis simplification is realized.
And step 3: adopting MLR method to complete the construction of multi-channel data fusion MLR model
In the process of constructing the multi-channel data MLR model, input vectors of the model are 14 comprehensive variables obtained through PCA extraction. And then constructing a nonstandard MLR equation between the MWL level and the principal component extracted by PCA by adopting a stepwise regression method, wherein the MLR standard error is 0.13657, and the correlation coefficient is 0.41627.
And 4, step 4: adopting BP neural network to complete the construction of multi-channel data fusion model
The training process for constructing the BP neural network model is shown in fig. 6, and the input vectors of the training process are 14 comprehensive variables obtained by PCA extraction, that is, the number of nodes of the input layer is 14. Since the expected output result of the model is the mental load assessment level, i.e., the number of output nodes is 1. In addition, 80% of samples are randomly selected as training samples and 20% are selected as testing samples, and the number of hidden layer nodes is finally determined to be 10 through continuous debugging. Finally, the relevant parameters of the constructed BP neural network model are shown in table 4.
TABLE 4 BP neural network parameter design
| Number of | Programming language | Size and breadth |
| 1 | |
14 |
| 2 | |
10 |
| 3 | net.nOut | 1 |
| 4 | net.w | 0.5 |
| 5 | net.trainParam.goal | 0.001 |
| 6 | net.trainParam.epochs | 100000 |
| 7 | net.trainParam.lr | 0.001 |
| 8 | net.trainParam.mc | 0.1 |
| 9 | net.trainParam.show | 1000 |
| 10 | net.trainParam.time | inf |
And a fifth stage: the method for evaluating the abnormal human in-loop flight simulation monitoring model comprises the following specific implementation steps:
step 1: real-time monitoring of BP neural network monitoring model
After the above analysis, the set parameters are input into the BP neural network (the relevant parameter settings can be seen in table 4). Through multiple simulation simulations of MATLAB, prediction results of a training set and a test set are respectively obtained, as shown in FIGS. 7 and 8, wherein the mean square error of a test set model is 0.01855, the correlation coefficient is 0.71374, and the ratio of correct MWL prediction is 76.92%, which means that the trained BP neural network can be better used as a prediction model of MWL level to a certain extent. In addition, in the trained optimal BP neural network model, the statistics of the prediction results for the MWL training set and the test set are shown in tables 5 and 6.
TABLE 5 statistics of MWL prediction results for BP neural network training set
| ID | MWL | Predict | Right and wrong | ID | MWL | Predict | Right and wrong |
| 1 | 0.4628 | 0.4645 | T | 29 | 0.5766 | 0.5780 | |
| 2 | 0.5766 | 0.5788 | |
30 | 0.4628 | 0.4645 | T |
| 3 | 0.5766 | 0.5772 | T | 31 | 0.3970 | 0.3981 | T |
| 4 | 0.4628 | 0.4629 | T | 32 | 0.4628 | 0.4666 | T |
| 5 | 0.7226 | 0.7193 | T | 33 | 0.2000 | 0.2029 | |
| 6 | 0.3970 | 0.3987 | T | 34 | 0.4628 | 0.4663 | T |
| 7 | 0.5766 | 0.5744 | T | 35 | 0.2000 | 0.1987 | |
| 8 | 0.3970 | 0.3991 | T | 36 | 0.3970 | 0.3988 | T |
| 9 | 0.5766 | 0.5713 | T | 37 | 0.2000 | 0.1972 | |
| 10 | 0.2000 | 0.2009 | T | 38 | 0.2000 | 0.2019 | T |
| 11 | 0.2000 | 0.1998 | T | 39 | 0.2000 | 0.2004 | |
| 12 | 0.4628 | 0.4641 | |
40 | 0.3970 | 0.3981 | T |
| 13 | 0.3970 | 0.3999 | T | 41 | 0.2000 | 0.2005 | |
| 14 | 0.2000 | 0.2189 | T | 42 | 0.7226 | 0.7229 | T |
| 15 | 0.7226 | 0.7253 | T | 43 | 0.3970 | 0.3934 | T |
| 16 | 0.5766 | 0.5780 | T | 44 | 0.7226 | 0.7253 | T |
| 17 | 0.7226 | 0.7247 | T | 45 | 0.4628 | 0.4600 | T |
| 18 | 0.4628 | 0.4691 | T | 46 | 0.7226 | 0.7234 | T |
| 19 | 0.2000 | 0.2089 | T | 47 | 0.3970 | 0.3871 | |
| 20 | 0.7226 | 0.7228 | T | 48 | 0.7226 | 0.7226 | T |
| 21 | 0.5766 | 0.5767 | T | 49 | 0.4628 | 0.4713 | T |
| 22 | 0.7226 | 0.7225 | |
50 | 0.3970 | 0.3977 | T |
| 23 | 0.5766 | 0.5840 | T | 51 | 0.2000 | 0.1956 | T |
| 24 | 0.5766 | 0.5746 | T | 52 | 0.3970 | 0.3982 | T |
| 25 | 0.5766 | 0.5776 | T | 53 | 0.3970 | 0.3903 | T |
| 26 | 0.3970 | 0.3984 | T | 54 | 0.2000 | 0.2049 | T |
| 27 | 0.4628 | 0.4599 | T | 55 | 0.4628 | 0.4661 | T |
| 28 | 0.5766 | 0.5795 | T |
TABLE 6 BP neural network test set MWL prediction result statistics
Step 2: contrast evaluation of BP neural network monitoring model
Accuracy, mean Square Error (MSE) and correlation coefficient (R) evaluated from MWL level, respectively 2 ) Analysis shows that the accuracy, MSE and R of the monitoring model obtained by the BP neural network are correct 2 The model is superior to an MLR model, namely, the constructed BP neural network model can be used as a mental load monitoring model of abnormal people in a circular flight simulation environment to a certain extent.
TABLE 7 comparative analysis table for each monitoring model based on single/multi-channel data type
Claims (4)
1. A method for monitoring abnormal people in a ring flight simulation environment is characterized by comprising the following steps: the method is sequentially developed according to the following five stages:
step one, building a human-in-loop flight simulation environment and a monitoring platform;
designing the injection type of the abnormal state and the alarm mode thereof;
measuring the simulation index parameters of the abnormal people in the ring flight;
constructing an abnormal human in-loop flight simulation monitoring model;
evaluating an abnormal human in-loop flight simulation monitoring model;
the method for establishing the abnormal human in-loop flight simulation monitoring model comprises the following steps: on one hand, the analysis processing of experimental monitoring data is completed, and on the other hand, the model construction of multi-channel monitoring data is realized; the specific implementation steps are further subdivided as follows:
step 1: developing significance test, and constructing a single-channel multiple linear regression model;
step 2: using principal component analysis to realize multi-channel experimental data dimension reduction analysis;
and step 3: constructing a multi-channel data fusion multi-linear regression model by adopting a multi-linear regression method;
and 4, step 4: adopting a BP neural network to complete the construction of a multi-channel data fusion model;
the 'evaluation abnormal state human in-loop flight simulation monitoring model' refers to the following steps: on one hand, the real-time monitoring of the abnormal state person in the ring flight simulation environment is carried out, on the other hand, the rationality of a monitoring model of the abnormal state person in the ring flight simulation environment is evaluated, and the specific implementation steps are subdivided as follows:
step 5.1: real-time monitoring of a BP neural network monitoring model;
through multiple times of simulation, the prediction results of the training set and the test set are respectively obtained, and the prediction capability of the BP neural network monitoring model is tested;
step 5.2: carrying out comparative evaluation on the BP neural network monitoring model;
through comparative analysis, from the three aspects of correct rate, mean square error and correlation coefficient, the multichannel index data parameters are synthesized to obtain a monitoring model with optimal performance.
2. The method for monitoring the abnormal human in the loop flight simulation environment according to the claim 1, wherein the method comprises the following steps:
the step one of building a human-in-the-loop flight simulation environment and a monitoring platform means that: on one hand, a corresponding software and hardware platform basis is provided for the subsequent development of the in-loop flight simulation environment, and on the other hand, a virtual and relatively real flight simulation environment is realized; the specific implementation steps are further subdivided as follows:
step 1.1: construction of software platform for simulating environment of man-in-the-loop flight
Selecting X-Plane as flight simulation software, and rearranging a dial plate of a flight simulation system by using a panel maker to complete the construction of a software platform for the man-in-the-loop flight simulation environment;
step 1.2: construction of hardware platform for human-in-loop flight simulation environment
Adopting a surrounding layout method, using three curved surface display screens as simple layout of the aircraft cockpit, and simultaneously completing the construction of a hardware platform in the ring flight simulation environment by matching with hardware facilities such as a handle, a keyboard and a mouse;
step 1.3: construction of human-in-loop flight simulation environment monitoring platform
And the construction of the in-loop flight simulation monitoring platform is completed by combining the functional design requirements of monitoring and evaluation and using a measurement and analysis module of corresponding physiological indexes and eye movement data in a matched manner.
3. The method for monitoring the abnormal human in the loop flight simulation environment according to the claim 1, wherein the method comprises the following steps:
the step two of designing the injection type of the abnormal state and the alarm mode thereof refers to: realizing fault injection of abnormal states and designing a corresponding alarm mode;
the types of the abnormal states specifically comprise three aspects of human errors, functional faults and environmental disturbances, flight mission faults are divided into six fault categories, namely external faults, aircraft faults, instrument faults, engine faults, wing faults and navigation aid equipment faults, and each fault category comprises multiple conditions; the types of fault injection include both system level and device meter level; injection can be either random or on demand at fixed points in time and events.
4. The method for monitoring the abnormal human in the loop flight simulation environment according to the claim 1, wherein the method comprises the following steps:
the step three of measuring abnormal human in-loop flight simulation index parameters refers to: designing an abnormal human in-loop flight simulation environment and an experimental scheme for monitoring the abnormal human in-loop flight simulation environment, collecting and measuring related index parameters of a tested person, and providing data support for subsequently constructing an abnormal human in-loop flight simulation monitoring model, wherein the specific implementation steps are subdivided as follows:
step 3.1: designing an experimental scheme flow;
step 3.2: setting the difficulty of an experiment task;
step 3.3: generating an experiment task event;
step 3.4: and measuring experimental index parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011301339.7A CN112257190B (en) | 2020-11-19 | 2020-11-19 | Method for monitoring abnormal people in ring flight simulation environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011301339.7A CN112257190B (en) | 2020-11-19 | 2020-11-19 | Method for monitoring abnormal people in ring flight simulation environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112257190A CN112257190A (en) | 2021-01-22 |
| CN112257190B true CN112257190B (en) | 2023-01-20 |
Family
ID=74266318
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011301339.7A Active CN112257190B (en) | 2020-11-19 | 2020-11-19 | Method for monitoring abnormal people in ring flight simulation environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257190B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117111639B (en) * | 2023-10-19 | 2024-01-26 | 浙江容祺科技有限公司 | Unmanned aerial vehicle flight optimal route optimizing method in complex environment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103784119A (en) * | 2014-01-24 | 2014-05-14 | 北京航空航天大学 | Pilot mental load measurement system based on flight simulator |
| WO2020173020A1 (en) * | 2019-02-26 | 2020-09-03 | 北京控制工程研究所 | Flight control intelligent data analysis and decision support system for spacecraft rendezvous and docking |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2510556C (en) * | 2004-07-01 | 2013-02-19 | Cae Inc. | Method and system for simulation-based troubleshooting and fault verification in operator-controlled complex systems |
-
2020
- 2020-11-19 CN CN202011301339.7A patent/CN112257190B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103784119A (en) * | 2014-01-24 | 2014-05-14 | 北京航空航天大学 | Pilot mental load measurement system based on flight simulator |
| WO2020173020A1 (en) * | 2019-02-26 | 2020-09-03 | 北京控制工程研究所 | Flight control intelligent data analysis and decision support system for spacecraft rendezvous and docking |
Non-Patent Citations (2)
| Title |
|---|
| 基于通用型模拟飞行系统的脑力状态评定台研制;白永超 等;《航天医学与医学工程》;20180228;第31卷(第1期);正文第1-3部分 * |
| 飞机驾驶舱人-机-环实验平台研究;郭云东 等;《航空计算技术》;20170125(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112257190A (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Morris et al. | Electrooculographic and performance indices of fatigue during simulated flight | |
| Bernhardt et al. | The effects of dynamic workload and experience on commercially available EEG cognitive state metrics in a high-fidelity air traffic control environment | |
| Brookings et al. | Psychophysiological responses to changes in workload during simulated air traffic control | |
| Gartner et al. | Pilot workload and fatigue: A critical survey of concepts and assessment techniques | |
| CN115191018A (en) | Evaluation of a person or system by measuring physiological data | |
| CN112257190B (en) | Method for monitoring abnormal people in ring flight simulation environment | |
| Jiang et al. | Correlation evaluation of pilots’ situation awareness in bridge simulations via eye-tracking technology | |
| Lampton et al. | Human performance measurement in virtual environments | |
| Zhang et al. | Modeling pilot mental workload using information theory | |
| Reinerman-Jones et al. | Developing methodology for experimentation using a nuclear power plant simulator | |
| CN117547270A (en) | Pilot cognitive load feedback system with multi-source data fusion | |
| Galant et al. | Analysis of pilot’s cognitive overload changes during the flight | |
| Jiang et al. | Mental workload artificial intelligence assessment of pilots’ EEG based on multi-dimensional data fusion and LSTM with attention mechanism model | |
| CN115462755A (en) | Pilot cognitive ability assessment method, system and storage medium | |
| Huang et al. | Modeling and analysis of fatigue detection with multi-channel data fusion | |
| Berlik et al. | Overview of the workload assessment methods in the aspect of improvement of the operator-technical subsystem relations on the example of a pilot | |
| Koglbauer et al. | Applications of Cardiac and Electrodermal Activity Assessment in Aviation | |
| Sharma et al. | Cognitive ability criterion for expertise in air traffic control task | |
| Mead et al. | Investigating facial electromyography as an indicator of cognitive workload | |
| Wilson | Progress in the psychophysiological assessment of workload | |
| Chen et al. | A Sensitivity Analysis Method of Physiological Signals for Assessing Pilot Workload | |
| Mercado | Assessing the effectiveness of workload measures in the nuclear domain | |
| Davies et al. | Emotional investment in naturalistic data collection | |
| Shao et al. | Evaluation of Fatigue Induced During the Flight Tasks | |
| Vogl et al. | Task Development to Validate Psychophysiological Monitoring of Cognitive Workload for Real-Time Operator State Monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |