CN108241431B - Task adjusting method and device - Google Patents

Task adjusting method and device Download PDF

Info

Publication number
CN108241431B
CN108241431B CN201611216913.2A CN201611216913A CN108241431B CN 108241431 B CN108241431 B CN 108241431B CN 201611216913 A CN201611216913 A CN 201611216913A CN 108241431 B CN108241431 B CN 108241431B
Authority
CN
China
Prior art keywords
data
resting
real
electroencephalogram
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611216913.2A
Other languages
Chinese (zh)
Other versions
CN108241431A (en
Inventor
于路
许利群
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Communications Ltd Research Institute
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Communications Ltd Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Communications Ltd Research Institute filed Critical China Mobile Communications Group Co Ltd
Priority to CN201611216913.2A priority Critical patent/CN108241431B/en
Publication of CN108241431A publication Critical patent/CN108241431A/en
Application granted granted Critical
Publication of CN108241431B publication Critical patent/CN108241431B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G06Q10/06311—Scheduling, planning or task assignment for a person or group

Abstract

The invention provides a task adjusting method and device, relates to a data processing technology, and is used for ensuring stable operation of a man-machine system. The task adjusting method comprises the following steps: acquiring electrophysiological data of a current operator; respectively inputting the electrophysiological data as a preset functional state prediction model, a psychological state recognition model and a fatigue state discrimination model, and respectively operating the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model to obtain a predicted functional state, a psychological state and a fatigue state of the current operator; acquiring a corresponding task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator; and adjusting the task of the current operator according to the task adjusting strategy. The invention is mainly used in a human-computer interaction system.

Description

Task adjusting method and device
Technical Field
The present invention relates to data processing technologies, and in particular, to a task adjusting method and device.
Background
With the development of automation technology, man-machine systems are in many areas: military (such as fighter plane driving), industry (such as nuclear power station operation), civil aviation (such as air traffic control and security inspection), medical treatment (such as laparoscopic surgery), video monitoring, game players, driving (vehicle-mounted information system) and the like are widely applied. In complex man-machine systems with high security requirements, such as: in nuclear power stations, air traffic control, fighter aircraft driving and the like, operators complete certain tasks together with an automatic system under uncertain environment, dynamic tasks, complex decisions and the like. If the operator cannot maintain a good state in the human-computer interaction system and lacks or does not have the emergency handling capability for an emergency, serious consequences such as a reactor accident, an air crash, a knock-down and the like may be caused. Therefore, in order to ensure the safe and normal operation of the system, the operator should have good handling capacity for the assigned tasks at all times, and a good human-computer interaction state is ensured.
However, human operators are different from machines, and it is difficult to perform tasks stably for a long time, and as the working time increases and the operation tasks become complicated, states of fatigue, stress, anxiety, and the like occur, and it is not guaranteed to maintain a good state at any time. Therefore, it is necessary to grasp the physiological and psychological states of the operator at any time and properly adjust the human-computer interaction system to ensure the safe and normal operation of the system.
The functional state of the operator is a dynamic capability that reflects the ability of the operator to perform the currently assigned task, and is the result and performance of the human body's numerous physiological and psychological activities regulated by the brain in order for the human body to meet or adapt to external needs. The functional state of the operator is the comprehensive assessment of mental load, workload, physical condition, alertness level and the like of the operator, and not only can reflect the working state of the operator of a complex man-machine system, but also can reflect the state of working crowd in daily offices.
The prior art mainly focuses on analyzing the current functional state of a human body by collecting physiological signals of electroencephalogram, electrocardio and the like of the human body, but does not relate to a scheme for applying an analysis result of the functional state to a human-computer interaction system so as to ensure the stable operation of the human-computer interaction system.
Disclosure of Invention
In view of this, the present invention provides a task adjusting method and device to ensure stable operation of a human-machine system.
In order to solve the above technical problem, the present invention provides a task adjusting method, including:
acquiring electrophysiological data of a current operator;
respectively inputting the electrophysiological data as a preset functional state prediction model, a psychological state recognition model and a fatigue state discrimination model, and respectively operating the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model to obtain a predicted functional state, a psychological state and a fatigue state of the current operator;
acquiring a corresponding task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator;
and adjusting the task of the current operator according to the task adjusting strategy.
Wherein, prior to the step of obtaining electrophysiological data of the current operator, the method further comprises:
and respectively training the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model.
Wherein the step of training the functional state prediction model comprises:
respectively acquiring real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state in a preset testing environment;
acquiring an input variable of a BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
and taking the input variable as the input of the BP neural network, training the BP neural network, and training the function state prediction model.
The step of obtaining the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data and the resting electrocardiograph data comprises the following steps:
respectively calculating a relative value of electroencephalogram alpha wave band power, a relative value of electroencephalogram beta wave band power, a relative value of electroencephalogram theta wave band power, a relative value of electroencephalogram wave band power, a relative value of heart rate, a relative value of RR interval high-frequency power, a relative value of mean square root of adjacent RR interval differences and a relative value of RR interval standard deviation according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
wherein the content of the first and second substances,
Figure GDA0001263005350000031
Figure GDA0001263005350000032
Figure GDA0001263005350000033
wherein alpha _ ratio represents the relative value of the power of the alpha wave band of the brain electricity, and alphaexRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
Wherein the step of training the mental state recognition model comprises:
respectively acquiring real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state in a preset testing environment;
training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data and a BP neural network;
and training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
The step of training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data and a BP neural network comprises the following steps:
calculating an input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model;
wherein the input variables include: data sample entropy, standard deviation of electrocardio P wave length and electrocardio R wave length mean value.
The step of training a catatonic degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data and a BP neural network comprises the following steps:
calculating an input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model;
wherein the input variables include: heart rate variability relative value, standard deviation of electrocardio P wave length, relative value of mean square root of adjacent RR interval differences, relative value of percentage of number of interval differences with adjacent interval difference exceeding 50ms in total interval difference number;
wherein, the
Figure GDA0001263005350000041
Figure GDA0001263005350000042
Figure GDA0001263005350000043
HRV _ ratio represents the relative value of the heart rate variability,
Figure GDA0001263005350000044
a value representing the real-time heart rate variability,
Figure GDA0001263005350000045
representing a resting heart rate variability value;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
PNNSD _ ratio represents the relative value of the number of interval differences with adjacent interval differences of more than 50ms as a percentage of the total number of interval differences, PNNSDexRepresenting the percentage of interval differences in which the real-time adjacent interval difference exceeds 50ms, the PNNSDrestThe number of interval differences representing a difference between rest adjacent intervals of more than 50ms is a percentage of the total number of interval differences.
Wherein the step of training the fatigue state discrimination model comprises:
respectively acquiring electroencephalogram data of a plurality of persons to be tested in a preset test environment;
calculating electroencephalogram multi-scale entropy of the electroencephalogram data under multiple scales;
and taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network, and training the fatigue state discrimination model.
The step of obtaining a corresponding task adjustment strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator comprises the following steps:
and searching a corresponding relation between a preset state and a task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator, and acquiring the corresponding task adjusting strategy.
In a second aspect, the present invention provides a task adjustment apparatus, comprising:
the data acquisition module is used for acquiring electrophysiological data of the current operator;
a state obtaining module, configured to use the electrophysiological data as input of a preset functional state prediction model, a preset psychological state recognition model, and a preset fatigue state discrimination model, respectively run the functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model, and obtain a predicted functional state, a psychological state, and a fatigue state of the current operator;
the strategy acquisition module is used for acquiring a corresponding task regulation strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator;
and the task adjusting module is used for adjusting the task of the current operator according to the task adjusting strategy.
Wherein the apparatus further comprises:
the first training module is used for training the function state prediction model;
the second training module is used for training the psychological state recognition model;
and the third training module is used for training the fatigue state discrimination model.
Wherein the first training module comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state;
the variable acquisition submodule is used for acquiring input variables of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
and the training submodule is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the functional state prediction model.
Wherein the variable acquisition submodule is specifically configured to:
respectively calculating a relative value of electroencephalogram alpha wave band power, a relative value of electroencephalogram beta wave band power, a relative value of electroencephalogram theta wave band power, a relative value of electroencephalogram wave band power, a relative value of heart rate, a relative value of RR interval high-frequency power, a relative value of mean square root of adjacent RR interval differences and a relative value of RR interval standard deviation according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
wherein the content of the first and second substances,
Figure GDA0001263005350000061
Figure GDA0001263005350000062
Figure GDA0001263005350000063
wherein alpha _ ratio represents the relative value of the power of the alpha wave band of the brain electricity, and alphaexRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
Wherein the second training module comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state;
the first training submodule is used for training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network;
and the second training submodule is used for training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
Wherein the first training submodule comprises:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model;
wherein the input variables include: data sample entropy, standard deviation of electrocardio P wave length and electrocardio R wave length mean value.
Wherein the second training submodule comprises:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model;
wherein the input variables include: heart rate variability relative value, standard deviation of electrocardio P wave length, relative value of mean square root of adjacent RR interval differences, relative value of percentage of number of interval differences with adjacent interval difference exceeding 50ms in total interval difference number;
wherein, the
Figure GDA0001263005350000081
Figure GDA0001263005350000082
Figure GDA0001263005350000083
HRV _ ratio represents the relative value of the heart rate variability,
Figure GDA0001263005350000084
a value representing the real-time heart rate variability,
Figure GDA0001263005350000085
representing a resting heart rate variability value;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
PNNSD _ ratio represents the relative value of the number of interval differences with adjacent interval differences of more than 50ms as a percentage of the total number of interval differences, PNNSDexRepresenting the percentage of interval differences in which the real-time adjacent interval difference exceeds 50ms, the PNNSDrestThe number of interval differences representing a difference between rest adjacent intervals of more than 50ms is a percentage of the total number of interval differences.
Wherein the third training module comprises:
the data acquisition submodule is used for respectively acquiring electroencephalogram data of a plurality of persons to be tested in a preset test environment;
the calculation submodule is used for calculating electroencephalogram multi-scale entropy of the electroencephalogram data under multiple scales;
and the training submodule is used for taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network and training the fatigue state discrimination model.
The strategy acquisition module is specifically configured to search a corresponding relationship between a preset state and a task adjustment strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator, and acquire the corresponding task adjustment strategy.
The technical scheme of the invention has the following beneficial effects:
in the embodiment of the invention, the predicted functional state, the psychological state and the fatigue state of the current operator are obtained by inputting the electrophysiological data of the current operator into a preset functional state prediction model, a psychological state recognition model and a fatigue state discrimination model, so that a corresponding task adjusting strategy is obtained, and the task of the current operator is adjusted. The scheme of the embodiment of the invention can adjust the task according to the specific condition of the current operator, thereby ensuring the stable operation of the man-machine system when operating the man-machine system.
Drawings
FIG. 1 is a flowchart of a task adjustment method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a task adjustment method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a BP neural network;
FIG. 4 is a schematic diagram of a task adjustment apparatus according to a third embodiment of the present invention;
fig. 5 is a block diagram of a task adjustment apparatus according to a third embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the present invention will be made with reference to the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
Example one
As shown in fig. 1, a task adjusting method according to a first embodiment of the present invention includes:
step 101, acquiring electrophysiological data of a current operator.
The electrophysiological data of the current operator comprise electroencephalogram data, electrocardiogram data and the like. In various practical applications, the portable electroencephalogram and electrocardio acquisition equipment can be used for acquiring the electroencephalogram, electrocardio, respiration and other electrophysiological data of the current operator at fixed time intervals.
And 102, respectively taking the electrophysiological data as the input of a preset functional state prediction model, a preset psychological state identification model and a preset fatigue state judgment model, respectively operating the functional state prediction model, the psychological state identification model and the fatigue state judgment model, and obtaining the predicted functional state, the psychological state and the fatigue state of the current operator.
In the embodiment of the present invention, a functional state prediction model, a psychological state recognition model, and a fatigue state discrimination model may be trained in advance, and then the electrophysiological data is used as the input of a preset functional state prediction model, a preset psychological state recognition model, and a preset fatigue state discrimination model, and the functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model are respectively operated to obtain the predicted functional state, the psychological state, and the fatigue state of the current operator.
And 103, acquiring a corresponding task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator.
In the embodiment of the invention, the corresponding relation between the preset state and the task adjusting strategy is searched according to the predicted functional state, the psychological state and the fatigue state of the current operator, and the corresponding task adjusting strategy is obtained.
And 104, adjusting the task of the current operator according to the task adjusting strategy.
As can be seen from the above, in the embodiment of the present invention, the predicted functional state, the psychological state, and the fatigue state of the current operator are obtained by inputting the electrophysiological data of the current operator into the preset functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model, so as to obtain the corresponding task adjustment strategy, and adjust the task of the current operator. The scheme of the embodiment of the invention can adjust the task according to the specific condition of the current operator, thereby ensuring the stable operation of the man-machine system when operating the man-machine system.
Example two
In a complex human-computer interaction system, as the working time increases, the operator is likely to feel tired, and anxiety, and the functional state of the operator gradually deteriorates. Thus, when complex and unexpected events occur, the decision cannot be made correctly and misoperation is easy to happen, which causes safety accidents or other losses, such as: nuclear power station accidents, airport traffic accidents, airplane misoperation, etc. In order to avoid the situation, people consider to monitor the functional state of the operator to master whether the operator is competent for the current work task in real time, and once the condition that the functional state of the operator is not good is monitored, measures are taken to adjust the work task of the automatic system to remind the operator to improve the alertness level; in severe cases, the status of the operator is sent to the management department for emergency action.
In practical application, a functional state prediction model, a psychological state recognition model and a fatigue state discrimination model are established by exploring the relationship between electrophysiological data (such as electroencephalogram and electrocardio) of an operator and the functional state and the psychological state of the operator. Predicting the functional state of the electro-physiological data, identifying the psychological state and the fatigue state of the electro-physiological data, and combining the functional state, the psychological state and the fatigue state to make an automatic system task regulation scheme, an operator state regulation scheme and an emergency state regulation scheme.
In the second embodiment of the invention, the portable electroencephalogram and electrocardio acquisition equipment is utilized to acquire the electroencephalogram, electrocardio, respiration and other electrophysiological data of the current operator, monitor the functional state of the current operator and adjust the human-computer interaction system, thereby ensuring a good human-computer interaction state. The specific scheme is as follows:
and (3) acquiring electrophysiological data: in the working process of an operator, acquiring the electroencephalogram, electrocardio and other electrophysiological data of the operator by utilizing portable electroencephalogram and electrocardio acquisition equipment; the functional state of the operator is identified by analyzing the electrophysiological signals: establishing an operator functional state prediction model based on electroencephalogram, electrocardio and other electrophysiological data and an operator mental state identification model, wherein the models are general models irrelevant to individuals; designing a system task regulation scheme based on the function state prediction result of the operator and the psychological state of the operator; according to the designed adjusting scheme, the states of the system and the operators are adjusted, a good human-computer interaction state is kept, the system safety is guaranteed, and the working efficiency is improved.
First, a brief description of the experimental conditions will be given.
In this case, the operation tasks with different difficulties are designed, the operation performance of the operator (i.e. the person to be tested) under different task conditions is obtained, and various electrophysiological indicators of the operator are collected, such as: brain electricity, electrocardio, etc. The experiment usually lasts for a long time (such as 1.25, 1.5 and 1.75 hours, and the time range is optimal in the specific application) so as to obtain the performance and physiological data of the operator in the physical fatigue state.
In the embodiment of the present invention, the experimental material used is Cabin Air Management simulation software (automatic improved Cabin Air Management System, aacams). A capsule air management system is software that simulates human operators in supervising and controlling industrial processes. The system comprises five subsystems which are used for respectively controlling the oxygen concentration, the carbon dioxide concentration, the temperature, the pressure and the humidity in the closed cabin. The subsystems are coupled with each other, and the change of any subsystem can affect other indexes. Such as: when the oxygen valve is opened and the content of oxygen is increased, the atmospheric pressure in the cabin is increased, and the concentration of carbon dioxide is reduced; when the nitrogen concentration is increased to increase the atmospheric pressure in the chamber, the oxygen concentration and the carbon dioxide concentration are caused to decrease while the temperature in the chamber is slowly increased, and so on. Each index has a control threshold and a normal threshold, and when each index changes within the normal threshold, the health of personnel in the cabin is not affected; the control threshold is a stricter index, which is a control target of the automatic control system, and when each index is within the control threshold range, the system is regarded as a fault-free operation. The coupling between subsystems and the threshold limits of the indicators make the operation of the system more complex.
The control system is a typical man-machine interaction system, the control task is completed by the automatic system at ordinary times, and an operator monitors the control task; once the automation system fails, immediate manual intervention by an operator is required to ensure system safety. In the experiment, the number of the subsystems which are manually controlled is changed, the operation difficulty can be adjusted, and therefore the task load of the operator is adjusted, and the purpose of acquiring different functional state data of the operator is achieved.
Each participant needs to participate in two experiments, each experiment is divided into 8 stages, the first stage is a resting stage, and the participant does not do any operation and relaxes for at least 5 minutes; the second to seventh stages, each stage 12, 15, 18 minutes, rest for 1.5, 2, 3 minutes between two adjacent stages, and the last stage is also in a resting state. The above time allocation works best in this particular application environment.
The experimental task is to operate the aCAMS system and ensure the safe operation of the system, namely to ensure that each atmospheric index is within a control threshold. The number of subsystems needing manual operation in each stage is different, and the number of subsystems needing manual operation in 6 operation stages in sequence is as follows: 1,3,4,4,3,1. The particular arrangement described above works best in this particular application environment. After each stage of task, a subjective evaluation table is required to be filled, and the subjective evaluation table comprises 3 items: the degree of effort, the degree of strain, the degree of fatigue, scored as a percentage, with higher scores indicating greater degrees of the project.
In the process of executing tasks, a portable electroencephalograph is used for collecting electroencephalogram data; and acquiring the electrocardio data by using the portable electrocardio equipment.
As shown in fig. 2, a task adjusting method according to a second embodiment of the present invention includes:
step 201, respectively training the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model.
Operator functional status is a dynamic capability that reflects the operator's ability to complete the task currently assigned. In general, when the functional status of the operator is good, the operation performance is good, otherwise, the operation performance is poor. The research shows that: the correlation among the functional state, the task operation performance, part of physiological and psychological indexes of the operators is high. Therefore, reflecting the functional state of the operator through the operation performance is a natural idea; on the other hand, if the physiological and psychological states of the operator can be simultaneously determined, such as: fatigue, tension and the like, and the task performance and the psychological state are combined, so that the functional state of the operator can be comprehensively judged, and an appropriate system regulation scheme can be further formulated.
In order to better obtain the functional state of the operator, a functional state prediction model, a psychological state recognition model and a fatigue state discrimination model are respectively trained in the embodiment of the invention.
(1) Training functional state prediction model
In the process of training the functional state prediction model, the following processes may be included:
respectively acquiring real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state in a preset testing environment; acquiring an input variable of a BP (Back Propagation) neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data; and taking the input variable as the input of the BP neural network, training the BP neural network, and training the function state prediction model. The predetermined test environment may be any test environment.
The acquired electroencephalogram and electrocardiograph data are preprocessed, for example, data cleaning, denoising and the like are performed on the original electroencephalogram and electrocardiograph data, for example, noise and repeated data in the electroencephalogram and electrocardiograph data are removed, missing data are supplemented, and a data source which is as clean and accurate as possible is provided for subsequent data analysis and modeling.
For the obtained data, various indexes of electroencephalogram, electrocardio and other data are needed for data analysis and modeling, including power spectrums, heart rates, heart rate variability and related parameters of different electrocardio waveforms of alpha, beta and theta of each channel of an EEG (electroencephalogram). Here, the data sampling rate is 250, 400, 450, 500, 750, 1000Hz, and the data segment length for calculating each index is 30, 45, 60, 90s, which is generally not longer than 120 s. The particular arrangement described above works best in this particular application environment. The input/output variables of the model are selected by methods such as variance analysis, calculation of correlation coefficients, and the like.
The model input features shown in Table 1 were selected by the I/O variable selection method. In order to solve individual differences among different operators, relative values of all indexes are calculated, namely, the data acquired during the experiment of the operators are compared with the average value of the data acquired during the rest of the operators, and the relative values are calculated. The screening results show that: for the data set of the mixed data of all the participants, the correlation between the relative value of each index and the performance of the operator has obvious advantages. By comparison, 8 features shown in table 1 were selected as input variables of the BP neural network.
TABLE 1 BP neural network input variables
Figure GDA0001263005350000131
Figure GDA0001263005350000141
Wherein alpha _ ratio represents the relative value of the power of the alpha wave band of the brain electricity, and alphaexRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean evolution representing the square of the difference between adjacent RR intervals at rest；
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
In practical application, the functional state of the operator needs to be predicted, so the model input is data of the operator in the t-th time unit, and the output is a predicted value of the operator in the (t +1) -th time unit, namely the prediction of the functional state of the operator, specifically the operation performance of the operator.
In an embodiment of the present invention, depending on the number of input/output features, as shown in FIG. 3: a BP neural network with 8 inputs-1 outputs is used. The network comprises three layers: the system comprises an input layer, a hidden layer and an output layer, wherein the input layer comprises 8 nodes, the hidden layer comprises 16 nodes, and the output layer comprises 1 node.
The BP algorithm is a supervised learning algorithm, and the main idea is as follows: inputting a learning sample, repeatedly adjusting and training the weight and the deviation of the network by using a back propagation algorithm to enable the output vector to be as close to the expected vector as possible, finishing training when the error square sum of the network output layer is smaller than a specified error, and storing the weight and the deviation of the network. The network learning mode pair refers to training data, namely 8 input indexes of a time unit t and performance output of a time unit t + 1; the error is the error of the output of the network model and the output of the learning mode pair. And after training, determining the connection weight and the threshold value between layers to obtain model parameters.
(2) Training psychological state recognition model
The mental state recognition model comprises a struggle degree judgment model and a tension state judgment model.
In the process, real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state are respectively obtained in a preset testing environment; training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data and a BP neural network; and training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
And in the process of training the effort degree discrimination model, calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data. And then, taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model. Wherein the input variables include: the data sample entropy, the standard deviation of the electrocardiograph P wave length, and the electrocardiograph R wavelength degree mean are shown in Table 2.
TABLE 2. effort discrimination model input variables
Name of variable Of significance Computing
EEG_Samentr EEG raw data sample entropy Computing 30s EEG raw data sample entropy
ECG_Pstd Standard deviation of electrocardio P wave length Calculating the standard deviation of the P-wave length within 30 seconds
ECG_Rmean Mean value of R wavelength of electrocardio Calculate the mean of the R-wave length in 30 seconds
And in the process of training a tension degree judging model, calculating an input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data. And then, taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model. Wherein the input variables include: the relative values of the heart rate variability, the standard deviation of the electrocardio P-wave length, the relative value of the square root of the mean value of the adjacent RR interval differences, and the relative value of the percentage of the number of the interval differences with the adjacent interval difference exceeding 50ms in the total interval difference number are shown in the table 3.
TABLE 3 operator stress level discriminant model input features
Figure GDA0001263005350000161
(3) Training fatigue state discrimination model
In the process, electroencephalogram data of a plurality of persons to be tested are respectively acquired under a preset test environment, and electroencephalogram multi-scale entropies of the electroencephalogram data under a plurality of scales (such as 2-6) are calculated. And taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network, and training the fatigue state discrimination model. The output of the model is a fatigue score.
Step 202, acquiring electrophysiological data of the current operator.
The electrophysiological data of the current operator comprise electroencephalogram data, electrocardiogram data and the like. In various practical applications, the data can be acquired at fixed time intervals.
And 203, respectively inputting the electrophysiological data as a preset functional state prediction model, a preset psychological state identification model and a preset fatigue state judgment model, respectively operating the functional state prediction model, the psychological state identification model and the fatigue state judgment model, and obtaining the predicted functional state, the psychological state and the fatigue state of the current operator.
And 204, acquiring a corresponding task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator.
In the embodiment of the invention, the corresponding relation between the preset state and the task adjusting strategy is searched according to the predicted functional state, the psychological state and the fatigue state of the current operator, and the corresponding task adjusting strategy is obtained.
In the step, a system task mediation scheme is designed according to the operator functional state prediction model, the psychological state discrimination model and the fatigue state discrimination model.
The adjusting method depends on the functional state, the effort degree, the tension degree and the fatigue degree of the operator, which are obtained by the judgment of the model, such as: the functional state of the operator can be divided into 3 grades of good, medium and poor, and the rest dimensions are divided into 3 grades of high, medium and low. The design of the task regulation scheme takes the overall goals of moderate effort degree and tension degree and low fatigue degree as the overall goal, and a regulation scheme library is designed. The above correspondence can be shown in table 4 below:
TABLE 4 correspondence of states to task adjustment strategies
Functional status Degree of effort Degree of tension Degree of fatigue Task adjustment strategy
Good taste In In In Is not adjusted
Good taste Height of Height of Height of Task reduction
Difference (D) In Height of Height of Task reduction
Difference (D) Is low in Is low in Is low in Task augmentation
… … … … …
An operator wears wearable electroencephalogram and electrocardio acquisition equipment to acquire electroencephalogram and electrocardio data in real time. According to the collected data, the functional state of the next time interval is obtained by applying an operator functional state prediction model, a psychological state discrimination model and a fatigue state discrimination model, the psychological state and the fatigue state of the current time interval are input into an adjustment scheme library to obtain a task adjustment scheme, and automatic adjustment is realized by the system.
And step 205, adjusting the task of the current operator according to the task adjustment strategy.
As can be seen from the above, in the embodiment of the present invention, the predicted functional state, the psychological state, and the fatigue state of the current operator are obtained by inputting the electrophysiological data of the current operator into the preset functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model, so as to obtain the corresponding task adjustment strategy, and adjust the task of the current operator. The scheme of the embodiment of the invention can adjust the task according to the specific condition of the current operator, thereby ensuring the stable operation of the man-machine system when operating the man-machine system. In addition, the functional state, the psychological state and the fatigue state of the personnel are monitored through the wearable equipment, and the monitoring method is more accurate and real-time than the prior art; the real-time performance enables the monitoring of the physical and mental states of the person to be used in production practice, and the method has important significance for the safe and normal operation of a complex man-machine system.
EXAMPLE III
As shown in fig. 4, a task adjusting apparatus according to a third embodiment of the present invention includes:
a data obtaining module 501, configured to obtain electrophysiological data of a current operator; a state obtaining module 502, configured to use the electrophysiological data as input of a preset functional state prediction model, a preset psychological state identification model, and a preset fatigue state discrimination model, respectively run the functional state prediction model, the psychological state identification model, and the fatigue state discrimination model, and obtain a predicted functional state, a psychological state, and a fatigue state of the current operator; a strategy obtaining module 503, configured to obtain a corresponding task adjustment strategy according to the predicted functional state, the psychological state, and the fatigue state of the current operator; and a task adjusting module 504, configured to adjust the task of the current operator according to the task adjusting policy.
As shown in fig. 5, the apparatus further includes:
a first training module 505, configured to train the functional state prediction model; a second training module 506, configured to train a mental state recognition model; and a third training module 507, configured to train a fatigue state discrimination model.
Wherein the first training module 505 comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state; the variable acquisition submodule is used for acquiring input variables of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data; and the training submodule is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the functional state prediction model.
Specifically, the variable acquisition submodule is specifically configured to:
respectively calculating a relative value of electroencephalogram alpha wave band power, a relative value of electroencephalogram beta wave band power, a relative value of electroencephalogram theta wave band power, a relative value of electroencephalogram wave band power, a relative value of heart rate, a relative value of RR interval high-frequency power, a relative value of mean square root of adjacent RR interval differences and a relative value of RR interval standard deviation according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
wherein the content of the first and second substances,
Figure GDA0001263005350000181
Figure GDA0001263005350000191
Figure GDA0001263005350000192
wherein alpha _ ratio represents the relative value of the power of the alpha wave band of the brain electricity, and alphaexRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
Wherein the second training module 506 comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state;
the first training submodule is used for training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network;
and the second training submodule is used for training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
Specifically, the first training submodule includes:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data; the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model; wherein the input variables include: data sample entropy, standard deviation of electrocardio P wave length and electrocardio R wave length mean value.
Specifically, the second training submodule includes:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data; the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model; wherein the input variables include: heart rate variability relative value, standard deviation of electrocardio P wave length, relative value of mean square root of adjacent RR interval differences, relative value of percentage of number of interval differences with adjacent interval difference exceeding 50ms in total interval difference number;
wherein, the
Figure GDA0001263005350000201
Figure GDA0001263005350000202
HRV _ ratio represents the relative value of the heart rate variability,
Figure GDA0001263005350000203
a value representing the real-time heart rate variability,
Figure GDA0001263005350000204
representing a resting heart rate variability value;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
PNNSD _ ratio represents the relative value of the number of interval differences with adjacent interval differences of more than 50ms as a percentage of the total number of interval differences, PNNSDexRepresenting the percentage of interval differences in which the real-time adjacent interval difference exceeds 50ms, the PNNSDrestThe number of interval differences representing a difference between rest adjacent intervals of more than 50ms is a percentage of the total number of interval differences.
Wherein the third training module 507 comprises:
the data acquisition submodule is used for respectively acquiring electroencephalogram data of a plurality of persons to be tested in a preset test environment; the calculation submodule is used for calculating electroencephalogram multi-scale entropy of the electroencephalogram data under multiple scales; and the training submodule is used for taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network and training the fatigue state discrimination model.
Specifically, the policy obtaining module 503 is specifically configured to search a corresponding relationship between a preset state and a task adjusting policy according to the predicted functional state, the psychological state, and the fatigue state of the current operator, and obtain a corresponding task adjusting policy.
The working principle of the device according to the invention can be referred to the description of the method embodiment described above.
As can be seen from the above, in the embodiment of the present invention, the predicted functional state, the psychological state, and the fatigue state of the current operator are obtained by inputting the electrophysiological data of the current operator into the preset functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model, so as to obtain the corresponding task adjustment strategy, and adjust the task of the current operator. The scheme of the embodiment of the invention can adjust the task according to the specific condition of the current operator, thereby ensuring the stable operation of the man-machine system when operating the man-machine system.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be physically included alone, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) to execute some steps of the transceiving method according to various embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (14)

1. A task adjustment method, comprising:
acquiring electrophysiological data of a current operator;
respectively inputting the electrophysiological data as a preset functional state prediction model, a psychological state recognition model and a fatigue state discrimination model, and respectively operating the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model to obtain a predicted functional state, a psychological state and a fatigue state of the current operator;
acquiring a corresponding task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator;
adjusting the task of the current operator according to the task adjusting strategy;
before the step of acquiring electrophysiological data of the current operator, the method further comprises:
respectively training the functional state prediction model, the psychological state recognition model and the fatigue state discrimination model, wherein the psychological state recognition model comprises: respectively acquiring real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state in a preset testing environment; training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data and a BP neural network, wherein an input variable of the BP neural network is calculated according to the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data and the resting electrocardiograph data; taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model; the input variables include: data sample entropy, standard deviation of electrocardio P wave length and electrocardio R wave length mean value.
2. The method of claim 1, wherein the step of training the functional state prediction model comprises:
respectively acquiring real-time electroencephalogram data and real-time electrocardiogram data of a plurality of persons to be tested, and resting electroencephalogram data and resting electrocardiogram data of the plurality of persons to be tested in a resting state in a preset testing environment;
acquiring an input variable of a BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
and taking the input variable as the input of the BP neural network, training the BP neural network, and training the function state prediction model.
3. The method of claim 2, wherein the step of obtaining the input variable of the BP neural network from the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, and the resting electrocardiograph data comprises:
respectively calculating a relative value of electroencephalogram alpha wave band power, a relative value of electroencephalogram beta wave band power, a relative value of electroencephalogram theta wave band power, a relative value of electroencephalogram wave band power, a relative value of heart rate, a relative value of RR interval high-frequency power, a relative value of mean square root of adjacent RR interval differences and a relative value of RR interval standard deviation according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
wherein the content of the first and second substances,
Figure FDA0002577622130000021
Figure FDA0002577622130000022
Figure FDA0002577622130000023
wherein, alpha _ ratio represents the power of electroencephalogram alpha wave bandRelative value of (a)exRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
4. The method of claim 1, wherein the step of training the mental state recognition model further comprises:
and training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
5. The method of claim 4, wherein the step of training a stress level discrimination model based on the real-time electroencephalogram data, the real-time electrocardiograph data, the resting electroencephalogram data, the resting electrocardiograph data, and a BP neural network comprises:
calculating an input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
taking the input variable as the input of the BP neural network, training the BP neural network, and training the effort degree discrimination model;
wherein the input variables include: heart rate variability relative value, standard deviation of electrocardio P wave length, relative value of mean square root of adjacent RR interval differences, relative value of percentage of number of interval differences with adjacent interval difference exceeding 50ms in total interval difference number;
wherein, the
Figure FDA0002577622130000031
Figure FDA0002577622130000032
Figure FDA0002577622130000033
HRV _ ratio represents the relative value of the heart rate variability,
Figure FDA0002577622130000034
a value representing the real-time heart rate variability,
Figure FDA0002577622130000035
representing a resting heart rate variability value;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
PNNSD _ ratio represents the adjacent interval differenceRelative number of interval differences of more than 50ms in percentage of total number of interval differences, PNNSDexRepresenting the percentage of interval differences in which the real-time adjacent interval difference exceeds 50ms, the PNNSDrestThe number of interval differences representing a difference between rest adjacent intervals of more than 50ms is a percentage of the total number of interval differences.
6. The method of claim 1, wherein the step of training the fatigue state discrimination model comprises:
respectively acquiring electroencephalogram data of a plurality of persons to be tested in a preset test environment;
calculating electroencephalogram multi-scale entropy of the electroencephalogram data under multiple scales;
and taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network, and training the fatigue state discrimination model.
7. The method according to any one of claims 1-6, wherein the step of deriving a corresponding task adjustment strategy based on the predicted functional state, psychological state, and fatigue state of the current operator comprises:
and searching a corresponding relation between a preset state and a task adjusting strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator, and acquiring the corresponding task adjusting strategy.
8. A task adjustment device, comprising:
the data acquisition module is used for acquiring electrophysiological data of the current operator;
a state obtaining module, configured to use the electrophysiological data as input of a preset functional state prediction model, a preset psychological state recognition model, and a preset fatigue state discrimination model, respectively run the functional state prediction model, the psychological state recognition model, and the fatigue state discrimination model, and obtain a predicted functional state, a psychological state, and a fatigue state of the current operator;
the strategy acquisition module is used for acquiring a corresponding task regulation strategy according to the predicted functional state, the psychological state and the fatigue state of the current operator;
the task adjusting module is used for adjusting the task of the current operator according to the task adjusting strategy;
the device further comprises:
the first training module is used for training the function state prediction model;
the second training module is used for training the psychological state recognition model;
the third training module is used for training a fatigue state discrimination model;
the second training module comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state;
the first training submodule is used for training an effort degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network;
the first training submodule includes:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model;
wherein the input variables include: data sample entropy, standard deviation of electrocardio P wave length and electrocardio R wave length mean value.
9. The apparatus of claim 8, wherein the first training module comprises:
the data acquisition sub-module is used for respectively acquiring real-time electroencephalogram data and real-time electrocardio data of a plurality of persons to be tested under a preset test environment, and resting electroencephalogram data and resting electrocardio data of the plurality of persons to be tested in a resting state;
the variable acquisition submodule is used for acquiring input variables of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
and the training submodule is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the functional state prediction model.
10. The apparatus of claim 9, wherein the variable acquisition submodule is specifically configured to:
respectively calculating a relative value of electroencephalogram alpha wave band power, a relative value of electroencephalogram beta wave band power, a relative value of electroencephalogram theta wave band power, a relative value of electroencephalogram wave band power, a relative value of heart rate, a relative value of RR interval high-frequency power, a relative value of mean square root of adjacent RR interval differences and a relative value of RR interval standard deviation according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data and the resting electrocardio data;
wherein the content of the first and second substances,
Figure FDA0002577622130000051
Figure FDA0002577622130000061
Figure FDA0002577622130000062
wherein alpha _ ratio represents the relative value of the power of the alpha wave band of the brain electricity, and alphaexRepresenting the power of the alpha band of the real-time brain electrical wave, alpharestRepresenting the power of alpha wave band of resting electroencephalogram;
beta _ ratio represents the relative value of the power of the beta band of the brain, betaexRepresenting power of beta band of real-time brain electrical wave, betarestRepresenting the power of a beta wave band of resting electroencephalogram;
theta _ ratio represents the relative value of the power of the electroencephalogram theta band, thetaexRepresenting power of real-time electroencephalogram theta band, thetarestRepresenting the power of a theta waveband of resting electroencephalogram;
'ratio' represents the relative value of the brain band power,exrepresents the power of the real-time brain wave band,restrepresenting resting electroencephalogram band power;
HR _ ratio represents the relative value of the heart rate, HRexRepresenting real-time heart rate values, HRrestRepresenting a resting heart rate value;
HF _ ratio denotes the relative value of the RR interval high-frequency power, HFexRepresenting real-time RR interval high-frequency power, HFrestRepresenting the resting RR interval high-frequency power;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
SDNN _ ratio represents the relative value of the standard deviation of RR intervals, SDNNexRepresenting standard deviation of real-time RR intervals, SDNNrestThe standard deviation of the resting RR interval is indicated.
11. The apparatus of claim 8, wherein the second training module further comprises:
and the second training submodule is used for training a tension degree discrimination model according to the real-time electroencephalogram data, the real-time electrocardio data, the resting electroencephalogram data, the resting electrocardio data and the BP neural network.
12. The apparatus of claim 11, wherein the second training submodule comprises:
the variable acquisition unit is used for calculating the input variable of the BP neural network according to the real-time electroencephalogram data, the real-time electrocardiogram data, the resting electroencephalogram data and the resting electrocardiogram data;
the training unit is used for taking the input variable as the input of the BP neural network, training the BP neural network and training the effort degree discrimination model;
wherein the input variables include: heart rate variability relative value, standard deviation of electrocardio P wave length, relative value of mean square root of adjacent RR interval differences, relative value of percentage of number of interval differences with adjacent interval difference exceeding 50ms in total interval difference number;
wherein the content of the first and second substances,
Figure FDA0002577622130000071
Figure FDA0002577622130000072
Figure FDA0002577622130000073
HRV _ ratio represents the relative value of the heart rate variability,
Figure FDA0002577622130000074
a value representing the real-time heart rate variability,
Figure FDA0002577622130000075
representing a resting heart rate variability value;
RMSSD _ ratio represents the relative value of the mean square root of the difference between adjacent RR intervals, RMSSDexMean square root, RMSSD, representing the square of the real-time adjacent RR interval differencesrestMean square root representing the square of the difference between the resting adjacent RR intervals;
PNNSD _ ratio represents the relative value of the number of interval differences with adjacent interval differences of more than 50ms as a percentage of the total number of interval differences, PNNSDexRepresenting the percentage of interval differences in which the real-time adjacent interval difference exceeds 50ms, the PNNSDrestTo representThe number of interval differences between adjacent intervals at rest differing by more than 50ms is a percentage of the total number of interval differences.
13. The apparatus of claim 8, wherein the third training module comprises:
the data acquisition submodule is used for respectively acquiring electroencephalogram data of a plurality of persons to be tested in a preset test environment;
the calculation submodule is used for calculating electroencephalogram multi-scale entropy of the electroencephalogram data under multiple scales;
and the training submodule is used for taking the electroencephalogram multi-scale entropy as the input of a BP neural network, training the BP neural network and training the fatigue state discrimination model.
14. The device according to any one of claims 8 to 13, wherein the policy obtaining module is specifically configured to search a corresponding relationship between a preset state and a task adjustment policy according to the predicted functional state, the psychological state, and the fatigue state of the current operator, and obtain the corresponding task adjustment policy.
CN201611216913.2A 2016-12-26 2016-12-26 Task adjusting method and device Active CN108241431B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611216913.2A CN108241431B (en) 2016-12-26 2016-12-26 Task adjusting method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611216913.2A CN108241431B (en) 2016-12-26 2016-12-26 Task adjusting method and device

Publications (2)

Publication Number Publication Date
CN108241431A CN108241431A (en) 2018-07-03
CN108241431B true CN108241431B (en) 2020-10-30

Family

ID=62704924

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611216913.2A Active CN108241431B (en) 2016-12-26 2016-12-26 Task adjusting method and device

Country Status (1)

Country Link
CN (1) CN108241431B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114052747A (en) * 2020-08-10 2022-02-18 西安领跑网络传媒科技股份有限公司 Brain state adjusting method and device, computer equipment and storage medium
CN112450950B (en) * 2020-12-10 2021-10-22 南京航空航天大学 Brain-computer aided analysis method and system for aviation accident

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101401724A (en) * 2001-06-13 2009-04-08 康普麦迪克斯有限公司 Methods and apparatus for monitoring consciousness
US8798735B1 (en) * 2002-07-12 2014-08-05 Bionova Technologies Inc. Method and apparatus for the estimation of anesthetic depth using wavelet analysis of the electroencephalogram
CN103989485A (en) * 2014-05-07 2014-08-20 朱晓斐 Human body fatigue evaluation method based on brain waves
CN104385951A (en) * 2014-07-02 2015-03-04 河北科技大学 Vehicle seat system capable of monitoring physiology and psychology of rider
CN104665849A (en) * 2014-12-11 2015-06-03 西南交通大学 Multi-physiological signal multi-model interaction-based high-speed railway dispatcher stress detecting method
CN107788970A (en) * 2017-11-15 2018-03-13 中国航天员科研训练中心 A kind of Mental Workload appraisal procedure based on the fusion of multi-physiological-parameter method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101401724A (en) * 2001-06-13 2009-04-08 康普麦迪克斯有限公司 Methods and apparatus for monitoring consciousness
US8798735B1 (en) * 2002-07-12 2014-08-05 Bionova Technologies Inc. Method and apparatus for the estimation of anesthetic depth using wavelet analysis of the electroencephalogram
CN103989485A (en) * 2014-05-07 2014-08-20 朱晓斐 Human body fatigue evaluation method based on brain waves
CN104385951A (en) * 2014-07-02 2015-03-04 河北科技大学 Vehicle seat system capable of monitoring physiology and psychology of rider
CN104665849A (en) * 2014-12-11 2015-06-03 西南交通大学 Multi-physiological signal multi-model interaction-based high-speed railway dispatcher stress detecting method
CN107788970A (en) * 2017-11-15 2018-03-13 中国航天员科研训练中心 A kind of Mental Workload appraisal procedure based on the fusion of multi-physiological-parameter method

Also Published As

Publication number Publication date
CN108241431A (en) 2018-07-03

Similar Documents

Publication Publication Date Title
Şen et al. A comparative study on classification of sleep stage based on EEG signals using feature selection and classification algorithms
Hsu et al. Automatic sleep stage recurrent neural classifier using energy features of EEG signals
Laine et al. Selection of input features across subjects for classifying crewmember workload using artificial neural networks
Sarkar et al. Classification of cognitive load and expertise for adaptive simulation using deep multitask learning
Tian et al. A hierarchical classification method for automatic sleep scoring using multiscale entropy features and proportion information of sleep architecture
CN111680913B (en) Overload work detection method and system for warmen
CN110123266B (en) Maneuvering decision modeling method based on multi-modal physiological information
Dell’Agnola et al. Cognitive workload monitoring in virtual reality based rescue missions with drones
CN108241431B (en) Task adjusting method and device
Zhang et al. Classifying human operator functional state based on electrophysiological and performance measures and fuzzy clustering method
Wang et al. Decoding pilot behavior consciousness of EEG, ECG, eye movements via an SVM machine learning model
Lesmana et al. Sleep stage identification using the combination of ELM and PSO based on ECG signal and HRV
Qu et al. Classification of mental workload based on multiple features of ECG signals
WO2024032728A1 (en) Method and apparatus for evaluating intelligent human-computer coordination system, and storage medium
Vatankhah et al. An intelligent system for diagnosing sleep stages using wavelet coefficients
Chai et al. Classification of driver fatigue in an electroencephalography-based countermeasure system with source separation module
CN116504362A (en) Sleep adjustment scheme generation method, device, equipment and storage medium
Wei et al. An investigation of pilot emotion change detection based on multimodal physiological signals
CN115017996A (en) Mental load prediction method and system based on multiple physiological parameters
Cao et al. Recognition of cognitive load with a stacking network ensemble of denoising autoencoders and abstracted neurophysiological features
Zhang et al. Pattern classification of instantaneous mental workload using ensemble of convolutional neural networks
Wang et al. Research on fatigue identification methods based on low-load wearable ECG monitoring devices
Li et al. Assessment of firefighter-training effectiveness in China based on human-factor parameters and machine learning
Bousbia-Salah et al. Time-frequency processing method of epileptic EEG signals
EP4343500A1 (en) Method and apparatus for evaluating intelligent human-computer coordination system, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant