CN109460145B - Automatic system decision rapid intervention method based on brain recognition - Google Patents
Automatic system decision rapid intervention method based on brain recognition Download PDFInfo
- Publication number
- CN109460145B CN109460145B CN201811204037.0A CN201811204037A CN109460145B CN 109460145 B CN109460145 B CN 109460145B CN 201811204037 A CN201811204037 A CN 201811204037A CN 109460145 B CN109460145 B CN 109460145B
- Authority
- CN
- China
- Prior art keywords
- decision
- electroencephalogram
- action
- automatic system
- intervention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Dermatology (AREA)
- Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Human Computer Interaction (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to a brain recognition-based automatic system decision rapid intervention method, which belongs to the technical field of automation and solves the automatic intervention problem of an automatic system, and the method comprises the steps of training a wrong decision action-electroencephalogram characteristic model process; and (4) carrying out real-time error decision action detection by using the error decision action-electroencephalogram characteristic model, and carrying out an online decision intervention process. The invention realizes quick intervention on the automatic system, timely corrects or prevents wrong decisions or unfavorable decisions of the automatic system, and reduces the loss caused by the wrong decisions or the unfavorable decisions of the automatic system.
Description
Technical Field
The invention relates to the technical field of automation, in particular to a brain recognition-based automatic system decision rapid intervention method.
Background
In industrial production, an automatic system plays an extremely important role, and the production efficiency is effectively improved. Due to the limitations of the intelligence level of the automation system, when the automation system automatically completes the operation, an erroneous decision or an adverse decision may be made, thereby causing unpredictable influence. Moreover, when the automated system makes a wrong decision or an unfavorable decision, it is currently mainly dependent on manual intervention. But the manual intervention mode has the defect of long response time. Therefore, a more efficient and faster intervention method is urgently needed to achieve fast intervention on the automation system.
Disclosure of Invention
In view of the above analysis, the present invention aims to provide a brain-recognition-based method for fast intervention of an automated system decision, so as to solve the problem of fast intervention when an automated system makes a wrong decision or an unfavorable decision, effectively shorten intervention response time, and reduce the loss caused by the wrong decision or the unfavorable decision of the automated system.
The purpose of the invention is mainly realized by the following technical scheme:
an automatic system decision intervention method based on brain recognition comprises,
training a decision action-electroencephalogram feature model: establishing a decision-making action training sample set, and synchronously acquiring electroencephalogram signals of operators watching the automatic system when the automatic system executes the training sample set, extracting corresponding electroencephalogram characteristics, and performing model training to obtain a training error decision-making action-electroencephalogram characteristic model;
and (3) online real-time decision intervention: collecting electroencephalogram signals of an operator watching an automatic system to execute decision-making actions in real time, extracting corresponding electroencephalogram characteristics, inputting the electroencephalogram characteristics into the wrong decision-making action-electroencephalogram characteristic model for detection, and sending an intervention control instruction to the automatic system for decision-making intervention when the wrong decision-making action is detected.
Further, the decision action training sample set comprises correct decision action samples and wrong decision action samples.
Furthermore, the electroencephalogram signals of the operators are synchronously collected when the operators stare at the automatic system to execute the decision-making action, and the electroencephalogram signals of the operators are immediately extracted after the automatic system executes the decision-making action through a sliding window with preset duration.
Further, the electroencephalogram features include time domain features, space domain features, and frequency domain features.
Further, an ESSP method is adopted to extract the electroencephalogram characteristics of the electroencephalogram signals in the sliding window.
Further, the detection model training process includes,
1) when each training sample is executed by the automatic system, the electroencephalogram signal of an operator is watched by the automatic system, wherein the electroencephalogram signal is the electroencephalogram signal in a preset duration sliding window;
2) extracting electroencephalogram characteristics of electroencephalogram signals corresponding to each training sample, and establishing an electroencephalogram characteristic and decision action type mapping data set;
3) inputting the electroencephalogram characteristics as input, and inputting the decision action types as labels into a detection model to carry out model parameter training to obtain model parameters;
4) and establishing a corresponding relation between the error decision action and the electroencephalogram characteristics according to the obtained model parameters, so as to obtain the error decision action-electroencephalogram characteristic model.
Further, the detection model is an SVM support vector machine model.
Further, the online real-time decision intervention process comprises,
1) collecting electroencephalogram signals of an operator watching the execution of the automatic system in a sliding window with preset duration in the process of executing the automatic system in real time;
2) extracting the electroencephalogram characteristics of the electroencephalogram signals, inputting an error decision action-electroencephalogram characteristic model, and detecting error decision actions;
3) if no error decision action exists in the detection result, returning to 1); if the detection result has a wrong decision action, outputting a decision intervention instruction to the automatic system;
4) and the automatic system receives the decision intervention instruction and intervenes the wrong decision action.
Further, the preset duration of the sliding window is set according to the response speed of the human brain to the emergency.
Further, the preset duration of the sliding window ranges from 100ms to 500 ms.
The invention has the following beneficial effects:
the brain cognition-based automatic system rapid decision intervention method has the advantage of short response time. Compared with a manual intervention method, the brain-recognition-based automatic system quick decision intervention method can realize quick intervention on the automatic system, timely correct or prevent wrong decisions or unfavorable decisions of the automatic system, and reduce loss caused by the wrong decisions or the unfavorable decisions of the automatic system.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is a flow chart of a training method for a detection model according to an embodiment of the present invention;
FIG. 2 is a flowchart of a real-time intervention method for online decision-making according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and which together with the embodiments of the invention serve to explain the principles of the invention.
The embodiment of the invention discloses an automatic system decision intervention method based on brain recognition,
the automation system described in this embodiment includes unmanned vehicles, drones, and the like.
The brain-based recognition automated system decision intervention method of the present embodiment includes,
training a decision action-electroencephalogram feature model: establishing a decision-making action training sample set, and synchronously acquiring electroencephalogram signals of operators watching the automatic system when the automatic system executes the training sample set, extracting corresponding electroencephalogram characteristics, and performing model training to obtain a training error decision-making action-electroencephalogram characteristic model;
and (3) online real-time decision intervention: collecting electroencephalogram signals of an operator watching an automatic system to execute decision-making actions in real time, extracting corresponding electroencephalogram characteristics, inputting the electroencephalogram characteristics into the wrong decision-making action-electroencephalogram characteristic model for detection, and sending an intervention control instruction to the automatic system for decision-making intervention when the wrong decision-making action is detected.
Specifically, as shown in fig. 1, the training of the detection model includes the following steps:
step S101, establishing a decision action training sample set;
the samples in the decision-making action training sample set are the event of executing the decision-making action for the automatic system, and the training sample set comprises the correct decision-making action samples and the wrong decision-making action samples.
In order to improve the training effect, the training samples in the set of established decision-making action training samples should have universality, need to be specially designed for the automatic system, and have a certain number, and can relate to various decision-making action types of the automatic system; for example, 1000 automated systems are designed to perform decision-action events, including 500 correct decision-action types and 500 incorrect decision-action types.
S102, when each training sample is executed by the automatic system, the electroencephalogram signal of an operator is watched on the automatic system;
when observing that the automatic system performs decision-making action, an operator can correspondingly change the electroencephalogram signal, and particularly, when observing that the automatic system makes a wrong or unfavorable decision, the operator can generate an electroencephalogram negative wave signal related to the wrong or unfavorable decision.
In the embodiment, a sliding window with preset duration is utilized to collect electroencephalograms of operators watching the automatic system to execute each training sample;
particularly, the reaction characteristic of the human brain determines that the reaction is the most violent in the emergency and slowly calms after the emergency occurs, so that the preset duration of the sliding window is set according to the reaction speed of the human brain to the emergency.
According to the reaction speed of the normal human brain, the preset duration range of the sliding window can be set to be 100ms to 500ms, for example 200ms, so that the electroencephalogram signals which are acutely reacted by the human brain when the human brain executes each training sample on the automatic system, especially when the sample is wrongly or unfavorably decided, can be acquired, and the interference of acquiring the electroencephalogram signals after the human brain is calm and the electroencephalogram signals acutely reacted can be avoided.
S103, extracting electroencephalogram characteristics of electroencephalogram signals corresponding to each training sample, and establishing an electroencephalogram characteristic and decision action type mapping data set;
preprocessing the acquired electroencephalogram signals, and then performing time domain, space domain and frequency domain analysis to obtain electroencephalogram characteristics including time domain characteristics, space domain characteristics and frequency domain characteristics;
wherein, the time domain characteristics comprise the length of the latency period of the N100 component, the amplitude and the like;
the spatial domain characteristics comprise different brain electroencephalogram signal intensities;
the frequency domain characteristics include the energy magnitudes of the different frequency bands.
Optionally, the electroencephalogram feature can be extracted by adopting an event-related potential spatial frequency mode, namely an ESSP method. The signal-to-noise ratio of the target signal can be improved by an ESSP method.
The ESSP method comprises the following basic steps:
the first step is as follows: and establishing a generating model of the spliced electroencephalogram by linearly combining the electroencephalogram signals of all times.
The second step is that: and obtaining the generating model parameters through maximum posterior estimation.
The third step: and (4) carrying out spatial frequency mode decomposition on the electroencephalogram signal through the established generative model to obtain the electroencephalogram characteristics.
S104, inputting the acquired electroencephalogram characteristics as input and the decision action type as a label into a detection model for model parameter training to acquire model parameters;
the adopted detection model can be an SVM support vector machine model; training a model by adopting training samples comprising 500 correct decision action types and 500 wrong decision action types and corresponding electroencephalogram characteristics to obtain classifier parameters;
and S105, establishing a corresponding relation between the wrong decision-making action and the electroencephalogram characteristics according to the obtained model parameters, and obtaining the wrong decision-making action-electroencephalogram characteristic model.
And performing parameter training of the SVM support vector machine model through the training sample of the wrong decision-making action type and the corresponding electroencephalogram characteristics to obtain classifier parameters capable of reflecting the corresponding relation between the wrong decision-making action and the electroencephalogram characteristics, and establishing a wrong decision-making action-electroencephalogram characteristic model.
Specifically, as shown in fig. 2, the online real-time decision intervention includes the following steps:
step S201, collecting electroencephalogram signals of an operator watching execution of the automatic system in a sliding window with preset duration in real time in the execution process of the automatic system;
in order to match the established error decision action-electroencephalogram feature model, the preset duration of the sliding window adopted in the step is the same as the preset duration adopted when the error decision action-electroencephalogram feature model is trained.
Step S202, extracting electroencephalogram characteristics of the electroencephalogram signals, inputting an error decision action-electroencephalogram characteristic model, and detecting error decision actions;
step S203, if no error decision action exists in the detection result, returning to the step S201, and continuously acquiring the electroencephalogram signals of the operator; if the detection result has a wrong decision action, outputting a decision intervention instruction to the automatic system;
and S204, the automatic system receives the decision intervention instruction and intervenes the wrong decision action.
Because the cognition of the human brain is directly interpreted, the automatic system is directly interfered by the interpretation result, and the execution of the automatic good system is interfered by the limbs without the operation personnel after the human brain reflects, the intervention response time is greatly shortened, the intervention efficiency is improved, and the loss caused by wrong decision or unfavorable decision of the automatic system can be reduced.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (9)
1. An automatic system decision intervention method based on brain recognition is characterized by comprising the following steps,
training a decision action-electroencephalogram feature model: establishing a decision-making action training sample set, and synchronously acquiring electroencephalogram signals of operators watching the automatic system when the automatic system executes the training sample set, extracting corresponding electroencephalogram characteristics, and performing model training to obtain a training error decision-making action-electroencephalogram characteristic model; the method specifically comprises the following steps:
step S101, establishing a decision action training sample set;
s102, acquiring electroencephalogram signals when an operator watches the automatic system to execute each training sample when the automatic system executes each training sample;
s103, extracting electroencephalogram characteristics of electroencephalogram signals corresponding to each training sample, and establishing an electroencephalogram characteristic and decision action type mapping data set;
s104, inputting the obtained electroencephalogram characteristics as input, and inputting the acquired electroencephalogram characteristics as a label into a detection model to train model parameters to obtain model parameters;
s105, establishing a corresponding relation between an error decision action and electroencephalogram characteristics according to the obtained model parameters to obtain an error decision action-electroencephalogram characteristic model;
and (3) online real-time decision intervention: collecting electroencephalogram signals of an operator watching an automatic system to execute decision-making actions in real time, extracting corresponding electroencephalogram characteristics, inputting the electroencephalogram characteristics into the wrong decision-making action-electroencephalogram characteristic model for detection, and sending an intervention control instruction to the automatic system for decision-making intervention when the wrong decision-making action is detected.
2. A decision intervention method according to claim 1, wherein the set of decision action training samples comprises correct decision action samples and wrong decision action samples.
3. The decision intervention method of claim 2, wherein the step of synchronously acquiring the electroencephalogram signals while the operator gazes at the automated system to perform the decision action is performed by immediately extracting the electroencephalogram signals of the operator from the automated system after performing the decision action through a sliding window of a preset duration.
4. The decision intervention method of claim 3, wherein the electroencephalogram features include time domain features, spatial domain features, and frequency domain features.
5. The decision intervention method of claim 4, wherein the ESSP method is used to extract electroencephalogram features of the electroencephalogram signal within the sliding window.
6. A decision intervention method according to claim 1, wherein the detection model is a SVM support vector machine model.
7. The decision intervention method of claim 1, wherein the online real-time decision intervention process comprises,
1) collecting electroencephalogram signals of an operator watching the execution of the automatic system in a sliding window with preset duration in the process of executing the automatic system in real time;
2) extracting the electroencephalogram characteristics of the electroencephalogram signals, inputting an error decision action-electroencephalogram characteristic model, and detecting error decision actions;
3) if no error decision action exists in the detection result, returning to 1); if the detection result has a wrong decision action, outputting a decision intervention instruction to the automatic system;
4) and the automatic system receives the decision intervention instruction and intervenes the wrong decision action.
8. Decision intervention method according to any of the claims 3-5 or 7, wherein the preset duration of the sliding window is set according to the speed of the human brain's reaction to the emergency.
9. The decision intervention method of claim 8, wherein the sliding window has a preset duration in a range of 100ms to 500 ms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811204037.0A CN109460145B (en) | 2018-10-16 | 2018-10-16 | Automatic system decision rapid intervention method based on brain recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811204037.0A CN109460145B (en) | 2018-10-16 | 2018-10-16 | Automatic system decision rapid intervention method based on brain recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109460145A CN109460145A (en) | 2019-03-12 |
| CN109460145B true CN109460145B (en) | 2021-09-07 |
Family
ID=65607729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811204037.0A Active CN109460145B (en) | 2018-10-16 | 2018-10-16 | Automatic system decision rapid intervention method based on brain recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109460145B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2231665A (en) * | 1989-04-26 | 1990-11-21 | Philip Chidi Njemanze | An artificial intelligent system for human error prevention in automated machine systems |
| KR20130097300A (en) * | 2012-02-24 | 2013-09-03 | 김웅식 | Method and system for phased recognition of brain waves |
| CN104216515A (en) * | 2014-07-25 | 2014-12-17 | 北京机械设备研究所 | Manned spacecraft noncontact operating and control method based on brain-computer interface |
| CN106175757A (en) * | 2016-07-11 | 2016-12-07 | 温州大学 | Behaviour decision making prognoses system based on brain wave |
| CN108236464A (en) * | 2017-12-29 | 2018-07-03 | 重庆邮电大学 | Feature extracting method and its Detection and Extraction system based on EEG signals |
| CN108491071A (en) * | 2018-03-05 | 2018-09-04 | 东南大学 | A kind of brain control vehicle Compliance control method based on fuzzy control |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015198309A1 (en) * | 2014-06-23 | 2015-12-30 | Hochman Eldad Izhak | Detection of human-machine interaction errors |
-
2018
- 2018-10-16 CN CN201811204037.0A patent/CN109460145B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2231665A (en) * | 1989-04-26 | 1990-11-21 | Philip Chidi Njemanze | An artificial intelligent system for human error prevention in automated machine systems |
| KR20130097300A (en) * | 2012-02-24 | 2013-09-03 | 김웅식 | Method and system for phased recognition of brain waves |
| CN104216515A (en) * | 2014-07-25 | 2014-12-17 | 北京机械设备研究所 | Manned spacecraft noncontact operating and control method based on brain-computer interface |
| CN106175757A (en) * | 2016-07-11 | 2016-12-07 | 温州大学 | Behaviour decision making prognoses system based on brain wave |
| CN108236464A (en) * | 2017-12-29 | 2018-07-03 | 重庆邮电大学 | Feature extracting method and its Detection and Extraction system based on EEG signals |
| CN108491071A (en) * | 2018-03-05 | 2018-09-04 | 东南大学 | A kind of brain control vehicle Compliance control method based on fuzzy control |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109460145A (en) | 2019-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109473119B (en) | Acoustic target event monitoring method | |
| CN114532993B (en) | Automatic detection method for electroencephalogram high-frequency oscillation signals of epileptic patients | |
| CN105786182B (en) | A kind of method and device based on gesture control surrounding devices | |
| CN107370823B (en) | Data acquisition and remote control method, device and computer readable storage medium | |
| CN108932581A (en) | The autonomous cognitive method and system of more physics domain information fusions | |
| CN110584597A (en) | Multi-channel electroencephalogram signal monitoring method based on time-space convolutional neural network and application | |
| CN113345399A (en) | Method for monitoring sound of machine equipment in strong noise environment | |
| CN109460145B (en) | Automatic system decision rapid intervention method based on brain recognition | |
| CN109062951A (en) | Based on conversation process abstracting method, equipment and the storage medium for being intended to analysis and dialogue cluster | |
| CN115344122A (en) | Sound wave non-invasive brain-computer interface and control method | |
| CN114863311A (en) | Automatic tracking method and system for inspection target of transformer substation robot | |
| CN109924974B (en) | EcoG-based epileptic neural signal decoding method and cloud decoding system | |
| CN109063652B (en) | Signal processing method, system and computer storage medium | |
| CN104991245A (en) | Unmanned aerial vehicle early warning apparatus and early warning method thereof | |
| CN112936259B (en) | Man-machine cooperation method suitable for underwater robot | |
| JPWO2019130840A1 (en) | Signal processing equipment, analysis system, signal processing method and signal processing program | |
| CN114609453A (en) | Robot joint current anomaly detection method and device based on statistical process control | |
| CN109325402B (en) | Signal processing method, system and computer storage medium | |
| CN205003281U (en) | Unmanned vehicles early warning device | |
| Wang et al. | EEG recognition through time-varying vector autoregressive model | |
| CN111730604A (en) | Mechanical clamping jaw control method and device based on human body electromyographic signals and electronic equipment | |
| Gjoreski | A method for combining classical and deep machine learning for mobile health and behavior monitoring | |
| CN116360598B (en) | Motor imagery motion control method and system based on brain waves | |
| CN114027855B (en) | Electroencephalogram signal decoding method and system for recognizing head movement intention | |
| Zimenko Konstantin et al. | Analysis and processing of electromyogram signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |