CN110244854A - A kind of artificial intelligence approach of multi-class eeg data identification - Google Patents
A kind of artificial intelligence approach of multi-class eeg data identification Download PDFInfo
- Publication number
- CN110244854A CN110244854A CN201910638019.1A CN201910638019A CN110244854A CN 110244854 A CN110244854 A CN 110244854A CN 201910638019 A CN201910638019 A CN 201910638019A CN 110244854 A CN110244854 A CN 110244854A
- Authority
- CN
- China
- Prior art keywords
- eeg
- data
- embedded device
- eeg data
- lstm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Abstract
The invention discloses a kind of artificial intelligence approaches of multi-class eeg data identification, belong to eeg data technical field.Equipment is acquired by the EEG data under low cost and collects the eeg data of user, and is saved it in embedded device TX2.It proposes to customize design, enhances the signal interpretation of storehouse LSTM by attention mechanism, capture multiple features and realization accurately identifies individual EEG signal.Develop a low cost, in real time and end to end BCI system, can in embedded robot platform operating energy loss DeepBrain model and algorithm, a plurality of types of home tasks signals are executed according to the electroencephalogram of user and are inputted.The invention has the benefit that the temporal correlation between solving the problems, such as time series data, while a kind of coding method is proposed to realize multi-class eeg data identification, and then apply it to multiclass scene, cost is relatively low；The present invention can effectively improve discrimination up to 97.5%.
Description
Technical field
The present invention relates to a kind of artificial intelligence approaches of multi-class eeg data identification, belong to eeg data technical field.
Background technique
Torsion free modules (BCI) are configured as the emerging subdomains of human-computer interaction (HCI), achieve in recent years great
Progress.It is intended in general, BCI system collects electroencephalogram (EEG) signal using wearable device and is construed as various users.It should
Technology is applied to many BCI systems in different scenes.However, these existing systems have the shortcomings that one it is common, i.e., greatly
Majority be experimental prototype or they be for organization user (such as hospital and government) exploitation, therefore hardware cost is quite high
Expensive, which prevent the widely used application scenarios in people's daily life.The price that EEG acquires equipment is different, such as
799.00 dollars of EEG equipment price of the channel EMOTIVEPOC+14 movement, and about 99 dollars of Brainlink equipment price.It is this
EEG signal acquires equipment and is usually limited by price the use of ordinary user.Another of the robot of BCI control is specific
The disadvantage is that they only allow user to execute single movement.For example, using mobile robot before the direction of signal of AD HOC.
Since all there are many tracks inside and outside human body for mood, Emotion identification model is constructed using various methods,
Such as facial expression, sound etc..In these methods, the method based on EEG is measured as the promising side for Emotion identification
Method.Many discoveries of Neuscience support EEG to allow directly to assess " inherence " state of user.However, big in these researchs
It is most to have much relations with wet electrode (some that there is dozens of electrode).In addition to place electrode time cost and high price it
Outside, incoherent channel may be mixed into the noise in system, this can seriously affect the performance of system.Call user in the community HCI
Friendly usage is interactive to carry out effective brain-computer.It, can be with the fast development of wearable device and dry electrode technology
Develop wearable EEG application apparatus.For example, wearing this if equipment detects that he or she is in certain emotional state
The language of kind equipment or disabled person can show his or her mood to service robot.It is well known that being easy to pacify in HCI
The Emotion identification EEG equipment of dress is popular.In order to realize that this idea, the present invention are used for using relatively few number of electrode
EEG acquisition, EEG acquisition device is Brainlink, and in forehead position tool, there are two the dry electrodes of Noninvasive.When user is in
When different state (or loosening), Brainlink will show the breath light of different colours.Then by above system to collection
EEG data carries out Emotion identification.When neural network is for that can pass through manual construction sliding window when handling EEG signal data
Processing EEG time series data is come the connection between the data before and after completing.Deep neural network has been used to EEG number
According to classification.Long short-term memory (LSTM) is Recognition with Recurrent Neural Network, equipped with control to the special gate of the access of memory cell
Mechanism.Shot and long term memory cell under door control mechanism allows information to pass through unmodifiedly in many time steps.Due to
Door can prevent the rest part of network from modify the content of memory cell, therefore the reservation of LSTM network in multiple time steps
Signal and propagate mistake time it is more much longer than common RNN.By independently being read from memory cell, in write-in and erasing
Hold, door can also be trained to handle the selection of input signal and the carelessness of other parts.LSTM is stacked by along depth dimensions
LSTM unit connection composition.It is conducive to store and generate the mode of longer distance.And introduce attention selection device then and be in order to
It solves the problems, such as to stack the considerations of memory that innately there is of LSTM is hypodynamic and user customizes.
In current main research method, more researchs mainly handle short-term time related sequence data, some of
Method collects the single channel EEG from frontal cortex merely with device, is only limitted to calculate moos index.And other method sides
Overweight the more multichannel electroencephalogram from brain cortex and more advanced equipment.Although advanced equipment can be brought more diversified
EEG data, it is well known that each additional electrode requires to spend many money.Therefore, it may finally acquire and set in data
Standby upper cost several thousand dollars, this is unfavorable to one complete feasible BCI system of design.Few researchs attempt to establish one
It is feasible, high-precision, the Emotion identification system based on electroencephalogram that is user friendly and being easy to deployment.
Summary of the invention
The technical problems to be solved by the invention are as follows: it is less for existing classification scene, a kind of multi-class brain electricity number is provided
According to the artificial intelligence approach of identification.
The technical scheme adopted by the invention is as follows: including EEG helmet bluetooth-capable, embedded device TX2,
Robot carrier, comprising the following steps:
Step 1, data collection；EEG helmet sends the order of transmission data by bluetooth module to embedded device TX2,
Then wait embedded device TX2 confirmation is received to reply message, after successful reception, brain electricity number that EEG helmet is collected into
According to will be shown on the remote terminal of embedded device TX2, and plug-in can be to the brain for being stored in embedded device TX2
Electric data are listened to, after then eeg data that is untreated in embedded device TX2, being newly collected into is pre-processed,
Be passed in the good DeepBrain model of precondition, the model can the intention to user identified in real time, finally will know
Other result is stored in embedded device TX2；
Step 2, it feeds back；Pass through socket programming or the connection of serial communication between embedded device TX2 and robot carrier
Robot carrier is fed back to when mode is by the fructufy of classification；
Step 3, human-computer interaction；It is interacted between embedded device TX2 and robot carrier using client server mode,
By embedded device TX2 sending action instruction to robot carrier, robot carrier completes corresponding movement after receiving instruction.
The embedded device TX2 includes: with robot carrier communication steps
Step 1, bluetooth module bright light is normal when initialization；
Step 2, when bluetooth module is attempted to establish connection with embedded device TX2, remote terminal screen can show connection procedure,
It can be shown after successful connection " successful connection ", enter data transmission state immediately；If embedded device TX2 not with bluetooth module
It is successfully established connection, then can show relevant error information on the screen of remote terminal, user is at this moment needed to reaffirm bluetooth mould
Whether block line is correct and re-executing transmission order later；Step 3, the data that embedded device TX2 is received will be with
Metric form shows that numberical range differs in size for 0~100 or so, excessive for numerical value on a terminal screen in real time
Or the exceptional value being negative, the processing mode being averaged, which is added, herein by the normal value before and after exceptional value is handled,
And exceptional value is replaced with the average value；
Step 4, every 180 decimal datas are preset and represent an eeg data file, EEG helmet can be deposited after collecting
It stores up in embedded device TX2, the plug-in in embedded device TX2 is immediately the eeg data being newly collected under file
It is loaded among preparatory trained model, model can carry out class prediction classification to data and provide final classification results；
Step 5, classification results are fed back into robot carrier in real time, robot carrier can be made according to different classification results
Corresponding movement；
Step 6, step 1 is repeated to 5, until not new eeg data is added in embedded device TX2.The when ordinal number
It is specifically included that according to used DeepBrain model
Wherein fi, ff, foAnd fmInput gate is respectively indicated, door is forgotten, out gate and input modulation door, ⊙ indicate to multiply (square by element
Battle array multiplication)；σ is logistic function, for a S type function and the value x brought into formula is output in [0,1] section；
Tanh is tangent function；cijIndicate the state (memory) in i-th layer in j-th of LSTM unit；
It is expressed as follows operation:
Wherein W, W' indicate corresponding weight, and b indicates deviation；
Then, using backpropagation time (BPTT) algorithm training pattern.
Further, the EEG helmet signal identification for customizing user is enhanced using attentional selection device and stacks LSTM
Expression ability；
Attentional selection device receives final LSTM layers as the attention weight W' by operation measurementattAre as follows:
And calculate normalized attention weight:
W′attFor Watt=softmax (W 'att)；
Dropout layers are a kind of Regularization Techniques, can be embodied in formula are as follows:
Wherein symbol D indicates Dropout operator, indicates to set zero for the random subset of bracket intrinsic parameter；This makes LSTM
Unit more steady can execute intermediate operations and the popularization for improving EEG signal identification model, finally
Softmax layers for classifying to four kinds of EEG helmet signal modes, including relaxation state signal mode, relax to specially
Note condition signal pattern is absorbed in relaxation state signal mode, is absorbed in condition signal pattern.
The advantageous effect of present invention is that: the temporal correlation that the present invention can be very good to solve between time series data is asked
Topic, while a kind of coding method is proposed to realize multi-class eeg data identification, and then applies it to multiclass scene, and
Cost is relatively low for device therefor of the present invention.In terms of discrimination, compared to existing eeg data recognition methods, the present invention can be mentioned effectively
High discrimination is up to 97.5%.
Detailed description of the invention
Fig. 1 system architecture diagram.
Fig. 2 system flow chart.
The internal structure of Fig. 3 DeepBrain.
Fig. 4 user-robot interactive system.
Four kinds of EEG signal data of Fig. 5 180s acquisition.
Fig. 6 samples 60 four kinds of EEG signal data out of 180s.
The indices precision of different classifications model compares on Fig. 7 data set.
Precision and training time of Fig. 8 DeepBrain under different training data ratios.
Specific embodiment
1 to 8 pair of the preferred embodiment of the present invention is described further with reference to the accompanying drawing, and it is low that the present invention needs user to wear
The ready-made EEG helmet of cost carries out eeg data collection: carrying out coded treatment to eeg data first and collects, EEG
It is complete that the data being collected into when helmet presentation red light are all shown as the data being collected into when 1, EEG helmet presentation blue lamp
Portion is expressed as 0, within a specified time, so that it may and four class data are collected into, absorbed state a) is within a specified time always maintained at, it will
The eeg data being collected into the case of this is encoded to 11.B) within a specified time first it is absorbed in and loosens afterwards, will be collected into the case of this
Data encoding be 10.C) within a specified time first loosen and be absorbed in afterwards, be 01 by the data encoding being collected into the case of this.D) exist
It is always maintained at relaxation state in specified time, is 00 by the data encoding being collected into the case of this.The data being collected into are divided
For training set and test set and use it to trained model.Due to using different robot carriers, code platform makes
The tall and handsome embedded device Jetson embedded device TX2 up to company, the Linux behaviour of installation Ubuntu16.04 release
Make system.The Open Framework TensorFlow that the training and deployment of model are developed using Google, the side compiled by source code
TensorFlow is installed on embedded device TX2 by formula.After training model, save it in local, convenient for the later period plus
It carries and predicts that classification is used, be the part of data collection and code platform above.Robot carrier is first tried out
The NAO robot of AldebaranRobotics company development & production is mainly in view of NAO robot technology maturation, just when initial
It is used in exploitation.Later period by way of socket network programming when being in communication with, it was found that there is no pay attention to before
The some problems arrived, and these problems may influence whether being widely used for system.It is avoided firstly, robot NAO does not reach
The desired effect of barrier.Secondly, the communication connection based on socket network programming may be influenced by external factor, such as net
Network interference and malicious attack.Finally, robot NAO is slowly moved.Based on this, the customization version of wheeled robot equipment is developed
This.It about the customized version of wheeled robot equipment, is attached herein using serial communication, efficiently solves local area network ring
Two main problems in border.Robot carrier can carry out corresponding operation according to the classification results that code platform provides.
After three parts linkage is got up, the multi-class eeg data identifying system specific implementation based on embedded artificial intelligence
Steps are as follows: step 1: eeg data collection phase；EEG helmet only collects eeg data.When before collection starts, remotely
It can be shown on terminal screen and attempt whether connection succeeds three times, if screen shows success, then it represents that be successfully connected and will enter
Data collection state, otherwise, user need to check whether wiring is correct, and in a period of time restarting equipment later.Then, before
The data being newly collected into are loaded among preparatory trained model by the plug-in just finished writing in embedded device TX2, and
Continue waiting for the arrival of new data.
Step 2: model training stage；Storehouse LSTM based on attention selection device is combined with eeg data, is infused
The power storehouse LSTM that anticipates has extraordinary prediction classifying quality for prolonged voice sequence data, when being used in brain electricity
Between in data, to which conceivable result can be obtained.
EEG helmet provides the decimal data that reflection user's brain is intended to time-out.Although data acquisition equipment is very
Stablize, but its occasional is influenced by external environment, leads to individual abnormal datas.In light of this situation, by providing threshold value
To abandon the data higher than threshold value.Threshold value is the long-term overview setup of basis.Meanwhile calculating previous and the last one moment
Average value.And use this average value as the substitution for abandoning data.
As it was noted above, by correct coding is carried out to the two of collection kinds of data, to realize expected four kinds of classifying qualities.
The coding method of EEG data label is then single heat coding come the label to EEG data using a kind of typical coding method
It is handled.The mutation of model caused by model Logarithmic calculation in training process can be effectively avoided using this coding method to ask
Topic.In the initial stage of the more sort operations of EEG data, the result tested is always in lower numberical range.Normally
Classifiable data characteristics point is as shown in Figure 2.Based on this, secondary separation will be carried out to composite character.Designed function should expire
Sufficient the following conditions: F (u, v)！=F (v, u) and to meet F (u) and F (v) both as u==v represented by region should
It is isolated.Initial data can combine with attention storehouse LSTM well after above-mentioned processing.Wherein U, V
It is the variable in separate function F, only indicates the feature for needing to carry out secondary separation, is i.e. separate function F is needed mixed feature
Secondary separation operation is carried out, U is set, operation when V is model preprocessing is more conducive to the training and classification of model hereinafter.
In order to learn the meaning of user's signal of intent, i.e. one-dimensional vector (collecting a time point).Single input EEG is believed
It number is expressed asWherein K is the dimension in EEG original signal, is set as 30.Then, by EiAs model support
The input of structure is to carry out temporal characteristics study.Finally, according to the temporal characteristics X learntt, provide the result of classification.Fig. 2 is to mention
The flow chart of method out.The original EEG helmet data of input are single sample vectors, first using several full connections
Layer is used as hidden layer, then outputs it value and is input to the LSTM unit with attention selection device.In addition, arrow indicates in figure
LSTM layers of internal structure, htIt is the output valve of LSTM unit in t-th of time step field, CtRepresent LSTM in t-th of time step
The state value of unit.
For the challenge of time series data processing, the feature of original time series is obtained using down-sampling technology first
Subsequence.Down-sampling reduces the complexity of original time series, becomes easier to mode of learning.Meanwhile in order to accelerate mould
The convergence rate of type is normalized to normalized temporal sequence data using min-max, this is the linear transformation of initial data.
Value after conversion is mapped to section [0,1].
Part is handled in temporal characteristics, LSTM structure has been proven that its powerful temporal characteristics extractability.LSTM can
To explore the time dependence of feature by the internal state of network, this allows it to show time trend behavior.LSTM unit is logical
Cross the input for introducing door control mechanism to control data, storage and output.As shown in Fig. 2, LSTM unit receives previous time step
The output of LSTM internal element and the input of current time stepping sample.Use the attention LSTM model comprising four components:
One input layer, several hidden layers, attention selection device and an output layer.LSTM unit is located in hidden layer, in Fig. 2
Rectangle shown in.If a batch input EEG data includes ns(referred to as batch size) EEG helmet sample, and always input number
It is [n according to following 3D shapes,30,1]。
The data in i-th layer are enabled to be expressed asWherein j
Indicate j-th of EEG sample, KiIndicate the dimension in i-th layer.Assuming that the weight between layer i and layer i+1 can be usedIt indicates, for example,Indicate the weight between the 2nd layer and the 3rd layer.Indicate i-th layer of biasing.
Calculating between i-th layer data and i+1 layer data can be expressed as
LSTM layers of calculating is as follows:
Wherein fi, ff, foAnd fmInput gate is respectively indicated, door is forgotten, out gate and input modulation door, ⊙ indicate to multiply (square by element
Battle array multiplication).σ is Logistic function, and tanh is tangent function, cijIndicate that the state in i-th layer in j-th of LSTM unit (is deposited
Reservoir).This is the most important part for detecting time series correlation between EEG data sample.It indicates such as
Lower operation:
Wherein W, W' and b indicate corresponding weight and deviation.Then, mould is trained using backpropagation time (BPTT) algorithm
Type.Finally, obtaining prediction result.The value of learning rate is set as 10-4, batch size value is 64.
Other than the stacking LSTM structure that the time dependence of processing EEG signal is proposed, different users may have
There are various EEG signal modes, and needs to be extended design to solve the memory deficiency and sophisticated category of stacking LSTM
The problem of.In order to which the solution of customization is presented, the present invention passes through the enhancing storehouse LSTM of attention mechanism in DeepBrain
Framework.Enhancing storehouse LSTM based on attention can promote the EEG data feature between DeepBrain study different people,
EEG signal individual is accurately identified to make system of the invention realize.Enhancing based on attention stacks the whole of LSTM
Body framework is as shown in Figure 3.Usage history timestamp predicts current time stamp.Embeding layer is enhancing of the first layer based on attention
Stack LSTM.Then, long dependence is captured using LSTM is stacked.In order to customize the identification of user's EEG signal, attention is used
Selector stacks the expressive faculty of LSTM to enhance.Attentional selection device receives final LSTM layers of state value as attention weightAnd attention weight is normalized to W in attention selection deviceatt=
softmax(W′att).Concatenation layers then for connecting the output of storehouse LSTM and attentional selection device.Dropout layers are
A kind of Regularization Technique, for improving the popularization of EEG signal identification model.Final softmax layer is then for four kinds
EEG signal mode is classified.The detailed process of algorithm provides in Fig. 2, Fig. 3.
Step 3: brain-man-controlled mobile robot carrier stage；The key point of brain control system is the EEG letter for accurately identifying user
Number.EEG data is sensitive to brain background activity and environmental factor, and the present invention, which passes through, applies suitable feature representation and classification
To identify that the mankind are intended to.
User-robot interactive system is as shown in Figure 4.The EEG signal that user is intended to is by wearable EEG acquisition equipment hair
It is sent to real-time system.Embedded A I (embedded device TX2) plate uses trained DeepBrain model identification user's in advance
It is intended to, action command is then sent to robot carrier and executes corresponding movement.Between Embedded A I plate and robot carrier
It is attached using the mode of socket network programming or serial communication.
Based on the accuracy of DeepBrain EEG signal classification, and then develop a brain control system, such as Fig. 4
It is shown.Complete online BCI system is by several big component parts: the deep learning model of customization, based on tall and handsome embedding up to Jetson
Enter the Embedded A I real-time system of formula equipment TX2, the inexpensive EEG acquisition equipment of robot carrier and user's wearing.It is deep
Degree learning model is to have been off trained, this facilitates the intention for real-time and accurately identifying user, which is brain control
The core of system processed.Real-time BCI system includes four parts: EEG acquisition equipment, user, Embedded A I plate are (embedded to set
Standby TX2), robot carrier and bluetooth module.
Consider that two different robot carriers are interacted with user herein when initial.For robot carrier NAO, lead to
It crosses LAN and connects it with client server mode.The practical provided according to the target group that robot carrier is serviced
It is recommended that having found the limitation of some NAO itself herein.Firstly, robot NAO does not reach expected in terms of avoiding obstacles
Effect.Secondly, the connection type with robot NAO may be influenced by external factor, such as network interferences and malice are attacked
It hits.Third, robot NAO are mobile slow.Based on this, the present invention has developed the customized version of wheeled robot equipment.For fixed
The robot carrier of plate-making originally, is attached using serial communication and embedded device TX2, is efficiently solved in lan environment
Two main problems.
Experimental result: the deep learning model of design is assessed using the local EEG signal data set of collection.To experiment number
It is pre-processed according to collection, and is compared with other methods.Specifically, using the brain electric data collecting equipment of low cost.Although
The equipment can resist the noise of multiple signal paths from the user, but its occasional is by external environments such as weather and sound
It influences.Therefore, it sometimes appear that exceptional data point in data set.Therefore, in order to improve the Stability and veracity of result, having must
Carry out data prediction.Unripe EEG data is collected in 180 seconds, as shown in Figure 5.A), b), c), d) four kinds of states
Illustrate referring to [0018] section；It can be seen that data are very intensive and seem disorderly and unsystematic.If without data prediction appropriate
Method, training pattern and prediction are brought huge challenge by it.In Fig. 6, inexpensive EEG data collecting device provides reflection user
Brain emotional state decimal representation fractional value, be the digital representation of above-mentioned EEG mode, the hits in 180s is
60 times.When the EEG signal value of user is very high, show that brain is most likely in Beta or Gamma mode.On the contrary, when user's
When being worth lower, show that brain is most likely in Delta or Theta mode.It is low for the Alpha mode of the medium consciousness of user
The EEG data acquisition equipment of cost will provide corresponding numerical value according to different user'ss (sex).
Using five indexs to fully assess methodical performance: precision (Accuracy), accuracy rate
(Precision), recall rate (Recall), F1Score, area under the curve (AUC).These indexs have been widely used for assessment machine
Device learning algorithm.The details of index are as follows:
Wherein really (TP) indicates that the positive sample number being positive by model prediction, false positive (FP) indicate the negative sample being positive by model prediction
This number, very negative (TN) are indicated by the negative sample number that model prediction is negative and the positive sample that vacation negative (FN) expression is negative by model prediction
Number.
In general, accuracy is used to judge the model that target is classification, simultaneously because purpose is identification EEG data class
Not, thus accuracy rate and recall rate be also assessment models important indicator.Accuracy rate is mainly used for judging whether classifier can be just
Classification results really are obtained, that is, are concentrated mainly on identification exceptional sample.In addition, recall rate mainly assesses whether classifier can be known
Not all exceptional samples.FβScore is the combination of the first two index, if β is less than 1, then it represents that recall rate is even more important.On the contrary,
Influence of the accuracy rate to model quality evaluation is bigger.F1Score is used as the General Introduction of the performance about algorithm.
In performance study, it is compared to DeepBrain and its other party to illustrate the efficiency of DeepBrain.The present invention
The method of design is a kind of mixed model, it carries out feature learning using LSTM, carries out intention knowledge using softmax classifier
Not.In an experiment, EEG data is randomly divided into two parts: training dataset and test data set.By DeepBrain and SVM,
MLP, LSTM, storehouse LSTM are compared.In addition, listing key parameter here: multilayer perceptron (MLP) (hides node layer
For 30), and the LSTM with 32 units.
The result shows that DeepBrain has higher precision than other methods, while also than other deep learning models
(such as MLP or LSTM) is performed better than when running.In addition, compared with the existing brain electricity sort research for being absorbed in binary classification,
DeepBrain is run under multiclass scene, has still reached high-caliber accuracy.The experiment proves that DeepBrain is in local
0.975 more niceties of grading are reached under data set.In order to illustrate the robust feature model for the raw EEG data that the present invention designs
Advantage, experimental result is as shown in Figure 7, it can be seen that DeepBrain is better than respectively in terms of nicety of grading (Accuracy)
MLP, LSTM, SVM and storehouse LSTM.Next influence of the experiment for trained different number data to model accuracy.Knot
As shown in figure 8, with more training datas, precision can also improve fruit.Time needed for training pattern, with training data
Linear change is presented in ratio increase.Finally, currently it is also known that when using different classifiers, suitably reducing learning rate can be helped
Help raising accuracy.
Claims (4)
1. a kind of artificial intelligence approach of multi-class eeg data identification, including EEG helmet bluetooth-capable, insertion
Formula equipment TX2, robot carrier, which comprises the following steps:
Step 1, data collection；EEG helmet sends the order of transmission data by bluetooth module to embedded device TX2,
Then wait embedded device TX2 confirmation is received to reply message, after successful reception, brain electricity number that EEG helmet is collected into
According to will be shown on the remote terminal of embedded device TX2, and plug-in can be to the brain for being stored in embedded device TX2
Electric data are listened to, after then eeg data that is untreated in embedded device TX2, being newly collected into is pre-processed,
Be passed in the good DeepBrain model of precondition, the model can the intention to user identified in real time, finally will know
Other result is stored in embedded device TX2；
Step 2, it feeds back；Pass through socket programming or the connection of serial communication between embedded device TX2 and robot carrier
Robot carrier is fed back to when mode is by the fructufy of classification；
Step 3, human-computer interaction；It is interacted between embedded device TX2 and robot carrier using client server mode,
By embedded device TX2 sending action instruction to robot carrier, robot carrier completes corresponding movement after receiving instruction.
2. a kind of artificial intelligence approach of multi-class eeg data identification according to claim 1, which is characterized in that described
Embedded device TX2 includes: with robot carrier communication steps
Step 1, bluetooth module bright light is normal when initialization；
Step 2, when bluetooth module is attempted to establish connection with embedded device TX2, remote terminal screen can show connection procedure,
It can be shown after successful connection " successful connection ", enter data transmission state immediately；If embedded device TX2 not with bluetooth module
It is successfully established connection, then can show relevant error information on the screen of remote terminal, user is at this moment needed to reaffirm bluetooth mould
Whether block line is correct and re-executing transmission order later；
Step 3, the data that embedded device TX2 is received will show on a terminal screen in real time in the form of metric,
Numberical range differs in size for 0~100 or so, for the exceptional value that numerical value is excessive or is negative, herein by before and after exceptional value
Normal value be added the processing mode that is averaged and handled, and exceptional value is replaced with the average value；
Step 4, every 180 decimal datas are preset and represent an eeg data file, EEG helmet can be deposited after collecting
It stores up in embedded device TX2, the plug-in in embedded device TX2 is immediately the eeg data being newly collected under file
It is loaded among preparatory trained model, model can carry out class prediction classification to data and provide final classification results；
Step 5, classification results are fed back into robot carrier in real time, robot carrier can be made according to different classification results
Corresponding movement.
Step 6, step 1 is repeated to 5, until not new eeg data is added in embedded device TX2.
3. a kind of artificial intelligence approach of multi-class eeg data identification according to claim 1, which is characterized in that timing
DeepBrain model used by data specifically includes that
cij=ff⊙ci(j-1)+fi⊙fm
Wherein fi, ff, foAnd fmInput gate is respectively indicated, door is forgotten, out gate and input modulation door, ⊙ indicate to multiply (square by element
Battle array multiplication)；σ is logistic function, for a S type function and the value x brought into formula is output in [0,1] section；
Tanh is tangent function；cijIndicate the state (memory) in i-th layer in j-th of LSTM unit；
It is expressed as follows operation:
Wherein W, W' indicate corresponding weight, and b indicates deviation；
Then, using backpropagation time (BPTT) algorithm training pattern.
4. a kind of artificial intelligence approach of multi-class eeg data identification according to claim 3, which is characterized in that
The EEG helmet signal identification for further customizing user, enhances the expression for stacking LSTM using attentional selection device
Ability；
Attentional selection device receives final LSTM layers as the attention weight W' by operation measurementattAre as follows:
And calculate normalized attention weight:
W'attFor Watt=softmax (W'att)；
Dropout layers are a kind of Regularization Techniques, can be embodied in formula are as follows:
Wherein symbol D indicates Dropout operator, indicates to set zero for the random subset of bracket intrinsic parameter；This makes LSTM
Unit more steady can execute intermediate operations and the popularization for improving EEG signal identification model, finally
Softmax layers for classifying to four kinds of EEG helmet signal modes, including relaxation state signal mode, relax to specially
Note condition signal pattern is absorbed in relaxation state signal mode, is absorbed in condition signal pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910638019.1A CN110244854A (en) | 2019-07-16 | 2019-07-16 | A kind of artificial intelligence approach of multi-class eeg data identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910638019.1A CN110244854A (en) | 2019-07-16 | 2019-07-16 | A kind of artificial intelligence approach of multi-class eeg data identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110244854A true CN110244854A (en) | 2019-09-17 |
Family
ID=67892384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910638019.1A Pending CN110244854A (en) | 2019-07-16 | 2019-07-16 | A kind of artificial intelligence approach of multi-class eeg data identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110244854A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610168A (en) * | 2019-09-20 | 2019-12-24 | 合肥工业大学 | Electroencephalogram emotion recognition method based on attention mechanism |
| CN111134666A (en) * | 2020-01-09 | 2020-05-12 | 中国科学院软件研究所 | Emotion recognition method of multi-channel electroencephalogram data and electronic device |
| CN111543988A (en) * | 2020-05-25 | 2020-08-18 | 五邑大学 | Adaptive cognitive activity recognition method and device and storage medium |
| CN114115546A (en) * | 2022-01-27 | 2022-03-01 | 湖南大学 | Electroencephalogram data element learning and human-computer intelligent interaction system for multi-robot control |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105559777A (en) * | 2016-03-17 | 2016-05-11 | 北京工业大学 | Electroencephalographic identification method based on wavelet packet and LSTM-type RNN neural network |
| CN108304917A (en) * | 2018-01-17 | 2018-07-20 | 华南理工大学 | A kind of P300 signal detecting methods based on LSTM networks |
| CN108415554A (en) * | 2018-01-18 | 2018-08-17 | 大连理工大学 | A kind of brain man-controlled mobile robot system and its implementation based on P300 |
| CN108542386A (en) * | 2018-04-23 | 2018-09-18 | 长沙学院 | A kind of sleep state detection method and system based on single channel EEG signal |
| CN109124625A (en) * | 2018-09-04 | 2019-01-04 | 大连理工大学 | A kind of driver fatigue state horizontal mipmap method |
| CN109472194A (en) * | 2018-09-26 | 2019-03-15 | 重庆邮电大学 | A kind of Mental imagery EEG signals characteristic recognition method based on CBLSTM algorithm model |
-
2019
- 2019-07-16 CN CN201910638019.1A patent/CN110244854A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105559777A (en) * | 2016-03-17 | 2016-05-11 | 北京工业大学 | Electroencephalographic identification method based on wavelet packet and LSTM-type RNN neural network |
| CN108304917A (en) * | 2018-01-17 | 2018-07-20 | 华南理工大学 | A kind of P300 signal detecting methods based on LSTM networks |
| CN108415554A (en) * | 2018-01-18 | 2018-08-17 | 大连理工大学 | A kind of brain man-controlled mobile robot system and its implementation based on P300 |
| CN108542386A (en) * | 2018-04-23 | 2018-09-18 | 长沙学院 | A kind of sleep state detection method and system based on single channel EEG signal |
| CN109124625A (en) * | 2018-09-04 | 2019-01-04 | 大连理工大学 | A kind of driver fatigue state horizontal mipmap method |
| CN109472194A (en) * | 2018-09-26 | 2019-03-15 | 重庆邮电大学 | A kind of Mental imagery EEG signals characteristic recognition method based on CBLSTM algorithm model |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610168A (en) * | 2019-09-20 | 2019-12-24 | 合肥工业大学 | Electroencephalogram emotion recognition method based on attention mechanism |
| CN110610168B (en) * | 2019-09-20 | 2021-10-26 | 合肥工业大学 | Electroencephalogram emotion recognition method based on attention mechanism |
| CN111134666A (en) * | 2020-01-09 | 2020-05-12 | 中国科学院软件研究所 | Emotion recognition method of multi-channel electroencephalogram data and electronic device |
| CN111543988A (en) * | 2020-05-25 | 2020-08-18 | 五邑大学 | Adaptive cognitive activity recognition method and device and storage medium |
| CN114115546A (en) * | 2022-01-27 | 2022-03-01 | 湖南大学 | Electroencephalogram data element learning and human-computer intelligent interaction system for multi-robot control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110244854A (en) | A kind of artificial intelligence approach of multi-class eeg data identification | |
| CN111652066B (en) | Medical behavior identification method based on multi-self-attention mechanism deep learning | |
| CN108764207B (en) | Face expression recognition method based on multitask convolutional neural network | |
| CN107516110B (en) | Medical question-answer semantic clustering method based on integrated convolutional coding | |
| Asim et al. | Context-aware human activity recognition (CAHAR) in-the-Wild using smartphone accelerometer | |
| CN108009521A (en) | Humanface image matching method, device, terminal and storage medium | |
| CN112800998B (en) | Multi-mode emotion recognition method and system integrating attention mechanism and DMCCA | |
| CN111639544A (en) | Expression recognition method based on multi-branch cross-connection convolutional neural network | |
| CN109190566A (en) | A kind of fusion local code and CNN model finger vein identification method | |
| CN103745235A (en) | Human face identification method, device and terminal device | |
| CN111582342B (en) | Image identification method, device, equipment and readable storage medium | |
| CN113723238B (en) | Face lightweight network model construction method and face recognition method | |
| CN108364662A (en) | Based on the pairs of speech-emotion recognition method and system for differentiating task | |
| Jiang et al. | Variational deep embedding: A generative approach to clustering | |
| EP3944138A1 (en) | Method and apparatus for image recognition | |
| Ali et al. | Keystroke biometric user verification using Hidden Markov Model | |
| Khan et al. | Untran: Recognizing unseen activities with unlabeled data using transfer learning | |
| CN107193378A (en) | Emotion decision maker and method based on brain wave machine learning | |
| CN112418059A (en) | Emotion recognition method and device, computer equipment and storage medium | |
| KR20200018868A (en) | Method for Adaptive EEG signal processing using reinforcement learning and System Using the same | |
| CN109359610A (en) | Construct method and system, the data characteristics classification method of CNN-GB model | |
| US20210327418A1 (en) | Artificial intelligence apparatus for performing voice control using voice extraction filter and method for the same | |
| Khowaja et al. | Facial expression recognition using two-tier classification and its application to smart home automation system | |
| CN110633689B (en) | Face recognition model based on semi-supervised attention network | |
| Barua et al. | A deep learning approach for detecting tic disorder using wireless channel information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190917 |
|
| WD01 | Invention patent application deemed withdrawn after publication |