CN108445751A - Merge multiple target SSVEP ideas control methods and the application of recurrence plot and deep learning - Google Patents
Merge multiple target SSVEP ideas control methods and the application of recurrence plot and deep learning Download PDFInfo
- Publication number
- CN108445751A CN108445751A CN201810169303.4A CN201810169303A CN108445751A CN 108445751 A CN108445751 A CN 108445751A CN 201810169303 A CN201810169303 A CN 201810169303A CN 108445751 A CN108445751 A CN 108445751A
- Authority
- CN
- China
- Prior art keywords
- ssvep
- eeg signals
- electrode
- stimulation
- multiple target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
Abstract
A kind of multiple target SSVEP ideas control methods and the application of fusion recurrence plot and deep learning increase phase information, design multiple target SSVEP brain electricity experiment stimulation interface；Obtain the n kind SSVEP EEG signals that each subject induces through n stimulation picture in 8 or more subjects；Obtain the recurrence plot of EEG signals of 8 or more the subjects under the induction of different stimulated picture；Label is set as sample for each recurrence plot, builds data set；Depth convolutional neural networks model structure and parameter are built and optimized, the depth convolutional neural networks model of the recurrence plot for the SSVEP EEG signals induced by different stimulated picture of effectively classifying is determined to；By new subject's SSVEP EEG signals after phase space reconfiguration, with the depth convolutional neural networks model after the input optimization of recurrence diagram form, multiple target SSVEP EEG signals Accurate classifications are realized；Idea control instruction is generated, realizes the control of multiple target idea.The present invention is suitble to apply in the complex control field of multiple target.
Description
Technical field
The present invention relates to a kind of controls of SSVEP ideas.More particularly to a kind of more mesh of fusion recurrence plot and deep learning
Mark SSVEP ideas control methods and application.
Background technology
Brain-computer interface (Brain-Computer Interface, BCI) is not dependent on to be made of nervus peripheralis and muscle
Output channel, be directly connected to the communication system of brain and computer and external equipment.Brain machine interface system have it is noninvasive acquisition,
Advantage easy to operate and unique temporal resolution advantage.Brain machine interface system is usually made of four modules：Brain telecommunications
Number acquisition module, EEG feature extraction module, EEG signals tagsort module and peripheral control module.Feature
Extraction module and tagsort module are the most crucial parts of entire brain-computer interface, and being exactly based on the two modules can be by brain
Electric signal is converted to the control signal that can be identified by external equipment., there are faint property, complexity and not in the characteristics of EEG signals
Stability has relatively high requirement due to these characteristics of EEG signals for normal form design and feature extraction and classifying.
In recent years, SSVEP-BCI has obtained coming the extensive concern in each fields such as autoepistemic, neural engineering, clinical rehabilitation.But
Current SSVEP-BCI still has certain limitation, and the SSVEP frequency of stimulation of human body sensitivity is less, and convenient divider method
Limitation display can realize the quantity of frequency, cause the stimulation quantity that can be realized very limited, cannot meet BCI practical applications
Demand.By introducing phase information, can not only be significantly greatly increased the degree of freedom of SSVEP, and increase adjacent target can area
Divide property, so as to shorten the target identification time.
Complex network and deep learning theory are theoretical as the data analysis in forward position, have had in many fields and have widely answered
With.Complex network is by determining that the company side between node and node can be abstracted the inside labyrinth of complication system well
Or operation mechanism, open new visual angle for the research of complicated nonlinear system.Deep learning, especially with convolutional Neural net
Deep learning based on network is theoretical, can make full use of the powerful computing capability of computer, effectively realizes that complication system is polygonal
The extraction and capture of degree, multi-level features, it is theoretical in conjunction with complex network and deep learning, it can develop and more stablize the more of maturation
Target SSVEP idea control systems.
Invention content
It is suitble to apply in the complex control field of multiple target the technical problem to be solved by the invention is to provide a kind of
Merge multiple target SSVEP ideas control methods and the application of recurrence plot and deep learning.
The technical solution adopted in the present invention is：A kind of multiple target SSVEP idea controls of fusion recurrence plot and deep learning
Preparation method implements the multiple target SSVEP brain electricity experiment for increasing phase information, and SSVEP brain electricity is completed by electroencephalogramsignal signal collection equipment
Signal acquisition, merges Phase-space Reconstruction and depth convolutional neural networks are theoretical, realizes multiple target SSVEP EEG signals
Accurate recognition, and then brain-computer interface is developed, it realizes the control of multiple degrees of freedom idea, specifically comprises the following steps：
1) increase phase information, design multiple target SSVEP brain electricity experiment stimulation interface, including the total n stimulation figure of a rows b row
Piece, in a line, b stimulation picture initial phase is identical but frequency of stimulation is different, and in same row, a stimulation picture flickers frequency
Rate is identical but initial phase is different；
2) it based on the brain electricity of multiple target SSVEP described in step 1) tests stimulation interface, is set by eeg signal acquisition
Back-up Huo Qu not each subject induces through n stimulation picture in 8 or more subjects n kind SSVEP EEG signals；
3) to the n kind SSVEP EEG signals for n stimulation picture of each subject, determination is wherein each respectively
The time-frequency combination of electrode signal is distributed, extraction energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface
Row select threshold value using Phase-space Reconstruction, determine recurrence plot respectively, obtain n and correspond to n kind SSVEP EEG signals respectively
Energy value sequence recurrence plot, obtain 8 or more subjects different stimulated picture induction under EEG signals recurrence plot；
4) the phase and frequency information and electrode information of stimulation picture are corresponded to by recurrence plot, each to obtain is passed
Return figure setting label as sample, builds data set；
5) depth convolutional neural networks model is built, determines depth convolutional neural networks model structure and initialization model
Parameter, for the sample of 80% tape label as training set, the sample of 20% tape label is defeated by training set as test set in data set
Enter depth convolutional neural networks model, the study for having supervision, optimization depth convolution god are carried out to sample data feature in training set
It is determined to by test set sample for verifying depth convolutional neural networks model for having through network architecture and parameter
The depth convolutional neural networks of the recurrence plot of the energy value sequence for the SSVEP EEG signals that effect classification is induced by different stimulated picture
Model；
6) the subject's SSVEP EEG signals that will newly obtain determine the time-frequency combination point of wherein each electrode signal respectively
Cloth, extraction energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface, after phase space reconfiguration, with
Recurrence diagram form input step 5) in depth convolutional neural networks model after optimization, realize that multiple target SSVEP EEG signals are accurate
Really classification；
7) by multiple target SSVEP eeg signal classifications as a result, the anti-stimulation picture for pushing away subject and watching attentively, is noted with subject
The stimulation picture regarded generates idea control instruction as foundation, realizes the control of multiple target idea.
Sample rate when obtaining SSVEP EEG signals in step 2) is 1000Hz, 1~100Hz of bandpass filtering, and is carried out
Line 50Hz traps, the record occipitalia area electrodes signal sensitive to SSVEP, including in T5, T6, P3, P4, PZ, O1, O2 and OZ
Some or all of, reference electrode A2, grounding electrode GND, distribution of electrodes meet 10~20 international standard leads, in experiment
Keep electrode impedance at 5k ohm or less.
Phase-space Reconstruction is applied described in step 3), selectes threshold value, determines recurrence plot respectively, including：
(1) each SSVEP EEG signals obtained, are all obtained by p electrode, are obtained from each electrode
Signal length is L, determines the time-frequency combination distribution of wherein each electrode signal, extraction and picture flicker frequency in stimulation interface respectively
The corresponding energy value sequence X of the corresponding frequency range of rate, i.e.,：
WhereinIndicate that the length of k-th of electrode is the signal of L；
Phase space reconfiguration is carried out respectively, obtains the trajectory of phase space of k-th of electrode：
Wherein, N is the number of vector point after phase space reconfiguration, and m is Embedded dimensions, and τ is delay time,Indicate kth
The trajectory of phase space that a electrode obtains after phase space reconfiguration, t indicate serial number vectorial in trajectory of phase space, xk,iIt indicates
I-th of element of k-th of electrode in the energy value sequence of SSVEP EEG signals；
(2) for the signal of any two electrode α and λWithWherein α ≠ λ repeats (1) step and obtains respectively
To corresponding trajectory of phase spaceWherein u=1,2 ..., N, andWherein v=1,2 ..., N；Pass through intersection
Recurrence obtains the cross recurrence plots that a size is N × N：
Wherein, ε is threshold value, is set as the sum of 15% two electrode signal standard deviations, described two electrode signals
Refer to the current corresponding electrode of cross recurrence plots calculated,It indicates arbitrary two in two trajectory of phase space
The distance between vector point；Indicate ifThen value is 1, ifThen value is 0；WhereinIndicate the cross recurrence plots obtained by the signal of electrode α and λ；If intersecting
Recurrence plotValue be 1, then in recurrence plot be black, ifValue be 0, then be in recurrence plot white.
To the energy value sequence for the multi-electrode SSVEP EEG signals that single-subject person is induced by any stimulation pictureIt obtainsOpen recurrence plot.
Step 4) includes：Subject induces by watching any one stimulation picture attentively and generates multielectrode SSVEP brains telecommunications
Number, watch different stimulation pictures attentively and then induces different SSVEP EEG signals, each SSVEP EEG signals and the thorn watched attentively
Flicker frequency and the initial phase for swashing picture are associated, by the energy value sequence of SSVEP EEG signals after phase space reconfiguration
Obtained recurrence plot sets label according to corresponding flicker frequency, initial phase and related electrode.
A kind of application for the multiple target SSVEP idea control methods merging recurrence plot and deep learning, exploitation are based on multiple target
The brain machine interface system of SSVEP, user stimulate picture by targetedly watching SSVEP attentively, induce SSVEP EEG signals, warp
After convergence analysis, classification generate idea control instruction, support user in the case that speak, it is action inconvenient, complete it is how free
Idea control is spent, word input, smart home, field of play are widely used in.
Multiple target SSVEP ideas control methods and the application of the fusion recurrence plot and deep learning of the present invention, have following excellent
Gesture：
1) phase space reconfiguration is carried out to the energy value sequence of SSVEP EEG signals based on Phase Space Theory, can more preferably excavated
The polynary complication system SSVEP response messages of brain.
2) control of multiple degrees of freedom idea can be realized by the modulation of frequency-phase, and the complex control in multiple target is suitble to lead
It is applied in domain.
Description of the drawings
Fig. 1 is the flow chart of the multiple target SSVEP idea control methods of present invention fusion recurrence plot and deep learning；
Fig. 2 is multiple target SSVEP brain electricity experiment stimulation interface；
Fig. 3 is depth convolutional neural networks model schematic；
Fig. 4 a are the schematic diagrames of convolution operation in the embodiment of the present invention；
Fig. 4 b are the schematic diagrames that pondization operates in the embodiment of the present invention.
Specific implementation mode
Multiple target SSVEP idea controls with reference to embodiment and attached drawing to the fusion recurrence plot and deep learning of the present invention
Preparation method and application are described in detail.
The multiple target SSVEP idea control methods of the fusion recurrence plot and deep learning of the present invention, implement to increase phase information
Multiple target SSVEP brain electricity experiment, by electroencephalogramsignal signal collection equipment complete SSVEP EEG signals obtain, be based on phase space weight
The energy value sequence of the SSVEP EEG signals of each electrode is respectively embedded in higher dimensional space by structure theory, more preferable to reflect that brain is multiple
The substantive characteristics of miscellaneous system, it is more detailed from higher-dimension angle, clearly to the energy value sequence of SSVEP EEG signals carry out information
Excavation and feature extraction, determine recurrence plot.The follow-up depth convolutional neural networks that introduce are theoretical, can reflect SSVEP brain electricity
The recurrence plot of the energy value sequence signature of signal inputs for data, the accurate recognition of realization multiple target SSVEP EEG signals, and
Control of multiple degrees of freedom idea is realized by brain-computer interface on the basis of this.As shown in Figure 1, specifically comprising the following steps：
1) increase phase information, design multiple target SSVEP brain electricity experiment stimulation interface, including the total n stimulation figure of a rows b row
Piece, in a line, b stimulation picture initial phase is identical but frequency of stimulation is different, and in same row, a stimulation picture flickers frequency
Rate is identical but initial phase is different；
Fig. 2 provides the stimulation interface specific example of 4 rows 6 row：Frequency of use-phase coding method constructs one 24
Target identification system, target is by 6 frequencies (frequency range is 8~13Hz, frequency interval 1Hz) and 4 phase (0,0.5 π, π
With 1.5 π) encoded, each target be according to setting frequency and phase into line flicker picture；System interface
Main part is one 4 × 6 box array, totally 24 kinds of stimulation pictures；Each box size is 140 × 140 pixels, arbitrary two
Spacing between a adjacent box is 50 pixels；Visual stimulus is presented on an lcd display, which is 60Hz.
The phase is the initial phase that corresponding picture starts flicker.
2) it based on the brain electricity of multiple target SSVEP described in step 1) tests stimulation interface, is set by eeg signal acquisition
Back-up Huo Qu not each subject induces through n stimulation picture in 8 or more subjects n kind SSVEP EEG signals；
Wherein, sample rate when obtaining SSVEP EEG signals is 1000Hz, 1~100Hz of bandpass filtering, and is carried out online
50Hz traps, the record occipitalia area electrodes signal sensitive to SSVEP, including the portion in T5, T6, P3, P4, PZ, O1, O2 and OZ
Divide or all, reference electrode A2, grounding electrode GND, distribution of electrodes meet 10~20 international standard leads, protected in experiment
Electrode impedance is held at 5k ohm or less.
A specific example is provided in conjunction with Fig. 2：For single-subject person, primary complete experiment is tested comprising 6 sons, often
Subject needs to watch 24 stimulation pictures attentively successively in height experiment, i.e., first watches the stimulation figure of the first row attentively from left to right successively
Then piece watches next line attentively again, all watched attentively one time until all 24 pictures and complete one group of son experiment, subject only notes every time
Depending on a stimulation picture, referred to as Target Photo；Wherein, subject watches the single stimulation picture duration attentively for 5s, is then spaced
5s is recycled successively, one group of sub- Therapy lasted 240s.In order to slow down the visual fatigue of subject, two sub- trial interval 5min give
The time of having a rest of subject's abundance.In addition, subject is required that the when of flickering is being stimulated to avoid blink to reduce eye electricity to EEG as possible
The interference of EEG signals.
3) to the n kind SSVEP EEG signals for n stimulation picture of each subject, determination is wherein each respectively
The time-frequency combination of electrode signal is distributed, extraction energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface
Row select threshold value using Phase-space Reconstruction, determine recurrence plot respectively, obtain n and correspond to n kind SSVEP EEG signals respectively
Energy value sequence recurrence plot, obtain 8 or more subjects different stimulated picture induction under EEG signals recurrence plot；
Wherein, the application Phase-space Reconstruction, selectes threshold value, determines recurrence plot respectively, including：
(1) each SSVEP EEG signals obtained, are all obtained by p electrode, are obtained from each electrode
Signal length is L, determines the time-frequency combination distribution of wherein each electrode signal, extraction and picture flicker frequency in stimulation interface respectively
The corresponding energy value sequence X of the corresponding frequency range of rate, i.e.,：
WhereinIndicate that the length of k-th of electrode is the signal of L；
Phase space reconfiguration is carried out respectively, obtains the trajectory of phase space of k-th of electrode：
Wherein, N is the number of vector point after phase space reconfiguration, and m is Embedded dimensions, and τ is delay time,Indicate kth
The trajectory of phase space that a electrode obtains after phase space reconfiguration, t indicate serial number vectorial in trajectory of phase space, xk,iIt indicates
I-th of element of k-th of electrode in the energy value sequence of SSVEP EEG signals；
(2) for the signal of any two electrode α and λWithWherein α ≠ λ repeats (1) step and obtains respectively
To corresponding trajectory of phase spaceWherein u=1,2 ..., N, andWherein v=1,2 ..., N；Pass through intersection
Recurrence obtains the cross recurrence plots that a size is N × N：
Wherein, ε is threshold value, is set as the sum of 15% two electrode signal standard deviations, described two electrode signals
Refer to the current corresponding electrode of cross recurrence plots calculated, such as calculatesWhen, two electrode signals refer toWith Indicate the distance between arbitrary two vector point in two trajectory of phase space；Indicate ifThen value is 1, ifThen value is
0；WhereinIndicate the cross recurrence plots obtained by the signal of electrode α and λ；If cross recurrence plotsValue be 1, then exist
It is black in recurrence plot, ifValue be 0, then be in recurrence plot white.
To the energy value sequence for the multi-electrode SSVEP EEG signals that single-subject person is induced by any stimulation pictureIt obtainsOpen recurrence plot.
Above-mentioned multi-electrode SSVEP EEG signals refer to T5, T6, P3, P4, PZ, O1, O2, the OZ obtained by the experiment of brain electricity
Etc. the signal of some or all of electrode；By taking two channel datas of T5 and P3 as an example, according to above-mentioned recurrence plot construction method, for
A recurrence plot can be obtained in particular stimulation picture and sole user；By taking tetra- channel datas of T5, T6, P3, P4 as an example, according to upper
Recurrence plot construction method is stated, it is available for particular stimulation picture and sole userOpen recurrence plot.
4) the phase and frequency information and electrode information of stimulation picture are corresponded to by recurrence plot, each to obtain is passed
Return figure setting label as sample, builds data set；Including：
Subject induces by watching any one stimulation picture attentively and generates multielectrode SSVEP EEG signals, watch attentively different
Stimulation picture then induces different SSVEP EEG signals, the flicker frequency of each SSVEP EEG signals and the stimulation picture watched attentively
Rate and initial phase are associated, the recurrence plot that the energy value sequence of SSVEP EEG signals is obtained after phase space reconfiguration, root
Label is set according to corresponding flicker frequency, initial phase and related electrode.
For example, when subject watches flicker frequency f=11Hz attentively, when the stimulation picture of initial phase θ=π, can induce corresponding
Multi-electrode SSVEP EEG signals, after carrying out phase space reconfiguration to its energy value sequence, it is assumed that choose two number of electrodes of T5 and P3
According to obtaining recurrence plot, which can merge f=11Hz, and the information such as θ=π and electrode are into row label, such as can be labeled as ＜
11Hz, π, T5, P3 ＞.
5) depth convolutional neural networks (DCNN) model is built, determines depth convolutional neural networks model structure and initial
Change model parameter, as training set, the sample of 20% tape label will instruct the sample of 80% tape label as test set in data set
Practice collection input depth convolutional neural networks model, the study for having supervision is carried out to sample data feature in training set, optimizes depth
Convolutional neural networks model structure and parameter are determined to by test set sample for verifying depth convolutional neural networks model
The depth convolution god of the recurrence plot of energy value sequence for the SSVEP EEG signals induced by different stimulated picture of effectively classifying
Through network model；
Wherein, DCNN model trainings process is：
(1) as shown in figure 3, building the DCNN models including input layer, convolutional layer, pond layer, output layer etc.；
(2) it is input with the sample of tape label in training set for entire DCNN models, by the way of supervised learning pair
Network is trained, and optimal model parameters are determined using gradient descent method and error back propagation new mechanism；
In the training process, convolution operation is to determine part by convolution kernel and the similitude of the sample content currently covered
Feature, convolution kernel realize that the extraction of character pair in input sample, the character pair are related to convolution kernel after traversing.Convolution mistake
Journey can be described as：
As shown in fig. 4 a, it is illustrated by taking the convolution kernel of a length of side l=3 as an example, wherein i, j are convolution kernels in input sample
Coordinate in this, is subject to position of the convolution kernel center element in input sample, in fig.4 upper left corner convolution kernel institute
It is convolution kernel element x in position5Position, lower right corner convolution kernel position are convolution kernel element x5' position,
Wherein xmIndicate m-th of numerical value in convolution kernel, kmIndicate m-th of input sample under convolution kernel covering after reorganization sequence
Numerical value, bijIndicate convolution kernel corresponding biasing under this position, element x in convolution kernelmWith element k in input samplemCorresponding phase
Multiply, it is cumulative after plus corresponding position bias bij, by activation primitive f1After export Cij.Convolution kernel with step-length τ to the left or to
Lower movement obtains the convolution output C of different locationij, complete the feature extraction based on this convolution kernel.Different convolution kernels can carry
Different types of feature in input sample, multilayer convolution is taken to can be used for extracting the feature of different depth in input sample.Convolution kernel
It can at random be generated based on Gaussian Profile.
Down-sampling layer is also referred to as pond layer, and polymerization system is carried out to the feature of the different location of the characteristic pattern obtained by convolution
Meter effectively prevent the purpose of over-fitting, it can be achieved that reduction data dimension.Pond process can be described as：
Pij=f2(βijdown(C)+b′ij)
Wherein i, j are coordinates of the Chi Huahe in input, are with position of the pond core top left hand element in input sample
Standard, C indicate input as the element set under the kernel covering of forebay, βijWith b 'ijIndicate that Chi Huahe is corresponding under this position respectively
Weight and biasing, element set C weight beta is multiplied by after down-samplingij, along with biasing bi′jBy activation primitive f2Output
Pij.Wherein down () indicates down-sampling function, and there are commonly maximum pond, average pond, spatial pyramid ponds.Such as Fig. 4 b
It is shown, maximum pond process is illustrated by taking the Chi Huahe of length of side η=2 as an example, detailed process can be described as：
Pij=max { Cn| 0 ＜ n≤η2}
It indicates to the element extraction maximum value when the input under the kernel covering of forebay as output.
Input sample realizes feature extraction by multilayer convolution, Chi Hua, after characteristic information is sent to output layer.Using
Gradient descent method Optimized model parameter, and definition E (ω | A) it is model global errors of the parameter set ω on given training set A, really
Determine optimized parameter collection ω*, meetIndicate the set of the ω values when E (ω | A) reaches minimum value,
As ω*。
When E (ω | A) is the differentiable function of parameter ω, the gradient vector of partial derivative composition is
Based on gradient vector, error E (ω | A), undated parameter ω ' are minimized by gradient descent methodi=ωi+Δωi,
WhereinUse ω 'iI-th of parameter as next round model training, wherein η is Studying factors, as E (ω
| A) when obtaining minimum, derivative is equal to 0, and process terminates.At the beginning of training, parameter set ω can random initializtion, it can also be used for reference
His optimized parameter of model.
(3) the optimum depth convolutional neural networks model obtained by above-mentioned training, can be according to sample the characteristics of, implement accurate
True classification, classification results are some stimulated in interface in n picture.It, can be according to when thering is picture to enter model again
Such sample characteristics of extraction are classified.
6) the subject's SSVEP EEG signals that will newly obtain determine the time-frequency combination point of wherein each electrode signal respectively
Cloth, extraction energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface, after phase space reconfiguration, with
Recurrence diagram form input step 5) in depth convolutional neural networks model after optimization, realize that multiple target SSVEP EEG signals are accurate
Really classification；
7) by multiple target SSVEP eeg signal classifications as a result, the anti-stimulation picture for pushing away subject and watching attentively, is noted with subject
The stimulation picture regarded generates idea control instruction as foundation, realizes the control of multiple target idea.
The application of the fusion recurrence plot of the present invention and the multiple target SSVEP idea control methods of deep learning, exploitation is based on more
The brain machine interface system of target SSVEP, user stimulate picture by targetedly watching SSVEP attentively, induce SSVEP brain telecommunications
Number, it is fused analysis, classification after generate idea control instruction, support user in the case that speak, it is action inconvenient, complete it is more
Degree of freedom idea controls, and is widely used in word input, smart home, field of play.
Now by taking smart home as an example, user looks first at SSVEP stimulations interface, and interface content includes the operation to household electrical appliances,
Such as air-conditioning lifting/lowering temperature, TV, ON/OFF curtain etc. are opened/closed, the corresponding stimulation picture of each operation；User is according to oneself
Want the operation completed, selected stimulation picture is simultaneously watched attentively, and the SSVEP brain telecommunications for inducing and generating is obtained by brain wave acquisition equipment
Number, by phase space reconfiguration, DCNN theories and methods, determine the stimulation picture of user's observation；Finally by brain-computer interface to this
The corresponding household electrical appliances of picture send out corresponding instruction, execute the corresponding operation of picture, realize idea control.
Above to the description of the present invention and embodiment, it is not limited to which this, the description in embodiment is only the reality of the present invention
One of mode is applied, it is without departing from the spirit of the invention, any not inventively to design and the technical solution
Similar structure or embodiment, belongs to protection scope of the present invention.
Claims (6)
1. a kind of multiple target SSVEP idea control methods of fusion recurrence plot and deep learning, which is characterized in that implement to increase phase
The multiple target SSVEP brain electricity of information is tested, and completing SSVEP EEG signals by electroencephalogramsignal signal collection equipment obtains, and fusion is mutually empty
Between re-construction theory and depth convolutional neural networks it is theoretical, realize the accurate recognition of multiple target SSVEP EEG signals, and then develop
Brain-computer interface is realized the control of multiple degrees of freedom idea, is specifically comprised the following steps：
1) increase phase information, design multiple target SSVEP brain electricity experiment stimulation interface, including a rows b arranges total n stimulation picture, together
In a line, b stimulation picture initial phase is identical but frequency of stimulation is different, and in same row, a stimulation picture flicker frequency is identical
But initial phase is different；
2) based on the brain electricity of multiple target SSVEP described in step 1) tests stimulation interface, pass through electroencephalogramsignal signal collection equipment point
It Huo Qu not each subject induces through n stimulation picture in 8 or more subjects n kind SSVEP EEG signals；
3) to the n kind SSVEP EEG signals for n stimulation picture of each subject, wherein each electrode is determined respectively
The time-frequency combination of signal is distributed, extraction energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface,
Using Phase-space Reconstruction, threshold value is selected, determines recurrence plot respectively, n is obtained and corresponds to n kind SSVEP EEG signals respectively
The recurrence plot of energy value sequence obtains the recurrence plot of EEG signals of 8 or more the subjects under the induction of different stimulated picture；
4) the phase and frequency information and electrode information that stimulation picture is corresponded to by recurrence plot, for each obtained recurrence plot
Label is set as sample, builds data set；
5) depth convolutional neural networks model is built, determines depth convolutional neural networks model structure and initialization model ginseng
It counts, as training set, the sample of 20% tape label inputs training set as test set the sample of 80% tape label in data set
Depth convolutional neural networks model carries out the study for having supervision to sample data feature in training set, optimizes depth convolutional Neural
Network architecture and parameter are determined to by test set sample for verifying depth convolutional neural networks model for effective
The depth convolutional neural networks mould of the recurrence plot of the energy value sequence for the SSVEP EEG signals that classification is induced by different stimulated picture
Type；
6) the subject's SSVEP EEG signals that will newly obtain determine the time-frequency combination distribution of wherein each electrode signal, carry respectively
Energy value sequence corresponding with the corresponding frequency range of picture flicker frequency in stimulation interface is taken, after phase space reconfiguration, with recurrence
Diagram form input step 5) in optimization after depth convolutional neural networks model, realize multiple target SSVEP EEG signals accurately divide
Class；
7) by multiple target SSVEP eeg signal classifications as a result, the anti-stimulation picture for pushing away subject and watching attentively, is watched attentively with subject
Stimulation picture is foundation, generates idea control instruction, realizes the control of multiple target idea.
2. the multiple target SSVEP idea control methods of fusion recurrence plot and deep learning according to claim 1, feature exist
In sample rate when obtaining SSVEP EEG signals in step 2) is 1000Hz, 1~100Hz of bandpass filtering, and is carried out online
50Hz traps, the record occipitalia area electrodes signal sensitive to SSVEP, including the portion in T5, T6, P3, P4, PZ, O1, O2 and OZ
Divide or all, reference electrode A2, grounding electrode GND, distribution of electrodes meet 10~20 international standard leads, protected in experiment
Electrode impedance is held at 5k ohm or less.
3. the multiple target SSVEP idea control methods of fusion recurrence plot and deep learning according to claim 1, feature exist
In, Phase-space Reconstruction is applied described in step 3), selectes threshold value, determines recurrence plot respectively, including：
(1) each SSVEP EEG signals obtained, are all obtained by p electrode, the signal obtained from each electrode
Length is L, determines the time-frequency combination distribution of wherein each electrode signal, extraction and picture flicker frequency phase in stimulation interface respectively
The corresponding energy value sequence X of corresponding frequency range, i.e.,：
WhereinIndicate that the length of k-th of electrode is the signal of L；
Phase space reconfiguration is carried out respectively, obtains the trajectory of phase space of k-th of electrode：
Wherein, N is the number of vector point after phase space reconfiguration, and m is Embedded dimensions, and τ is delay time,Indicate k-th of electricity
The trajectory of phase space that pole obtains after phase space reconfiguration, t indicate serial number vectorial in trajectory of phase space, xk,iIndicate SSVEP brains
I-th of element of k-th of electrode in the energy value sequence of electric signal；
(2) for the signal of any two electrode α and λWithWherein α ≠ λ repeats (1) step and respectively obtains pair
The trajectory of phase space answeredWherein u=1,2 ..., N, andWherein v=1,2 ..., N；It is passed by intersecting
Return, obtains the cross recurrence plots that a size is N × N：
Wherein, ε is threshold value, is set as the sum of 15% two electrode signal standard deviations, and described two electrode signals refer to
The current corresponding electrode of cross recurrence plots calculated,Indicate arbitrary two vector in two trajectory of phase space
The distance between point；Indicate ifThen value is 1, ifThen value is 0；WhereinIndicate the cross recurrence plots obtained by the signal of electrode α and λ；If intersecting
Recurrence plotValue be 1, then in recurrence plot be black, ifValue be 0, then be in recurrence plot white.
4. the multiple target SSVEP idea control methods of fusion recurrence plot and deep learning according to claim 3, feature exist
In to the energy value sequence for the multi-electrode SSVEP EEG signals that single-subject person is induced by any stimulation pictureIt obtainsOpen recurrence plot.
5. the multiple target SSVEP idea control methods of fusion recurrence plot and deep learning according to claim 1, feature exist
In step 4) includes：Subject induces by watching any one stimulation picture attentively and generates multielectrode SSVEP EEG signals, watch attentively
Different stimulation pictures then induce different SSVEP EEG signals, each SSVEP EEG signals and the stimulation picture watched attentively
Flicker frequency and initial phase are associated, are passed what the energy value sequence of SSVEP EEG signals obtained after phase space reconfiguration
Gui Tu sets label according to corresponding flicker frequency, initial phase and related electrode.
6. a kind of application of the multiple target SSVEP idea control methods of fusion recurrence plot described in claim 1 and deep learning,
It is characterized in that, develops the brain machine interface system based on multiple target SSVEP, user is by targetedly watching SSVEP stimulation figures attentively
Piece induces SSVEP EEG signals, generates idea control instruction after fused analysis, classification, user is supported to speak, taking action not
In the case of convenient, the control of multiple degrees of freedom idea is completed, word input, smart home, field of play are widely used in.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810169303.4A CN108445751B (en) | 2018-02-28 | 2018-02-28 | Multi-target SSVEP idea control method fusing recursive graph and deep learning and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810169303.4A CN108445751B (en) | 2018-02-28 | 2018-02-28 | Multi-target SSVEP idea control method fusing recursive graph and deep learning and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108445751A true CN108445751A (en) | 2018-08-24 |
| CN108445751B CN108445751B (en) | 2021-03-16 |
Family
ID=63193139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810169303.4A Active CN108445751B (en) | 2018-02-28 | 2018-02-28 | Multi-target SSVEP idea control method fusing recursive graph and deep learning and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108445751B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109299751A (en) * | 2018-11-26 | 2019-02-01 | 南开大学 | The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data |
| CN109567793A (en) * | 2018-11-16 | 2019-04-05 | 西北工业大学 | A kind of ECG signal processing method towards cardiac arrhythmia classification |
| CN109784023A (en) * | 2018-11-28 | 2019-05-21 | 西安电子科技大学 | Stable state vision inducting brain electricity personal identification method and system based on deep learning |
| CN110443276A (en) * | 2019-06-30 | 2019-11-12 | 天津大学 | Time series classification method based on depth convolutional network Yu the map analysis of gray scale recurrence |
| CN111281382A (en) * | 2020-03-04 | 2020-06-16 | 徐州市健康研究院有限公司 | Feature extraction and classification method based on electroencephalogram signals |
| CN112396109A (en) * | 2020-11-19 | 2021-02-23 | 天津大学 | Motor bearing fault diagnosis method based on recursion graph and multilayer convolution neural network |
| CN112426705A (en) * | 2020-12-15 | 2021-03-02 | 华南师范大学 | Brain control gobang system based on SBCNN |
| CN114003048A (en) * | 2021-12-31 | 2022-02-01 | 季华实验室 | Multi-target object motion control method and device, terminal equipment and medium |
| CN111317468B (en) * | 2020-02-27 | 2024-04-19 | 腾讯科技（深圳）有限公司 | Electroencephalogram signal classification method, electroencephalogram signal classification device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023701A (en) * | 2009-09-17 | 2011-04-20 | 国立中央大学 | Visual drive method, control method and control system of brain wave human-computer interface |
| US20120299822A1 (en) * | 2008-07-25 | 2012-11-29 | National Central University | Communication and Device Control System Based on Multi-Frequency, Multi-Phase Encoded Visual Evoked Brain Waves |
| US20140058483A1 (en) * | 2012-08-25 | 2014-02-27 | National Chiao Tung University | Stimuli generating methods, devices and control systems to induce visual evoked potentials using imperceptible flickering multi-color lights |
| CN104536573A (en) * | 2014-12-30 | 2015-04-22 | 天津大学 | Brain-computer interface method based on high-frequency flicker emotional simulation |
| CN106650929A (en) * | 2016-10-11 | 2017-05-10 | 天津大学 | Recursive-graph-based deep learning model and its application in oil-water phase rate measurement |
| CN107168524A (en) * | 2017-04-19 | 2017-09-15 | 华南理工大学 | A kind of Steady State Visual Evoked Potential sorting technique based on deep learning mixed model |
-
2018
- 2018-02-28 CN CN201810169303.4A patent/CN108445751B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120299822A1 (en) * | 2008-07-25 | 2012-11-29 | National Central University | Communication and Device Control System Based on Multi-Frequency, Multi-Phase Encoded Visual Evoked Brain Waves |
| CN102023701A (en) * | 2009-09-17 | 2011-04-20 | 国立中央大学 | Visual drive method, control method and control system of brain wave human-computer interface |
| US20140058483A1 (en) * | 2012-08-25 | 2014-02-27 | National Chiao Tung University | Stimuli generating methods, devices and control systems to induce visual evoked potentials using imperceptible flickering multi-color lights |
| CN104536573A (en) * | 2014-12-30 | 2015-04-22 | 天津大学 | Brain-computer interface method based on high-frequency flicker emotional simulation |
| CN106650929A (en) * | 2016-10-11 | 2017-05-10 | 天津大学 | Recursive-graph-based deep learning model and its application in oil-water phase rate measurement |
| CN107168524A (en) * | 2017-04-19 | 2017-09-15 | 华南理工大学 | A kind of Steady State Visual Evoked Potential sorting technique based on deep learning mixed model |
Non-Patent Citations (2)
| Title |
|---|
| CHUAN JIA 等: "Frequency and Phase Mixed Coding in SSVEP-Based Brain–Computer Interface", 《IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING》 * |
| XIAOGANG CHEN 等: "Hybrid Frequency and Phase Coding for a High-Speed SSVEP-Based BCI Speller", 《CONF PROC IEEE ENG MED BIOL SOC》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109567793B (en) * | 2018-11-16 | 2021-11-23 | 西北工业大学 | Arrhythmia classification-oriented ECG signal processing method |
| CN109567793A (en) * | 2018-11-16 | 2019-04-05 | 西北工业大学 | A kind of ECG signal processing method towards cardiac arrhythmia classification |
| CN109299751A (en) * | 2018-11-26 | 2019-02-01 | 南开大学 | The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data |
| CN109299751B (en) * | 2018-11-26 | 2022-05-31 | 南开大学 | EMD data enhancement-based SSVEP electroencephalogram classification method of convolutional neural model |
| CN109784023A (en) * | 2018-11-28 | 2019-05-21 | 西安电子科技大学 | Stable state vision inducting brain electricity personal identification method and system based on deep learning |
| CN109784023B (en) * | 2018-11-28 | 2022-02-25 | 西安电子科技大学 | Steady-state vision-evoked electroencephalogram identity recognition method and system based on deep learning |
| CN110443276A (en) * | 2019-06-30 | 2019-11-12 | 天津大学 | Time series classification method based on depth convolutional network Yu the map analysis of gray scale recurrence |
| CN111317468B (en) * | 2020-02-27 | 2024-04-19 | 腾讯科技（深圳）有限公司 | Electroencephalogram signal classification method, electroencephalogram signal classification device, computer equipment and storage medium |
| CN111281382A (en) * | 2020-03-04 | 2020-06-16 | 徐州市健康研究院有限公司 | Feature extraction and classification method based on electroencephalogram signals |
| CN111281382B (en) * | 2020-03-04 | 2023-08-18 | 徐州市健康研究院有限公司 | Feature extraction and classification method based on electroencephalogram signals |
| CN112396109A (en) * | 2020-11-19 | 2021-02-23 | 天津大学 | Motor bearing fault diagnosis method based on recursion graph and multilayer convolution neural network |
| CN112426705A (en) * | 2020-12-15 | 2021-03-02 | 华南师范大学 | Brain control gobang system based on SBCNN |
| CN112426705B (en) * | 2020-12-15 | 2023-08-15 | 华南师范大学 | Brain-controlled gobang system based on SBCNN |
| CN114003048A (en) * | 2021-12-31 | 2022-02-01 | 季华实验室 | Multi-target object motion control method and device, terminal equipment and medium |
| CN114003048B (en) * | 2021-12-31 | 2022-04-26 | 季华实验室 | Multi-target object motion control method and device, terminal equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108445751B (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108445751A (en) | Merge multiple target SSVEP ideas control methods and the application of recurrence plot and deep learning | |
| US10929745B2 (en) | Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways | |
| JP6185142B2 (en) | Clinical response data mapping | |
| CN107168524A (en) | A kind of Steady State Visual Evoked Potential sorting technique based on deep learning mixed model | |
| CN110175595A (en) | Human body attribute recognition approach, identification model training method and device | |
| CN109785928A (en) | Diagnosis and treatment proposal recommending method, device and storage medium | |
| US11077301B2 (en) | Topical nerve stimulator and sensor for bladder control | |
| CN108463266A (en) | User interface for neural stimulation waveform construction | |
| CN105068644A (en) | Method for detecting P300 electroencephalogram based on convolutional neural network | |
| CN108446021B (en) | Application method of P300 brain-computer interface in intelligent home based on compressed sensing | |
| CN108960182A (en) | A kind of P300 event related potential classifying identification method based on deep learning | |
| CN106951247B (en) | Dynamic background display method and device | |
| CN106485688A (en) | High spectrum image reconstructing method based on neutral net | |
| CN107194426A (en) | A kind of image-recognizing method based on Spiking neutral nets | |
| CN105260025B (en) | Steady State Visual Evoked Potential brain machine interface system based on mobile terminal | |
| Heeger et al. | A recurrent circuit implements normalization, simulating the dynamics of V1 activity | |
| Oesterle et al. | Bayesian inference for biophysical neuron models enables stimulus optimization for retinal neuroprosthetics | |
| CN102654793B (en) | Electrocerebral-drive high-reliability control system based on dual-mode check mechanism | |
| CN110522412A (en) | Method based on multiple dimensioned brain function network class EEG signals | |
| CN106581858A (en) | Physiotherapy instrument | |
| US20170357274A1 (en) | Garment optimization | |
| CN110221681A (en) | The method of adjustment and equipment of image-recognizing method, image rendering time | |
| CN107122050A (en) | Stable state of motion VEP brain-machine interface method based on CSFL GDBN | |
| CN106022294A (en) | Intelligent robot-oriented man-machine interaction method and intelligent robot-oriented man-machine interaction device | |
| Sorrentino et al. | Dynamic filtering of static dipoles in magnetoencephalography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |