CN101430600A - Game auxiliary control method based on imagination electroencephalogram - Google Patents

Abstract

The invention belongs to the nerve engineering technical field of biomedical engineering. The method sequentially comprises the following steps: the specific stimulation procedure training is carried out on a testee firstly; the testee becomes familiar with the stimulation procedure and generates different imaginations according to the stimulation procedure, so that brain waves are generated according to the different imaginations; the brain waves are sent to a computer to be analyzed after being amplified and converted into digital signals; signal characteristics of the brain waves with different imaginations are extracted; finally, a control command of the testee to be sent out is judged and reduced according to the extracted signal characteristics of the brain waves, so as to achieve the process from imagination to movement. The invention mainly utilizes imagination of the brain waves which is the direct control process as a novel computer game input method. The method can help the disabled play computer games and can be applied to occasions that normal people cannot use and operate computers conveniently.

Description

Game auxiliary control method based on imagination brain electricity

Technical field

The present technique invention belongs to neural field of engineering technology in the biomedical engineering.

Technical background:

The electroencephalogramsignal signal analyzing of human brain to the difference imagination shown, the different motion imaginations can produce different reflections in different brain districts, according to this EEG signals difference, can extract the EEG signals feature of the different motion imaginations, utilize the sorting algorithm of setting, can make the brain electricity imagination be reduced into corresponding motion.

In today that computer and network technologies prevail, various computing machines and online game become a main flow in people's life, common recreation input and control are to import by physical equipments such as keyboard, mouse, operating rods, this makes muscle and neurological disorder personage, and playing games becomes a kind of dream.

Summary of the invention

Purpose of the present invention is exactly by external unit, makes brain imagination EEG signals without muscle and neural conduction, directly as computer input signal, reaches the game control purpose.

The present invention is based on the game auxiliary control method of imagination brain electricity, comprise following steps:

Step 1, subject are with the polar cap that powers on, and reference electrode is set;

Step 2, the stimulation programs that basis configures on computer screen, the subject makes the different imaginations according to testing requirements.

Step 3, subject make the various imaginations according to step 2, collect the brain wave of different imagination types;

Step 4, according to the brain wave that above-mentioned steps collects, by the brain wave acquisition instrument, be sent in signal analysis and the converting system, change into control command at last.

Step 41, by the electroencephalogramsignal signal analyzing system, according to the feature that extracts, judge the EEG signals characteristic type of input system;

Step 42, according to the result of 41 steps, the output corresponding action.

2, by the particular stimulation pattern EEG signals is carried out feature extracting method, key step:

Step 1, subject are with the polar cap that powers on, and reference electrode is set;

Step 2, the stimulation programs that basis configures on computer screen, the subject makes the different imaginations according to testing requirements.

Step 3, collect EEG signals and carry out pre-service;

Step 4, utilization signal processing method carry out feature extraction to EEG signals;

The result of step 5, applying step 4 classifies to EEG signals by sorting algorithm;

What the present invention designed is that imagination EEG signals is carried out sorting technique

Because the singularity of EEG signals, different EEG signals features and extraction sorting technique, different to different people's classify accuracy, in brain electricity input Gamecontrol system (GCS-EEGI) system, three kinds of methods have mainly been designed, according to subject's results of learning difference, system selects the highest algorithm of resolution automatically: short time discrete Fourier transform, second-order blind identification and phase locked method.

1, short time discrete Fourier transform method:

Original EEG is the signal on the time domain, energy distribution is disperseed, characteristic signal is buried among the noise, in order better to extract feature, original time-domain signal is transformed on the time-frequency domain, time-domain signal is converted to time frequency signal, the method that the present invention designs use is to use short time discrete Fourier transform (STFT), according to the conversion of signals analysis, find that it mainly is at 8～13Hz that signal energy is concentrated, mainly being the α wave band, secondly is at 18～30Hz, mainly is beta band (according to Schwab frequency categorization method).

If analyzed signal is x (t), t=-∞～∞, analysis window are g (t).The Fourier in short-term of definition non-stationary signal x (t)

Leaf transformation is:

$\mathrm{STFT}(t,w)=\stackrel{\∞}{\∫}\left[x\left(\mathrm{\τ}\right)g(\mathrm{\τ}-t)\right]e^{-\mathrm{jw\τ}}\mathrm{d\τ}$

G is an even function, so g (τ) g (e of τ-t) ^{-jw τ}With t is the center, is the center, does not rely on t and w along the time span of t, promptly along t

${\mathrm{\σ}}_t^2={\∫}_{-\∞}^{+\∞}{(\mathrm{\τ}-t)}^2\left|g_{t,w}\left(\mathrm{\τ}\right)\right|\mathrm{d\τ}={\∫}_{-\∞}^{+\∞}{\mathrm{\τ}}^2\left|g\left(\mathrm{\τ}\right)\right|\mathrm{d\τ}$

(2) frequency domain form of continuous short time discrete Fourier transform

According to Fourier transform character, the Fourier transform of two time-domain signal products equals the convolution of frequency domain separately, by

(1) formula can

:

Using this method carries out the method for EEG feature extraction and classification and mainly contains following steps:

1, Hjort data-switching;

In order to reduce the interference of peripheral electrode to actual signal, before carrying out data analysis, original signal is carried out data-switching, conversion method is as follows:

$C_i^H=c_i-\frac{1}{8}\underset{i\∈S}{\mathrm{\Σ}}{c}_j$

Wherein

Expression conversion back eeg data, c _iBe original eeg data, S represents c _i8 electrodes, wherein c of periphery _jThe original eeg data of representing these 8 electrodes.

Raw data is to lead by 64 to meet 10/20 international standard EEG amplifier collection in native system, and sampling rate is 250Hz, is reference electrode with left and right sides mastoid process, and the bandpass filter passband is 1-50Hz.

2, STFT conversion;

In order better to extract feature, we are transformed into original time-domain signal on the time-frequency domain, time-domain signal converts time frequency signal to, and the most frequently used method is to use short time discrete Fourier transform (STFT), and we adopt this paper is Spectrogram method in the Matlab tool box.

3, Feature Selection;

In native system, our Feature Selection mainly is to carry out Feature Selection according to fisher apart from size.At first each data segment of STFT signal is calculated the Fisher distance, choose the big data segment of fisher distance as signal characteristic.

4, classification;

The native system sorting algorithm mainly contains two kinds, and when using that fourier algorithm carries out feature extraction in short-term, what we preferentially selected for use is the linear classification algorithm.

2, second-order blind identification:

Make EEG signals continuous time of corresponding n the sensor of n column vector of x (t), then the EEG signals of corresponding i the sensor of xi (t).The linear instantaneous that each xi (t) can regard n source si (t) as mixes, and hybrid matrix is A, then

x(t)＝As(t)

The EEG signals x (t) that SOBI only utilizes sensor measurement to obtain obtains being similar to A-1 split-matrix W, makes

$\hat{s}\left(t\right)=\mathrm{Wx}\left(t\right)$

Be source signal continuous time that recovers.

The SOBI algorithm has two steps: at first sensor signal carry out zero-meanization, be shown below:

y(t)＝B(x(t)-<x(t)>)

Angle brackets＜〉show time average, so the average of y is zero.The value of matrix B makes the y (t) of the correlation matrix of y＜(t) ^TBe unit matrix, its value is provided by following formula

$B=\mathrm{diag}\left({\mathrm{\&lambda;}}_i^{-1/2}\right)U^T$

Wherein λ i is correlation matrix＜(x (t)-＜x (t) 〉) (x (t)-＜x (t) 〉) ^TEigenwert, U each row then be its characteristic of correspondence vector.

In second step, construct one group of diagonal matrix: choose one group of time delay τ s, the symmetrization correlation matrix of signal calculated y (t) and its time-delay signal y (t+ τ):

R _τ＝sym(<y(t)y(t+τ) ^T>)

Wherein

sym(M)＝(M+M ^T)/2

This is a function that asy matrix is changed into relevant symmetric matrix.The process of symmetrization has been lost some information, but effective solution is provided.

Calculated R τ, again R τ has been carried out diagonalization: by rotation matrix V, the utilization process of iteration makes

${\mathrm{\&Sigma;}}_{\mathrm{\&tau;}}{\mathrm{\&Sigma;}}_{i\&NotEqual;j}{\left(V^T{R}_{\mathrm{\&tau;}}V\right)}_{\mathrm{ij}}^2$

Obtain minimal value, then the estimation of separation matrix

W＝V ^TB

3, phase-locking

There are many methods to go to measure at present at signal x _i(t) and x _j(t) between synchronously, being used to analyze synchronously, more common method is classical relevant (consistance) Coh _Ij(f).Coherence function is by signal x _i(t) and x _j(t) (represent two electrode i, mutual spectral density function j) draws.Be defined as follows:

$S_{i,j}\left(f\right)=\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{X}_{\mathrm{in}}\left(f\right)X_{\mathrm{jn}}\left(f\right)$

X in formula (9) _i(f) be x _i(t) Fourier conversion.

Be signal x _jThe complex conjugate of signal Fourier transform (t).Complex phase responsibility number is that the amplitude square coherence spectrum is square normalization divided by the power spectrum of two signals of cross-spectrum.

${\mathrm{Coh}}_{\mathrm{ij}}\left(f\right)=\frac{{\left|S_{\mathrm{ij}}\left(f\right)\right|}^2}{\sqrt{S_{\mathrm{ii}}\left(f\right)S_{\mathrm{jj}}\left(f\right)}}$

Another kind of measuring of two signal Synchronization of measurement is phase-locked value PLV (phase 1ocking value), and the method is only considered the phase place of this signal.

PLV＝|<exp(j{Ф _i(t)-Ф _j(t)})>|

Here, Ф _i(t), Ф _j(t) be electrode i, the instantaneous phase of j.The calculating of this phase place can be passed through Hilbert (Hilbert) conversion or multiple Gabor wavelet transformation.Adopt the Hilbert conversion here, specifically describe as follows:

$x_i^{\%}\left(t\right)=\frac{1}{\mathrm{\&pi;}}\mathrm{PV}{\&Integral;}_{-\&infin;}^{\&infin;}\frac{x_i\left(\mathrm{\&tau;}\right)}{t-\mathrm{\&tau;}}\mathrm{d\&tau;}$

In the following formula definition, ${\mathrm{\&Phi;}}_i\left(t\right)=\arctan \frac{x_i^{\%}\left(t\right)}{x_i\left(t\right)}$ Be time series x _i(t) Hilbert conversion (being meant the EEG signal here), PV is meant Cauchy's principal value.This phase place can be by following calculating then:

${\mathrm{\&Phi;}}_i\left(t\right)=\arctan \frac{x_i^{\%}\left(t\right)}{x_i\left(t\right)}$

Before the instantaneous phase of calculating each electrode, need carry out bandpass filtering to this electrode signal, be to carry out instantaneous phase calculating like this to comprising the μ wave band.

Native system will directly replace the operation of keyboard and mouse by EEG signals, what the present invention used is imagination brain, it is later stage information characteristics extraction to the brain electricity, the method of using is that the real brain that passes through is imagined various mode of motion, it is carried out feature extraction and classification, thereby realize real from the abstract concrete motion of imagining.Realize directly from imagining the process of operation, electric brain as the recreation auxiliary input method, a kind of novel recreation input and control method are provided, can solve the problem that some disabled person can not play computer game, also can be used for the normal person uses the operational computations machine in inconvenience occasion.

The characteristics of this method are to utilize EEG signals at the difference between the different motion imagination, extract dissimilar EEG signals features, be converted to control signal at last, utilize certain external unit, break away from the dependence of motion to certain muscle, this system is significant in a lot of fields, can solve the problem that some disabled person can not play computer game, also can be used for the normal person uses the operational computations machine in inconvenience occasion.

Description of drawings

Fig. 1, the present invention utilize the brain electricity to carry out the game control process flow diagram

Fig. 2 is based on the game control feature extraction process flow diagram of imagination brain wave

Fig. 3 side-to-side movement system flowchart

The electrode riding position figure of Fig. 4 side-to-side movement system

Fig. 5 side-to-side movement is figure as a result.

Embodiment

The inventive method in brain electricity input Gamecontrol system (GCS-EEGI), is used for realizing that the controlled object of control carries out the example of two dimensional motion-move left and right, presses accompanying drawing 1,2,3 flow processs.Can realize through the following steps:

1, the subject is trained, by stimulus modality training subject, gather subject's EEG signals feature of different imagination processes, because native system mainly is to move on the plane, therefore main the collection led C3, C4 and the Cz electrode that meets under 10/20 international standard according to 64; Concrete electrode position as shown in Figure 4.

2, subject's using system is at first gathered EEG signals, by the eeg signal acquisition instrument, gathers EEG signals, the EEG signals input signal analytic system of the C3 that collects, C4 and Cz three conductive electrode;

3, the type of input EEG signals feature according to the 1 EEG signals feature of being extracted, is judged by electroencephalogramsignal signal analyzing system;

4, according to the 3 EEG signals characteristic types of judging, export corresponding operational order, and control the side-to-side movement of dolly on the screen.

Concrete experiment process as shown in Figure 2, about imagination electrode for encephalograms lay as shown in Figure 4, stimulus modality is to begin to occur one second screen stand-by period, show a cross, five seconds afterwards, train, two seconds random interval time appearred in imagination sign afterwards about screen occurred.

Experiment shows, this method can have effective realization to control controlled object by the issue an order of brain electricity to carry out two dimensional motion control.Average accuracy rate can reach 80%, and output speed can reach carries out subcommand output per 10 seconds, and concrete output mode and result are as shown in Figure 5.Fig. 5 is that Jiangxi Lantian College infotech research institute makes the Control of Automobile ridden in left or right direction surface chart based on imagination brain electricity, the right side, interface be about brain electricity output map comprising C3, Cz and C4 three derivatives according to electrode and level and vertical electric data, the surface chart left side is brain electric control signal output effect figure, middle two-wire is demarcated about being, the car color is redness if output signal is made mistakes then, otherwise is blue.

Claims (4)

1, based on the game auxiliary control method of imagination brain electricity, it is characterized in that: comprise following step:

Step 1, subject are with the polar cap that powers on, and reference electrode is set;

Step 2, the stimulation programs that basis configures on computer screen, the subject makes the different imaginations according to testing requirements;

Step 3, subject make the various imaginations according to step 2, collect the brain wave of different imagination types;

Step 4, according to the brain wave that above-mentioned steps collects, by the brain wave acquisition instrument, be sent in signal analysis and the converting system, change into control command at last;

Step 41, by the electroencephalogramsignal signal analyzing system, according to the feature that extracts, judge the EEG signals characteristic type of input system;

Step 42, according to the result of 41 steps, the output corresponding action.

2, according to claim 1 based on the game auxiliary control method of imagination brain electricity, it is characterized in that: by the particular stimulation pattern EEG signals is carried out feature extracting method, key step:

Step 1, subject are with the polar cap that powers on, and reference electrode is set;

Step 2, the stimulation programs that basis configures on computer screen, the subject makes the different imaginations according to testing requirements.

Step 3, collect EEG signals and carry out pre-service;

Step 4, utilization signal processing method carry out feature extraction to EEG signals;

The result of step 5, applying step 4 classifies to EEG signals by sorting algorithm

3, according to claim 1 based on the game auxiliary control method of imagination brain electricity, it is characterized in that: in brain electricity input Gamecontrol system (GCS-EEGI), realize that the controlled object of control carries out two dimensional motion, comprises the following steps to realize:

(1), the subject is trained, by stimulus modality training subject, gather subject's EEG signals feature of different imagination processes, because native system mainly is to move on the plane, therefore main the collection led C3, C4 and the Cz electrode that meets under 10/20 international standard according to 64;

(2), subject's using system, at first gather EEG signals, by the eeg signal acquisition instrument, gather EEG signals, the EEG signals input signal analytic system of the C3 that collects, C4 and Cz three conductive electrode;

(3), the electroencephalogramsignal signal analyzing system, according to the 1 EEG signals feature of being extracted, judge the type of input EEG signals feature;

(4), according to the 3 EEG signals characteristic types of judging, export corresponding operational order, and the side-to-side movement of dolly on the control screen.

4, as described in the claim 3 based on the game auxiliary control method of imagination brain electricity, it is characterized in that: comprise that following concrete steps realize:

About the imagination after electrode for encephalograms lays, stimulus modality is to begin to occur one second screen stand-by period, shows a cross, five seconds afterwards, train, the imagination identified about screen occurred, and occurred two seconds random interval time afterwards; This method can realize effectively that controlling controlled object by the issue an order of brain electricity carries out two dimensional motion control; Average accuracy rate can reach 80%, and output speed can reach carries out subcommand output per 10 seconds.

Images