CN102156541B - Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method - Google Patents
Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method Download PDFInfo
- Publication number
- CN102156541B CN102156541B CN2011100899841A CN201110089984A CN102156541B CN 102156541 B CN102156541 B CN 102156541B CN 2011100899841 A CN2011100899841 A CN 2011100899841A CN 201110089984 A CN201110089984 A CN 201110089984A CN 102156541 B CN102156541 B CN 102156541B
- Authority
- CN
- China
- Prior art keywords
- blood oxygen
- signal
- brain
- information
- frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000008280 blood Substances 0.000 title claims abstract description 62
- 210000004369 blood Anatomy 0.000 title claims abstract description 62
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 title claims abstract description 61
- 229910052760 oxygen Inorganic materials 0.000 title claims abstract description 61
- 239000001301 oxygen Substances 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000003993 interaction Effects 0.000 title claims abstract description 16
- 230000002360 prefrontal effect Effects 0.000 title claims abstract description 7
- 210000004556 brain Anatomy 0.000 claims abstract description 80
- 230000009471 action Effects 0.000 claims abstract description 29
- 238000001914 filtration Methods 0.000 claims abstract description 24
- 238000006243 chemical reaction Methods 0.000 claims abstract description 9
- 210000001061 forehead Anatomy 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 14
- 230000003321 amplification Effects 0.000 claims description 12
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 12
- 239000000523 sample Substances 0.000 claims description 12
- 238000003909 pattern recognition Methods 0.000 claims description 11
- 238000001228 spectrum Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 7
- 210000003128 head Anatomy 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 5
- 230000003595 spectral effect Effects 0.000 claims description 5
- 210000003710 cerebral cortex Anatomy 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 210000001015 abdomen Anatomy 0.000 claims description 3
- 230000007423 decrease Effects 0.000 claims description 3
- 230000003760 hair shine Effects 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 19
- 230000008859 change Effects 0.000 abstract description 16
- 238000012545 processing Methods 0.000 abstract description 9
- 238000011160 research Methods 0.000 abstract description 9
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 230000005611 electricity Effects 0.000 description 20
- 238000010521 absorption reaction Methods 0.000 description 11
- 230000001360 synchronised effect Effects 0.000 description 11
- 238000000605 extraction Methods 0.000 description 9
- 238000002329 infrared spectrum Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 210000004761 scalp Anatomy 0.000 description 5
- 210000001519 tissue Anatomy 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 5
- 238000004497 NIR spectroscopy Methods 0.000 description 4
- 210000005013 brain tissue Anatomy 0.000 description 4
- 210000003414 extremity Anatomy 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 206010033799 Paralysis Diseases 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000010183 spectrum analysis Methods 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- XUMBMVFBXHLACL-UHFFFAOYSA-N Melanin Chemical compound O=C1C(=O)C(C2=CNC3=C(C(C(=O)C4=C32)=O)C)=C2C4=CNC2=C1C XUMBMVFBXHLACL-UHFFFAOYSA-N 0.000 description 2
- 238000000342 Monte Carlo simulation Methods 0.000 description 2
- 238000000862 absorption spectrum Methods 0.000 description 2
- 210000001175 cerebrospinal fluid Anatomy 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 230000000763 evoking effect Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000005764 inhibitory process Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004936 stimulating effect Effects 0.000 description 2
- 210000001364 upper extremity Anatomy 0.000 description 2
- GPYKKBAAPVOCIW-HSASPSRMSA-N (6r,7s)-7-[[(2r)-2-amino-2-phenylacetyl]amino]-3-chloro-8-oxo-1-azabicyclo[4.2.0]oct-2-ene-2-carboxylic acid;hydrate Chemical compound O.C1([C@H](C(=O)N[C@@H]2C(N3C(=C(Cl)CC[C@@H]32)C(O)=O)=O)N)=CC=CC=C1 GPYKKBAAPVOCIW-HSASPSRMSA-N 0.000 description 1
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 description 1
- 102000000634 Cytochrome c oxidase subunit IV Human genes 0.000 description 1
- 108050008072 Cytochrome c oxidase subunit IV Proteins 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- 239000011358 absorbing material Substances 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 230000036770 blood supply Effects 0.000 description 1
- 230000008344 brain blood flow Effects 0.000 description 1
- 230000003727 cerebral blood flow Effects 0.000 description 1
- GTKRFUAGOKINCA-UHFFFAOYSA-M chlorosilver;silver Chemical compound [Ag].[Ag]Cl GTKRFUAGOKINCA-UHFFFAOYSA-M 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 108010002255 deoxyhemoglobin Proteins 0.000 description 1
- 230000009699 differential effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 235000013601 eggs Nutrition 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 230000005714 functional activity Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000007659 motor function Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000008557 oxygen metabolism Effects 0.000 description 1
- 230000003071 parasitic effect Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000005501 phase interface Effects 0.000 description 1
- 230000003334 potential effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 210000003625 skull Anatomy 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000000287 tissue oxygenation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Landscapes
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to the technical field of brain-computer interfaces, provides a novel research method for realizing recognition in different action modes by combining change characteristics of prefrontal electroencephalogram information and brain blood oxygen information induced by imaginary action, and designs a novel human-computer interaction device for synchronously acquiring the change characteristics of the two types of information. The technical scheme adopted in the invention is that: the prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method is realized by means of a near-infrared light source, a photodetector, a filtering and post-amplifying circuit, an electroencephalogram electrode, an electroencephalogram amplifier and a computer, wherein the computer is used for carrying out signal conversion and processing to obtain a result. The method is mainly applied to designing and manufacturing of a prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction device.
Description
Original bill application title: the human-computer interaction device of forehead EEG and blood oxygen information fusion, the applying date: on May 13rd, 2010, application number: 201010171583.6
Technical field
The present invention relates to the brain-computer interface technical field, specifically, relate to the human-computer interaction device of forehead EEG and blood oxygen information fusion.
Background technology
For the first time (Brain-Computer Interface, BCI) definition of the BCI that provides of international conference is: " BCI is a kind of communication control system that does not rely on brain nervus peripheralis and the normal output channel of muscle to brain-computer interface." in the present achievement in research; it mainly is through gathering and analyze different conditions servant's EEG signals; use certain engineering means between human brain and computing machine or other electronic equipment, to set up direct the interchange and control channel then; thus realize a kind of brand-new message exchange and control technology, can particularly those have lost basic extremity motor function but the patient that has a normal thinking provides a kind of approach that carries out information interchange and control with the external world for the disabled person.Promptly can not need language or limb action, directly express wish or handle external device through control brain electricity.For this reason, the BCI technology also more and more comes into one's own.
Up to the present, the most frequently used BCI system mostly is based on brain electric information, has occurred the BCI system based on brain blood oxygen information in recent years.Its know-why is consistent.As shown in Figure 1; Containing the brain electric information (blood oxygen information) that controls intention obtains from scalp or encephalic through electrode (detector); Extract brain electric information (the blood oxygen information) characteristic that the reflection user is intended to through signal Processing, and it is converted into the operational order of control external unit.And the main application target of BCI research is to help the disabled person of the serious paralysis of limbs to handle and use peripheral daily life instrument, to realize information interchange and device control to external world.
Summary of the invention
For overcoming the deficiency of prior art; Purport of the present invention is to propose a kind of new will imagine to move the forehead EEG that brings out and combine with the variation characteristic of brain blood oxygen two category informations and realize the research method to the identification of different patterns, and the man-machine interaction's device that designs a kind of new this two category informations variation characteristic of synchronous acquisition.
For achieving the above object, the technical scheme that the present invention adopts is: the man-machine interaction method of forehead EEG and blood oxygen information fusion, it is characterized in that, and realize by means of following array apparatus:
The near infrared light light source is used for the luminous cerebral cortex that shines;
Photodetector is used to survey the light intensity of reflected back, and the variation that intensity signal takes place reflects the variation of brain blood oxygen indirectly;
Filtering and back level amplifying circuit are used for the signal of photodetector output is carried out filtering and amplification, and the signal after filtering and the amplification is input to computing machine through data collecting card;
Electrode for encephalograms is used for probe power spectrum ratio decline ERD signal;
Eeg amplifier is used for electrode for encephalograms output signal is amplified and filtering, and the signal after filtering and the amplification is input to computing machine through data collecting card;
Said method comprises the following steps:
The data that capture card are input to computing machine are carried out time frequency analysis-Short Time Fourier Transform analysis, are used for obtaining with the Short Time Fourier Transform method the two-dimentional time-frequency collection of illustrative plates of EEG signals;
Separability between the imagination action of different mode is made Fisher to be analyzed; Obtain the two-dimentional time-frequency collection of illustrative plates of Fisher coefficient; Find obviously bigger position time corresponding section and frequency band of Fisher coefficient from collection of illustrative plates, the time frequency window of obtaining like this and it obtain power spectral density value as a characteristic of extracting after corresponding to the time frequency window of the two-dimentional time-frequency figure that Short Time Fourier Transform obtains;
Be used for digital signal is carried out the Discrete Fourier Transform analysis, through after the conversion, the brain blood oxygen signal of time domain converts frequency-region signal into, from the frequency-region signal that obtains, just seeks the characteristic of frequency band on the spectrogram;
The characteristic of extracting is used for training the svm classifier device; Obtain a model after the training; And then utilizing this model to come the imagination action of unknown pattern type is classified, the result who obtains is the pattern-recognition result of unknown pattern imagination action and obtains recognition correct rate.
The near-infrared wavelength that the near infrared light light source is sent is 760nm and 850nm.
Said filtering and the back level amplifying circuit in the level amplifying circuit of back are adopted the in-phase proportion computing circuit of adjustable gain; Adopt the second order Butterworth LPF to carry out filtering.
With the back of the body outside and the part abdomen outside brain district of electrode for encephalograms attached to adult head prefrontal lobe cortex, the outside all covers surface remainder except that light source and detector with special light screening material, and the source gauge length is set at 2.89cm.
Characteristics of the present invention are: experimentation carries out on forehead; In the time of handled easily; Also avoided the influence of hair and scalp, also can let health paralysis but necessity that the normally functioning disabled person of brains has washed hair before and after when using this device, having avoided using signals collecting.The brain-computer interface system that further research can improve is expected to obtain considerable social benefit and economic benefit.
Description of drawings
Figure 1B CI system and control thereof.
The structured flowchart of Fig. 2 apparatus of the present invention.
Fig. 3 HbO2 and Hb are at the absorption coefficient near infrared spectrum district.
Fig. 4 Monte Carlo simulation synoptic diagram.
The design of Fig. 5 constant-current source circuit.
Fig. 6 forehead sniffer layout.
Level amplification filtering circuit diagram behind Fig. 7.
Fig. 8 imagines psychomotor task period distribution plan.
Fig. 9 imagines psychomotor task quiescent stage interface.
Figure 10 imagines psychomotor task presentation period interface.
Figure 11 imagines psychomotor task imagination phase interface.
Figure 12 tests the FB(flow block) of capture program.
Embodiment
Invention has designed a kind of based on the brain electricity of imagination action and man-machine interaction's device of brain blood oxygen synchronous acquisition.The normal person can produce us in the corresponding function district and be referred to as relevant phenomenon (the Event Related Desynchronization that desynchronizes of incident when imagining that the right-hand man moves; ERD) brain electrical feature signal; And the while also can cause brain blood oxygen change in information; This variation can be passed through near-infrared spectrum technique, and (Near-infrared spectroscopy NIRS) records.This device will imagine with regard to design that the variation characteristic of these two types of signals of causing of action combines the imagination action of different mode will be discerned, and in the hope of can realizing both synchronous and combine, and obtain higher recognition accuracy.Its techniqueflow is: what a is built can be at the device that head part's forehead region is electric to brain simultaneously and brain blood oxygen change information is gathered; Then under experimental system instructs; Acquisition operations person's the brain electricity and the brain blood oxygen signal data that when carrying out the action of the different mode imagination, produce; With carrying out certain pre-service, time frequency analysis, the analysis of Fisher separability again after its storage; Use SVMs or HMM that the data of different mode are classified at last, obtain higher accuracy.
The present invention has selected for use ERD characteristic and changing features in the brain blood oxygen information in the brain electric information as the characteristic control signal.Dynamic EEG signals characteristic spectra power spectrum density will change when the people imagines limb action, and wherein the decline of power spectrum ratio is called ERD, and is the most obvious at 10-12Hz usually; Oxygenated blood red eggs in cerebral blood flow concentration white and deoxyhemoglobin changes simultaneously, can extract the spectral intensity variation characteristic that change in concentration causes.Can utilize thus ERD phenomenon that brain caused when imagining action thinking and blood oxygen variation characteristic phenomenon as thinking activities to stimulating the sign of incident effective response.Be different from traditional brain electric installation, the present invention will imagine that brain electricity ERD phenomenon and two kinds of different character information of blood oxygen variation phenomenon that action is brought out organically merge, and have designed a kind of novel compound device.Place the Ag-Agcl electrode through position relative with C3, C4 on experimenter's forehead and detect brain electricity ERD characteristic; Simultaneously on forehead, place near-infrared light source and detector detecting blood oxygen change in information, be used as the criteria for classification of discerning different patterns with brain blood oxygen change in information characteristic thereby extract the brain electricity of experimenter when imagining action by certain layout.The experimental implementation process of this device is carried out at forehead, and the experimental implementation of head brain electric installation wants simple relatively, and the problem that only needs to solve two category information synchronous acquisition gets final product.
Fig. 2 is the structural representation of apparatus of the present invention.This design comprises electrode for encephalograms and eeg amplifier, near-infrared light source and detector and light source constant-current driving, and the back level is amplified, filtering circuit, parts such as data collecting card, computing machine.The experimental arrangement of gathering brain electricity and brain blood oxygen information designs under the Labview platform, the synchronous acquisition of brain two category informations when using multichannel data collecting card to realize imagination action.The experimenter imagines psychomotor task accordingly by the indication of the experiment routine on the computer screen; Brain electricity and brain blood oxygen information all can produce corresponding variation when executing the task: EEG signals produce at cerebral cortex; After surveying by electrode for encephalograms through eeg amplifier amplify, filtering, get into data collecting card then and carry out importing computing machine after the analog to digital conversion; Brain blood oxygen signal then is after light source luminescent shines cerebral cortex; Survey the variation that variation that the intensity signal of reflected back takes place reflects brain blood oxygen indirectly by detector, this signal through follow-up amplification, filtering after import computing machine after the data collecting card analog to digital conversion.Collect to such an extent that brain electricity and brain blood oxygen information data are extracted its characteristic signal when the imagination psychomotor task through follow-up data processing again, thereby with the pattern-recognition of these feature application similar experimental duties when pattern is imagined in the unknown.
Main points of the present invention are that near-infrared spectrum technique measures sport technique segments such as feature extraction and the pattern-recognition of the design of the design of brain electricity that the principle of brain blood oxygen information change, imagination action bring out and brain blood oxygen variable signal harvester, synchronous acquisition experimental program, experimental data.
1 near-infrared spectrum technique is measured the principle of brain blood oxygen information change
Near-infrared spectrum technique: Near-Infrared Spectroscopy (NIRS) is meant a kind of technology of learning substance characteristics and concentration etc. through the research material at the absorption spectrum of near-infrared region.Its principle that detects the variation of brain blood flow is: the supply of blood flow of brain can produce local acknowledgement along with the localized variation of functional activity.When brain is in state of activation, can causes that LCBF and oxygen metabolism rate change, thereby cause the variation of blood oxygen concentration in the respective regions.
Tissue presents height forward scattering and low absorption characteristic near infrared light 700-900nm, makes photon can penetrate several centimetres the degree of depth; HbO2 and Hb have notable difference at the absorption coefficient near infrared spectrum district.HbO2 in the brain blood oxygen information and the change in concentration of Hb are calculated (revised dual wavelength Lambert-Beer law) by the variation of respective wavelength place light intensity.
(1), choosing of optimal wavelength:
How two wavelength mentioning in the above-mentioned principle this choose, and is the problem that at first should consider.In the adult brain tissue, the volume fraction of water is very high, so the absorption of water detects near infrared blood oxygen very big influence is arranged.And the optical absorption of water is relatively low at 700-900nm (" light window " district).The visual constant absorbing material that fixes of water in this light window just can be used as constant and consider.The absorption characteristic of cerebrospinal fluid and water are worked as in the actual clinical, also are used as constant.Other compositions in the brain tissue do not change with the oxygen supply situation like these compositions such as fat, melanin, cytochrome oxidases basically, can regard the stable absorption body as in the early-stage Study stage, thus optimal wavelength choose main according to Hb and HbO2 spectral characteristic.
The near infrared absorption characteristic of above-mentioned two types of haemoglobins is seen Fig. 3.At the optical window oral region, all there is peak value in both absorption spectrums.On scheming, can find out: at the 760nm wavelength, Hb absorbs and reaches peak value; At the 850nm wavelength, HbO2 reaches absorption peak.Therefore, we choose 760nm and these two wavelength of 850nm near infrared light to be used for brain blood oxygen detection of information.
(2), confirming of source gauge length:
The source gauge length is meant the horizontal range between light source and the target test point.In the design, promptly refer to the horizontal range between near-infrared light source and the photo-detector.Document shows that the source gauge length is very big to the penetration depth influence of light beam, therefore has only the reflective measurement purpose that just can reach head tissue.
Tissue greater than absorption, belongs to strong scattering medium to the near infrared scattering of light.The propagation of single photon in tissue is at random, the path that does not exist definite photon to advance.But still there is the average migration rule on the probability meaning in a large amount of photons, and this can obtain through the method for Monte Carlo simulation, and is as shown in Figure 4.The result proves that photon can be set under the situation of 2.89cm at source gauge length (r among Fig. 4), penetrates scalp, skull and cerebrospinal fluid layer behind the 1cm at least, surveys corticocerebral blood oxygen information.
The brain electricity that 2 imagination actions are brought out and the design of brain blood oxygen variable signal harvester
(1), light source and light source constant-current driving:
The model that adopts EPITEX company to produce is that the three-wavelength light emitting diode of SMT-L760/805/850 is as light source.Chip internal is integrated measures LED required 760,805, three wavelength of 850nm, and HbO2 and Hb are to the differential effect of different wave length light absorption when guaranteeing that tissue oxygenation changes.Adopt the integrated light source of multi-wavelength not only can optimize probe size, and can fully eliminate common discrete type light emitting diode because the discrete influence that measurement result is caused in locus.
The light emitting diode operate as normal of this model need give stable electric current power supply, so need utilize voltage reference design constant-current source circuit.The design adopts stabilivolt as voltage reference, forms constant current source in conjunction with triode, and is as shown in Figure 5.The electric current computing formula is: I=(Vd-Vbe)/R1.The constant current of output can be supplied power for power supply like this, satisfies current requirements through selecting R1.Wherein the 760nm light emitting diode needs the power supply of 75mA electric current, and the 850nm light emitting diode needs the power supply of 100mA electric current.
(2), detector:
The OPT101 that adopts the production of BURR-BROWN company is as the optical signal detection unit; This device inside is an integrated avalanche photodide and prime amplifier; Reduced many FAQs of discrete component, as: the error of leakage current generating, the noise that stray capacitance produces and the distortion etc. that gains.
(3), electrode for encephalograms:
The silver-silver chloride electrode that native system adopts Spes Medica company to produce picks up the forehead EEG signal as sensor.Eeg signal acquisition uses the unipolar lead mode, and (what advantage was that each leads record is the absolute value of electrode place scalp electrical potential activity; Be convenient to the later stage signal Processing); And with the public connection electrode for referencial use of ears, the latter is also as the earth terminal input amplifier simultaneously.
(4), topological design:
The design covers the back of the body outside and the brain district, the part abdomen outside of adult head prefrontal lobe cortex, and the outside all covers surface remainder except that light source and detector with special light screening material, and is as shown in Figure 6.Red point and white point are represented light source and position of detector respectively, and according to analyzing before, the source gauge length is set at 2.89cm, thereby obtain the layout like figure; Green point is then represented the position of electrode for encephalograms.Light screening material can reach the interference of getting rid of extraneous parasitic light, prevent that near infrared light is directly coupled on the detector without tissue attenuation, avoids the direct contacting skin of device and guarantees testee's safety.
(5), back level amplification, filtering circuit design:
The various signal amplitudes of human body all are smaller and the noise that mixes easily, need amplify and Filtering Processing.Electrode for encephalograms collects to such an extent that EEG signals directly get into eeg amplifier (the 9216SM type EEG signal amplifier that SYMTOP company produces), in amplifier, accomplishes and amplifies and filtering; And the blood oxygen signal also needs further amplification of back level and filtering after being detected by detector.This design adopts the in-phase proportion computing circuit of adjustable gain as back level amplifying circuit, and it can avoid the feeble signal distortion of prime output to improve common-mode rejection ratio simultaneously; Reduce output resistance, be convenient to analog input end cascade with data collecting card.Adopt the second order Butterworth LPF to increase the inhibition ability of system to high frequency noise, the noise signal more than the 10KHz is had great inhibition, therefore the electromagnetic interference (EMI) for most of spaces can be good at getting rid of.Specifically referring to Fig. 7.
Through amplification with filtered brain electricity and brain blood oxygen signal all with the analog input end cascade of data collecting card (USB-6251 of NI company production), after analog to digital conversion, be input to and form data file in the computing machine.
The design of 3 synchronous acquisition experimental programs
After hardware system is built completion, will improve the soft support of hardware system, just test capture program, scheme.Under the Labview platform, write the experiment capture program, realize the multiple input path synchronous acquisition brain electricity and the brain blood oxygen signal of Usage data collection card.The process of single experiment is as shown in Figure 8, wherein imagines 28 seconds psychomotor task times spent each time, comprises four continuous record slots (Fig. 8): first period (0-8 second) is quiescent stage, and the experimenter is in quiet attonity state (Fig. 9) in this period; Second period (9-10 second) is a presentation period, and system located in the 8th second to light lamp that is labeled as imagination left hand or the lamp of imagining the right hand at random constantly, pointed out as the imagination mode of this experiment.If prompt for imagination left hand, the experimenter imagines that in the 3rd period own left upper extremity does the motion of crooking one's arm; If prompt for the imagination right hand, then the experimenter imagines that in the 3rd period own right upper extremity does the motion (Figure 10) of crooking one's arm; The 3rd period (11-20 second) is the imagination phase, and the lantern festival that is labeled as beginning in the 10th second is constantly lighted, and the experimenter imagines action (Figure 11) according to the given imagination mode of presentation period; The 4th period (21-28 second) is quiescent stage, is the intermittent phase of the imagination between the phase, and this period, the same all warning lights with first period can extinguish.
Writing of this experiment capture program is that flow process according to block diagram shown in figure 12 is carried out; By signal acquisition module the brain electricity and the brain blood oxygen signal of input capture card analog input end are gathered; After data preprocessing module, be stored as the data that have label, these data are handled in follow-up pattern recognition module again.
The feature extraction of 4 experimental datas and pattern-recognition
4.1 the feature extraction of eeg data
(1), time frequency analysis-Short Time Fourier Transform
The brain electricity ERD phenomenon that causes owing to imagination action has specific generation frequency range and stimulates the regular time delay with respect to bringing out, and therefore can adopt Short Time Fourier Transform to combine the method for superposed average that spectrum component variation and characteristic distributions before and after stimulating are analyzed.Imply the stationarity hypothesis for processing signals in the spectral analysis method commonly used, the spectrum component that is to say the hypothesis signal is to distribute together on whole time shaft, and for the dynamic EEG signals of bringing out of time correlation, this hypothesis obviously is false.To this situation; Alternative method is to use the Time-Frequency Analysis Method that can extract time-domain and frequency-domain information simultaneously; Fourier analysis in short-term is one of Time-Frequency Analysis Method commonly used at present; Its hypothesis EEG signals has stationarity in short-term to a certain degree, that is to say that the spectrum distribution formula of signal in a limited time window is constant.
The method of Short Time Fourier Transform is at first to use the view window W (t) of a finite width that signal x (t) is observed, and then the signal after the windowing is carried out that Fourier transform obtains,
Here ω is an angular frequency, W
*(τ-t) is the W (complex conjugate function of τ-t).
When the view window of limited value length along the time shaft translation, can on the time-frequency plane of two dimension, obtain the time dependent information of spectrum distribution of signal, can obtain the two-dimentional time-frequency collection of illustrative plates of EEG signals like this.
(2), the Fisher separability is analyzed and feature extraction
The separability analysis is the common method in the feature extraction, and it mainly is used for the distribution of evaluating characteristic parameter in different classes of sample and whether has evident difference, and the more parameters that in general diversity factor is big more is suitable for the eigenwert as the classification of sample.
For imagination action evoked brain potential signal; The corresponding one type of sample of one type psychomotor task EEG signals; If EEG signals have the obvious difference that gets in the distribution of the power spectrum density on certain frequency range between a generic task sample and other sample, then this frequency range is the specificity frequency range of such imagination psychomotor task evoked brain potential signal.Have multiplely for the evaluation method of sample parameter separability, and the Fisher discriminatory analysis has been called one of effective method the most of feature extraction.Can select the Fisher evaluation function for use here, it defines as follows:
M wherein
1With m
2Be respectively the average of two category features, σ
1With σ
2Be the variance of two category features, the Fisher evaluation function be actually between the class of eigenwert dispersion with type in the ratio of dispersion, J more greatly then separability is high more, if the average of two category features equates that then J is 0, that is to say that two category feature linearities are inseparable.
When the separability between the imagination action of different mode is done the Fisher analysis; We obtain the two-dimentional time-frequency collection of illustrative plates of Fisher coefficient; We can see obviously bigger position time corresponding section and frequency band of Fisher coefficient from the collection of illustrative plates, the time frequency window of obtaining like this and it correspond to obtain behind the time frequency window of the two-dimentional time-frequency figure that Short Time Fourier Transform obtains power spectral density value just can be used as we extract a characteristic.On Fisher coefficient time-frequency figure, seek relative other time frequency windows of more Fisher coefficient and want obviously big time frequency window, just can find the characteristic that more can be used for pattern-recognition.
4.2 the feature extraction of brain blood oxygen signal
(1), Hb and HbO2 change in concentration-detector magnitude
The principle of near-infrared spectrum technique measurement blood oxygen signal has been introduced in the front; Just under imagination operating state; The brain tissue blood supply increases during near infrared light (760nm and 850nm) irradiation brain tissue, and the light of Hb in the blood and HbO2 absorption is than following absorb many of quiescent condition so.And the output of detector (signal that also promptly collects) is the voltage signal of reaction light intensity, and the magnitude that embodies change in concentration is like this just arranged on the amplitude of voltage signal, just collects to such an extent that voltage magnitude is little during than tranquillization under the imagination state.
(2), spectrum analysis
So far, EEG Processing is the main means of research brain rule.The EEG signals that directly obtain are time domain variable signals, are the functions of time.Yet many key characters of bio signal (comprising brain blood oxygen signal) can not show in time domain, and the method for handling with time domain can't extract these characteristics.In order to extract these characteristics, the most frequently used means are exactly to change through Fourier at present, convert time-domain signal to frequency-region signal, just will carry out spectrum analysis to brain blood oxygen signal.
Become digital signal after the processing such as the brain blood oxygen signal process filtering that we collect, amplification, A/D conversion, digital signal is just analyzed to be carried out Discrete Fourier Transform.Through after the conversion, the brain blood oxygen signal of time domain converts frequency-region signal into, and our frequency-region signal from obtaining is just sought the characteristic of frequency band on the spectrogram.
4.3 pattern-recognition-SVMs
Pattern-recognition is through brain electricity and brain blood oxygen signal Feature Extraction are picked out wherein entrained independence action message with classification.SVMs (SVM) is the new tool that in pattern-recognition and machine learning field, occurs in recent years; With the Statistical Learning Theory is the basis; Avoid effectively crossing the problem that traditional classifications such as study, dimension disaster, local minimum exist in the classical learning method, under condition of small sample, still have good model ability.It makes to the error in classification of unknown sample minimum through the optimum lineoid of structure.
The process of pattern-recognition is following: after having passed through feature extraction phases; We are used for training the svm classifier device with these characteristics of from sample, extracting; Obtain a model after the training; And then utilizing this model to come the imagination action of unknown pattern type is classified, the result who obtains is the pattern-recognition result and the recognition correct rate of unknown pattern imagination action.
Beneficial effect
Synchronous acquisition brain electricity and the human-computer interaction device of brain blood oxygen signal and the experiment that experimental arrangement is imagined right-hand man's psychomotor task according to above our design; We gather the experimental data of 80 subtasks to the experimenter; Be divided into 4 groups, every group 20 times, between the two adjacent groups 4 minutes at interval.To collecting to such an extent that all data are carried out above-mentioned analysis and extracted characteristic, use 3 groups of data to carry out the training of svm classifier device then, 1 group of data is used for the accuracy rate of testing classification device classification.Under only with the situation of EEG signals as characteristic of division; The classification accuracy average level that we obtain is 81.6%; Along with increase (increasing the is not special obvious characteristics) accuracy rate of brain electricity unique characteristics reduces on the contrary; And after the characteristic that has merged brain blood oxygen signal, carry out Classification and Identification again, accuracy rate can reach more than 90%.
This design of experimental result explanation can realize the synchronous acquisition to brain electricity and brain blood oxygen signal, and can in the blood oxygen signal, find the right-hand man is imagined the obvious characteristic that fine classification is carried out in action through the processing to experimental data; Explain also that simultaneously the recognition correct rate that has merged dissimilar obvious characteristics is higher than only using the recognition accuracy that type same characteristic features number obtains in.
The present invention has designed a kind of based on the brain electricity of imagination action and man-machine interaction's device of brain blood oxygen synchronous acquisition, to realize the applied research of different information fusion characteristics in the brain-computer interface technology.The operating process of this invention is carried out on forehead, in the time of handled easily, has also avoided the influence to signals collecting of hair and scalp, also can let health paralysis but trouble that the normally functioning disabled person of brains has washed hair before and after when using this device, having avoided using.The brain-computer interface system that further research can improve is expected to obtain considerable social benefit and economic benefit.
Claims (4)
1. the man-machine interaction method of forehead EEG and blood oxygen information fusion is characterized in that, realizes by means of array apparatus down:
The near infrared light light source is used for the luminous cerebral cortex that shines;
Photodetector is used to survey the light intensity of reflected back, and the variation that intensity signal takes place reflects the variation of brain blood oxygen indirectly;
Filtering and back level amplifying circuit are used for the signal of photodetector output is carried out filtering and amplification, and the signal after filtering and the amplification is input to computing machine through data collecting card;
Electrode for encephalograms is used for probe power spectrum ratio decline ERD signal;
Eeg amplifier is used for electrode for encephalograms output signal is amplified and filtering, and the signal after filtering and the amplification is input to computing machine through data collecting card;
Said method comprises the following steps:
(1) EEG signals that capture card are input to computing machine carry out the Short Time Fourier Transform analysis in the time frequency analysis, obtain the two-dimentional time-frequency collection of illustrative plates of EEG signals with the Short Time Fourier Transform method;
Separability between the imagination action of different mode is made Fisher to be analyzed; Obtain the two-dimentional time-frequency collection of illustrative plates of Fisher coefficient; Find obviously bigger position time corresponding section and frequency band of Fisher coefficient from collection of illustrative plates, the time frequency window of obtaining like this and it obtain power spectral density value as a characteristic of extracting after corresponding to the time frequency window of the two-dimentional time-frequency figure that Short Time Fourier Transform obtains;
(2) brain blood oxygen signal is carried out the Discrete Fourier Transform analysis, through after the conversion, the brain blood oxygen signal of time domain converts frequency-region signal into, from the frequency-region signal that obtains, just seeks the characteristic of frequency band on the spectrogram;
(3) characteristic of extracting is used for training the svm classifier device; Obtain a model after the training; And then utilizing this model to come the imagination action of unknown pattern type is classified, the result who obtains is the pattern-recognition result of unknown pattern imagination action and obtains recognition correct rate.
2. the man-machine interaction method of a kind of forehead EEG according to claim 1 and blood oxygen information fusion is characterized in that, the near-infrared wavelength that the near infrared light light source is sent is 760nm and 850nm.
3. the man-machine interaction method of a kind of forehead EEG according to claim 1 and blood oxygen information fusion is characterized in that, said filtering and the back level amplifying circuit in the level amplifying circuit of back is adopted the in-phase proportion computing circuit of adjustable gain; Adopt the second order Butterworth LPF to carry out filtering.
4. the man-machine interaction method of a kind of forehead EEG according to claim 1 and blood oxygen information fusion; It is characterized in that; With the back of the body outside and the part abdomen outside brain district of electrode for encephalograms attached to adult head prefrontal lobe cortex; The outside all covers surface remainder except that light source and detector with special light screening material, and the source gauge length is set at 2.89cm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100899841A CN102156541B (en) | 2010-05-13 | 2010-05-13 | Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100899841A CN102156541B (en) | 2010-05-13 | 2010-05-13 | Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101715836A Division CN101853070B (en) | 2010-05-13 | 2010-05-13 | Man-machine interaction device for information fusion of forehead EEG and blood oxygen |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102156541A CN102156541A (en) | 2011-08-17 |
| CN102156541B true CN102156541B (en) | 2012-05-23 |
Family
ID=44438064
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100899841A Active CN102156541B (en) | 2010-05-13 | 2010-05-13 | Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102156541B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102512160B (en) * | 2011-12-16 | 2013-05-15 | 天津大学 | Electroencephalogram emotional state feature extraction method based on adaptive tracking in different frequency bands |
| CN102715889B (en) * | 2012-06-11 | 2014-01-29 | 天津大学 | Mental load detection method |
| CN103271802A (en) * | 2013-05-24 | 2013-09-04 | 桂林电子科技大学 | Wheelchair system based on control of FNIRI and EEG |
| US10595741B2 (en) | 2014-08-06 | 2020-03-24 | Institute Of Automation Chinese Academy Of Sciences | Method and system for brain activity detection |
| CN104380297B (en) * | 2014-09-19 | 2017-01-18 | 中国科学院自动化研究所 | Data storage method for photoelectric synchronous brain activity recording |
| CN104224466A (en) * | 2014-09-25 | 2014-12-24 | 桂林电子科技大学 | Wheelchair control system based on detection of brain near-infrared function information |
| WO2016145607A1 (en) * | 2015-03-17 | 2016-09-22 | Bayerische Motoren Werke Aktiengesellschaft | Interaction between user and interactive device |
| CN109925582B (en) * | 2019-03-20 | 2021-08-03 | 天津大学 | Dual-mode brain-computer interactive motor nerve feedback training device and method |
| CN110680282B (en) * | 2019-10-09 | 2020-10-27 | 黑龙江洛唯智能科技有限公司 | Method, device and system for detecting temporary abnormal state of brain |
| CN112043287B (en) * | 2020-09-30 | 2021-07-20 | 重庆大学 | Noninvasive monitoring method and monitoring device for cerebral blood oxygen |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101049236A (en) * | 2007-05-09 | 2007-10-10 | 西安电子科技大学 | Instant detection system and detection method for state of attention based on interaction between brain and computer |
| CN101853070A (en) * | 2010-05-13 | 2010-10-06 | 天津大学 | Man-machine interaction device for information fusion of forehead EEG and blood oxygen |
-
2010
- 2010-05-13 CN CN2011100899841A patent/CN102156541B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101049236A (en) * | 2007-05-09 | 2007-10-10 | 西安电子科技大学 | Instant detection system and detection method for state of attention based on interaction between brain and computer |
| CN101853070A (en) * | 2010-05-13 | 2010-10-06 | 天津大学 | Man-machine interaction device for information fusion of forehead EEG and blood oxygen |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102156541A (en) | 2011-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101853070B (en) | Man-machine interaction device for information fusion of forehead EEG and blood oxygen | |
| CN102156541B (en) | Prefrontal electroencephalogram information and blood oxygen information fused human-computer interaction method | |
| Scholkmann et al. | A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology | |
| Zhai et al. | Stress detection in computer users based on digital signal processing of noninvasive physiological variables | |
| CN104720797B (en) | One kind is based on myoelectricity noise cancellation method in single pass EEG signals | |
| CN1247148C (en) | Child cognitive function development testing system | |
| CN101627909A (en) | Digital amplitude-integrated cerebral function monitor | |
| EP2696749A2 (en) | Diagnostic measuring apapratus with integrated spectrometer | |
| EP3758606B1 (en) | A method of selecting the intensity of a light source for monitoring an analyte in blood, and a device thereof | |
| CN103405225B (en) | A kind of pain that obtains feels the method for evaluation metrics, device and equipment | |
| CN102284137A (en) | Functional electrical stimulation multisource information fusion control method | |
| CN107644682A (en) | Mood regulation ability based on frontal lobe EEG lateralities and ERP checks and examine method | |
| US20200121206A1 (en) | Electroencephalography device and device for monitoring a subject using near infrared spectroscopy | |
| CN112450879A (en) | Near infrared spectrum detection method for modulating cognitive function of brain through ambient light | |
| Frigo et al. | Measuring cerebral activation from fNIRS signals: An approach based on compressive sensing and Taylor–Fourier model | |
| Poeppel et al. | Electromagnetic recording of the auditory system | |
| US20190029550A1 (en) | Method for detecting elements of interest in electrophysiological signals and detector | |
| CN202342031U (en) | Animal-event-related potential recording system | |
| CN101856221A (en) | Detection system for determining damage degree and damage positions of spinal cord | |
| CN105662389A (en) | System for detecting and evaluating human-brain activity | |
| CN103300849A (en) | Electroencephalogram signal processing method | |
| Lee et al. | A study on EEG based concentration power index transmission and brain computer interface application | |
| CN101716074A (en) | Evoked potential recorder based on time characteristic indicators | |
| Wang et al. | Classification of ADHD using fNIRS signals based on functional connectivity and interval features | |
| Zhai et al. | User stress detection in human-computer interactions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210226 Address after: Room 50, room 03, 8 / F, phase I, zhikongjian Plaza, Southeast of the intersection of Huizhi North Road and Huizhi Ring Road, Dongli Lake, Tianjin Patentee after: Yuxi Technology (Tianjin) Co.,Ltd. Address before: 300072 Tianjin City, Nankai District Wei Jin Road No. 92 Patentee before: Tianjin University |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240428 Address after: 12th Floor, Haitai Building, No. 6 Meiyuan Road, Huayuan Industrial Zone, Binhai New Area, Tianjin, 300392 Patentee after: Zhongdian Yunnao (Tianjin) Technology Co.,Ltd. Country or region after: China Address before: Room 50, room 03, 8 / F, phase I, zhikongjian Plaza, Southeast of the intersection of Huizhi North Road and Huizhi Ring Road, Dongli Lake, Tianjin Patentee before: Yuxi Technology (Tianjin) Co.,Ltd. Country or region before: China |
|
| TR01 | Transfer of patent right |