CN102715911B - Brain electric features based emotional state recognition method - Google Patents

Abstract

The invention discloses a brain electric features based emotional state recognition method. The method comprises the following steps of: data acquisition stage: under the condition of international emotional picture induction, extracting 64 brain electric data which is tested under the induction of different-happiness-level pictures; data pretreatment stage: carrying out four stages of reference electric potential variation, down sampling, band-pass filtering, electro-oculogram removal on the collected 64 brain electric data; feature extraction stage: extracting time domain features after signals after pretreatment are filtered by a common space model algorithm; and feature recognition: recognizing the features by using a support vector machine classifier, and differentiating different emotional states. According to the method, an OVR (one versus rest) common space model algorithm is used for removing the interference of background signals, and is used for the signal intensification of multiple types of emotion induced brain electricity; after the background signals are removed, the differences among different types of emotional brain electricity are intensified, the recognition accurate ratio of subjects is relatively ideal when the recognition is carried out by the time domain variance features, and the emotions of different happiness can be differentiated accurately.

Description

Emotional state recognition methods based on brain electrical feature

Technical field

The present invention relates to a kind of emotional state recognition methods.Particularly relate to the emotional state recognition methods based on brain electrical feature that the neural feedback of a kind of diagnosis that can be used for clinical dysthymic disorder's disease and therapeutic evaluation, emotion regulates.

Background technology

Emotion is the Premium Features of human brain, is guaranteeing organic existence and adaptation, and individual study, memory, decision-making are had to important impact.Emotion is also the source of individual variation, is the key component of many personal characteristics and mental pathology.Along with social development, each age, each field people's emotional disturbance is more and more, more and more serious, and the various disease incidences relevant to emotion are more and more higher, as depression, manic disorder, anxiety neurosis, obsession, affective disorder etc.Correct identification to emotion and adjusting, to improving human lives's quality, ensure that physical and mental health is significant.

At present domestic and international researcher is identified people's emotion by expression, voice, postural cue, use various modes to know method for distinguishing and obtained some effects, but due to the easy control of signal and the property pretended, result cannot be got rid of the impact of tested subjective factors, sometimes cannot observe potential, real affective state.Emotion is the result of nervous process coaction under cerebral cortex and cortex, and transient behavior is strong.Brain electricity is the spontaneous discharge activities that is not subject to manual control, there is temporal resolution height and simple and easy to do advantage, thereby to utilize brain electricity to identify emotion and disclose its complicated neuromechanism be feasible, diagnosis and treatment evaluation that Emotion identification research based on EEG is dysthymic disorder's disease provide new approach, simultaneously also for the neural feedback of negative emotions regulates and training provides scientific comprehensive theoretical basis.Emotion research based on brain electricity has received increasing concern in recent years.

Emotion identification based on brain electricity research at present, discrimination waits to improve.Because this class emotion research is carried out mostly under induced conditions, in evoked brain potential, except the brain electricity composition relevant to emotion, also contain a large amount of background signals, for example, in the evoked brain potential based on Emotional Picture, just contain some visual evoked potentials, spontaneous brain electricity etc.Effective extraction of the brain electrical feature relevant to emotion has been disturbed in the existence of background signal, has affected to a certain extent discrimination.

Summary of the invention

Technical problem to be solved by this invention is, provide a kind of by cospace Pattern Filter method, removal background signal disturbs, strengthen the EEG signals relevant to emotion, extract more effective feature improve multiclass emotion evoked brain potential discrimination the emotional state recognition methods based on brain electrical feature.

The technical solution adopted in the present invention is: a kind of emotional state recognition methods based on brain electrical feature, comprised as the next stage:

(1) data acquisition phase, under the international Emotional Picture induced conditions of described data acquisition Shi, extracts and testedly under the joyful degree grade of difference picture brings out, 64 leads eeg data;

(2) data preprocessing phase,

To collect 64 processing of leading eeg data and change reference potential, down-sampled, bandpass filtering, four steps of removal eye electricity;

(3) feature extraction phases,

Described feature extraction phases is that pretreated signal cospace pattern algorithm is carried out extracting temporal signatures after filtering;

(4) feature identification

After feature extraction, use support vector machine classifier to identify feature, different emotional states are distinguished.

Described data acquisition phase comprises carries out following steps:

1) experimental design: selected 45 pictures from international Emotional Picture storehouse, be divided into 3 grades by joyful degree scope, each grade 15 pictures; Grade 1,2,3 is corresponding passiveness, neutral, the active mood picture of representing respectively; Comprise 45 subtasks, stimulate task time is 14 seconds at every turn, has three periods, and picture presents period A, rest period B and reminds period C;

2) brain wave acquisition: the experimental arrangement that gathers EEG designs under stim2 platform, that brain wave acquisition instrument uses is scan4.3, the electrode for encephalograms cap of use is that 10-20 standard 64 is led.

Change reference potential in described data preprocessing phase is that the lead former reference potential at place of CZ is become and is positioned at the M1 at mastoid process position, both sides, the current potential leading in M2 place.

Down-sampled described in stage (2) is that the sample frequency of EEG signals is reduced to 128Hz by 1000Hz.

Bandpass filtering scope described in stage (2) is 1Hz～45Hz.

Removal eye electricity described in stage (2) is removed by the method for independent component analysis filtering.

Feature extraction phases described in stage (3) comprises the steps:

1) cospace Pattern Filter:

The original EEG signals of three generic tasks is the matrix of 64 × 384 dimensions: be made as X ₁, X ₂and X ₃wherein 64 is the port number that leads, and 384 is the number of data points of first 3 second of each passage under stimulation task, and the normalized covariance matrix of each class signal is respectively:

$R_i=\frac{X_i{X}_i^T}{\mathrm{trace}\left(X_i{X}_i^T\right)};$ i＝1，2，3

The X here ^trepresent the transposition of X, the mark of trace representing matrix.

Constructing synthetic sky news covariance matrix is:

R＝R ₁+R ₂+R ₃

R can be decomposed into:

$R=U_{0}A{U}_0^T$

U ₀be respectively its feature matrix and eigenvalue diagonal matrix with A, whitening transformation can make variance homogenization, and albefaction matrix is:

$W=A^{-1/2}{U}_0^T$

In order to set forth better this algorithm, first consider how to obtain the spatial filter of the evoked brain potential that joyful degree grade is 1, establish:

R ' ₁=R ₂+ R ₃, order:

$S_1=W_1{R}_1{W}_1^T$

$S_1^{\′}=W_1{R}_1^{\′}{W}_1^T$

If S ₁can resolve into:

$S_1=U_1{A}_1{U}_1^T$

S ' ₁can be broken down into:

$S_1^{\′}=U_1{A}_1^{\′}{U}_1^T$

And have:

A ₁+A′ ₁＝1

By formula $S_1=W_1{R}_1{W}_1^T,$ $S_1^{\′}=W_1{R}_1^{\′}{W}_1^T,$ $S_1=U_1{A}_1{U}_1^T,$ $S_1^{\′}=U_1{A}_1^{\′}{U}_1^T,$ $S_1^{\′}=U_1{A}_1^{\′}{U}_1^T{A}_1+A_1^{\′}=1$ Comprehensively, obtain:

${\left(W_1^T{U}_1\right)}^T{R}_1\left(W_1^T{U}_1\right)+{\left(W_1^T{U}_1\right)}^T{R}_1^{\′}\left(W_1^T{U}_1\right)=1$

The U here ₁column vector be matrix S ₁characteristic vector.Can find out, after conversion, the characteristic vector characteristic of correspondence value sum of signal covariance matrix is 1, so at X ₁remaining signal (X in the direction of variance yields maximum ₂and X ₃) variance yields just very little, so choose U ₁in with maximum eigenvalue characteristic of correspondence vector

corresponding to X ₁spatial filter, the projecting direction of signal is:

${\mathrm{SF}}_1=U_{1i}^T{W}_1$

Projection under corresponding pattern is:

Z ₁＝SF ₁X ₁

In like manner can obtain X ₂, X ₃spatial filter be respectively:

${\mathrm{SF}}_2=U_{2i}^T{W}_2,$ ${\mathrm{SF}}_3=U_{3i}^T{W}_3$

The spatial filter more than obtaining is 64 × 64 dimensions, each row vector is called as space filtering, corresponding spatial filter obtains filtered signal with corresponding signal multiplication, the emotion EEG signals after cospace Pattern Filter, the interference of having removed background signal;

2) in order to carry out better follow-up Classification and Identification, do after following variation as brain electrical characteristic values according to filtered signal Zm,

$f_m=\lg \frac{\mathrm{var}\left(Z_m\right)}{{\mathrm{\Σ}}_1^3\mathrm{var}\left(Z_m\right)};$ m＝1，2，3。

Emotional state recognition methods based on brain electrical feature of the present invention, the OVR cospace pattern algorithm of trial, the interference that can remove background signal, strengthens for the signal of multiclass emotion evoked brain potential.Remove after background signal interference, between different classes of emotion brain electricity, difference increases, and identifies with the variance feature in time domain, and subjects's recognition correct rate is comparatively desirable, can distinguish exactly the emotion of different joyful degree.

Accompanying drawing explanation

Fig. 1 is the flow chart of the inventive method;

Fig. 2 is the time distribution map of every subtask

In figure, A is that picture presents the period, and B is the rest period, and C reminds the period;

Fig. 3 is data pretreatment process figure of the present invention.

The specific embodiment

Below in conjunction with embodiment and accompanying drawing, the emotional state recognition methods based on brain electrical feature of the present invention is described in detail.

The emotional state recognition methods based on brain electrical feature that the present invention proposes, the evoked brain potential of multiclass Emotional Picture is used to cospace Pattern Filter, to strengthen the brain electricity composition relevant to emotion task, then extract feature and carry out Classification and Identification, obtain desirable discrimination.

As shown in Figure 1, the emotional state recognition methods based on brain electrical feature of the present invention, comprised as the next stage:

(1) data acquisition phase, under the international Emotional Picture induced conditions of described data acquisition Shi, extracts and testedly under the joyful degree grade of difference picture brings out, 64 leads eeg data;

(2) data preprocessing phase,

To collect 64 processing of leading eeg data and change reference potential, down-sampled, bandpass filtering, four steps of removal eye electricity;

(3) feature extraction phases,

Described feature extraction phases is that pretreated signal cospace pattern algorithm is carried out extracting temporal signatures after filtering;

(4) feature identification

After feature extraction, use support vector machine classifier to identify feature, different emotional states are distinguished.

4 stages of the emotional state recognition methods that the present invention is based on brain electrical feature are described respectively below:

(1) data acquisition phase,

Under the international Emotional Picture induced conditions of data acquisition Shi, extract tested emotion evoked brain potential under the joyful degree grade of difference (passive, neutral, positive) picture stimulates.International Emotional Picture system (the International Affective Picture System using in the present invention, IAPS) be to go through with noting institute (Emotion & Attention) a set of standardization emotional distress picture system that the several years works out by the state-run mental health of U.S. research center (National Institute of Mental Health, NIMH) emotion.It by many have emotion arousal effects, international addressable, formed by the colour picture of extensive semantic domain, these colour pictures have the scoring of three dimensions (joyful degree, degree of waking up, dominance).The present invention has selected the picture of different joyful degree scopes as stimulus material from IAPS, and the emotion that the representative of the joyful higher grade of degree of Emotional Picture induces is more positive, and the lower grade emotion inducing is more passive.The present invention utilizes these pictures to carry out emotion to experimenter to bring out, and records its EEG signals, carries out follow-up research.

From international Emotional Picture system, selected 45 pictures, be divided into 3 grades by joyful degree scope, grade 1,2,3 joyful degree averages are respectively 1.68,5.57,8.14, and each grade has 15 pictures, and in each grade, the joyful degree value of every pictures is close.Grade 1,2,3 is corresponding passiveness, neutral, the active mood picture of representing respectively.This experiment comprises 45 subtasks, and be 14 seconds each task time, comprises three periods, as shown in Figure 2.

Very first time section is that picture presents the period, continues 6s, presents an Emotional Picture on screen.This time period experimenter concentrates one's energy observe and experience picture, and (give a mark and bring the enjoyment level of oneself according to Emotional Picture to the joyful degree marking of picture according to emotional experience self-report method, minimum is 1 point, is up to 9 points, and mark is higher represents that enjoyment level is higher).Second period is the rest period, continues 6s, presents white circle on screen, and this period, experimenter loosened rest, calms down own emotion as far as possible, the scoring that the while is filled in the picture to just now presenting in given form.The 3rd period, for reminding the period, continues 2s, presents red circle on screen, reminds experimenter to concentrate one's energy, and next pictures will occur.

Select 13 experimenters (7 male, 6 women, are dextromanuality at 22～25 one full year of life of age) to participate in this experiment.Experimenter's vision is all normal or correct defects of vision normally, and experimenter is healthy, the medical history that is all a cup too low or psychosis family history, and experiment experimenter's on the same day the mental status is good.All 2 points in the afternoon of experiment time started, experimenter has sufficient lunch break noon, to guarantee having full mental status afternoon.About 60 minutes of experimental period, experiment early-stage preparations approximately 45 minutes, experimental period approximately 15 minutes.

The experiment stimulation programs that gathers brain electricity designs under stim2 platform, and that brain wave acquisition instrument uses is scan4.3, and the electrode for encephalograms cap of use is that 10-20 standard 64 is led.

(2) data preprocessing phase

Collect 64 and lead after EEG data, carry out data pretreatment.Data pretreatment process as shown in Figure 3, mainly comprises and changes reference potential, down-sampled, bandpass filtering, four steps of removal eye electricity.

Reference potential when Scan4.3 system acquisition is near CZ, and this signal amplitude that causes top region to be led is very low.Therefore first carry out reference potential conversion, become the M1, the M2 that are positioned at mastoid process position, the both sides place of leading with reference to current potential, the signal amplitude leading in top region is increased, be convenient to follow-up data processing.The sample frequency of system is 1000Hz, is mainly to change requirement rapidly in order to meet EEG signals.But the sample frequency of 1000Hz is far longer than the theoretical sample frequency of Nyquist's theorem, and sample frequency is crossed conference and is caused data volume to cross ambassador's subsequent treatment Efficiency Decreasing.Therefore, carry out the data that collect down-sampledly, the sample frequency of EEG signals is reduced to 128Hz by 1000Hz.Because current generally acknowledged brain wave frequency mainly concentrates on below 45Hz, so having carried out the bandpass filtering of 1Hz～45Hz, the present invention disturbs and high-frequency signal to remove direct current.In addition, because the experiment of the present invention design is by vision induced, the EEG signals therefore collecting inevitably can contain eye electricity (comprise eyeball upper and lower, move left and right nictation) and the impact that brings of electromyographic signal.Wherein eye electricity, especially the electro-ocular signal of nictation is strong especially, and what be subject to that eye electricity has the greatest impact is leading of forehead region.For the impact of the electro-ocular signal adulterating in EEG signals and electromyographic signal generation, the present invention gives filtering by the method for independent component analysis (Independent Component Analysis, ICA) filtering.

(3) feature extraction phases

Described feature extraction phases is that pretreated signal cospace pattern algorithm is carried out extracting temporal signatures after filtering.

The present invention adopts IAPS picture to bring out, the visual evoked potential being mingled with in signal etc. has affected extraction and the identification of the EEG signals composition relevant to emotion, for improving the signal to noise ratio of the brain electricity composition relevant to emotion, the present invention attempts signal to carry out cospace Pattern Filter.The thought of cospace pattern algorithm is to carry out the spatial filter of devise optimum based on two Simultaneous Diagonalization of Covariance Matrices, makes the variance of filtered EEG signals, according to different mission modes, is distinguished to greatest extent.Cospace pattern algorithm expands in polytypic application, and comparatively conventional is " one-to-many " (One Versus the Rest) cospace pattern algorithm, is called for short OVR algorithm.In this algorithm, each quasi-mode, with respect to other pattern of remainder, all can calculate an empty wave filter of hearing, and for example 3 class EEG signals have three spatial filters.Still less, the intrinsic dimensionality obtaining is lower, is conducive to the design of grader for the spatial model that this algorithm is selected.The evoked brain potential that it is 1,5,8 to joyful degree grade that the present invention attempts carries out pattern recognition, below in conjunction with example, this algorithm is done to concrete introduction:

The original EEG signals of (1) three generic task is the matrix of 64 × 384 dimensions: be made as X ₁, X ₂and X ₃wherein 64 is the port number that leads, and 384 is the number of data points of first 3 second of each passage under stimulation task, and the normalized covariance matrix of each class signal is respectively:

$R_i=\frac{X_i{X}_i^T}{\mathrm{trace}\left(X_i{X}_i^T\right)};$ i＝1，2，3 (1)

The X here ^trepresent the transposition of X, the mark of trace representing matrix.

Constructing synthetic sky news covariance matrix is:

R＝R ₁+R ₂+R ₃ (2)

R can be decomposed into:

$R=U_{0}A{U}_0^T---\left(3\right)$

U ₀be respectively its feature matrix and eigenvalue diagonal matrix with A.Whitening transformation can make variance homogenization, and albefaction matrix is:

$W=A^{-1/2}{U}_0^T---\left(4\right)$

In order to set forth better this algorithm, first consider how to obtain the spatial filter of the evoked brain potential that joyful degree grade is 1.If:

R ' ₁=R ₂+ R ₃, order:

$S_1=W_1{R}_1{W}_1^T---\left(5\right)$

$S_1^{\′}=W_1{R}_1^{\′}{W}_1^T---\left(6\right)$

If S ₁can resolve into:

$S_1=U_1{A}_1{U}_1^T---\left(7\right)$

S ' ₁can be broken down into:

$S_1^{\′}=U_1{A}_1^{\′}{U}_1^T---\left(8\right)$

And have:

A ₁+A′ ₁＝1 (9)

By formula (5), (6), (7), (8), (9), comprehensive, obtain:

${\left(W_1^T{U}_1\right)}^T{R}_1\left(W_1^T{U}_1\right)+{\left(W_1^T{U}_1\right)}^T{R}_1^{\′}\left(W_1^T{U}_1\right)=1---\left(10\right)$

The U here ₁column vector be matrix S ₁characteristic vector.Can find out, after conversion, the characteristic vector characteristic of correspondence value sum of signal covariance matrix is 1, so at X ₁remaining signal (X in the direction of variance yields maximum ₂and X ₃) variance yields just very little.So choose U ₁in with maximum eigenvalue characteristic of correspondence vector

corresponding to X ₁spatial filter, the projecting direction of signal is:

${\mathrm{SF}}_1=U_{1i}^T{W}_1---\left(11\right)$

Projection under corresponding pattern is:

Z ₁＝SF ₁X ₁ (12)

In like manner can obtain X ₂, X ₃spatial filter be respectively:

${\mathrm{SF}}_2=U_{2i}^T{W}_2,$ ${\mathrm{SF}}_3=U_{3i}^T{W}_3$

The spatial filter more than obtaining is 64 × 64 dimensions, and each row vector is called as space filtering, and corresponding spatial filter obtains filtered signal, for example Z with corresponding signal multiplication ₁.The emotion EEG signals of processing through CSP, the interference of having removed background signal.

(2), in order to carry out better follow-up Classification and Identification, can do after following variation as brain electrical characteristic values according to filtered signal Zm (m=1,2,3).

$f_m=\lg \frac{\mathrm{var}\left(Z_m\right)}{{\mathrm{\Σ}}_1^3\mathrm{var}\left(Z_m\right)};$ m＝1，2，3

(4) feature identification

Because sample data collection is less than normal, after feature extraction, use support vector machine (Support Vector Machine, SVM) grader feature to be identified, different emotional states are distinguished.13 tested recognition correct rates are desirable (average recognition rate is about 82.5%) comparatively, can distinguish exactly the emotion of different joyful degree.

The present invention is intended to propose a kind of new Emotion identification method, identify people's emotional state by extracting the spectrum signature of cospace Pattern Filter hindbrain electricity, then for the treatment evaluation of mental illness provides an objective appraisal method more, or provide nervous physiology information as feedback signal for mood regulation.Emotional state recognition methods based on brain electrical feature of the present invention can improve the accuracy of emotional state identification effectively, and obtains considerable Social benefit and economic benefit.Optimum implementation intends adopting patent transfer, technological cooperation or product development.

Claims (2)

1. the emotional state recognition methods based on brain electrical feature, is characterized in that, comprises as the next stage:

(1) data acquisition phase, under the international Emotional Picture induced conditions of described data acquisition Shi, extracts and testedly under the joyful degree grade of difference picture brings out, 64 leads eeg data, comprises and carries out following steps:

1) experimental design: selected 45 pictures from international Emotional Picture storehouse, be divided into 3 grades by joyful degree scope, each grade 15 pictures; Grade 1,2,3 is corresponding passiveness, neutral, the active mood picture of representing respectively; Comprise 45 subtasks, stimulate task time is 14 seconds at every turn, has three periods, and picture presents the lasting 6s of period A lasting 6s, rest period B and reminds period C to continue 2s;

2) brain wave acquisition: the experimental arrangement that gathers EEG designs under stim2 platform, that brain wave acquisition instrument uses is scan4.3, the electrode for encephalograms cap of use is that 10-20 standard 64 is led;

(2) data preprocessing phase,

To collect 64 processing of leading eeg data and change reference potential, down-sampled, bandpass filtering, four steps of removal eye electricity;

(3) feature extraction phases,

Described feature extraction phases is that pretreated signal cospace pattern algorithm is carried out extracting temporal signatures after filtering, comprises the steps:

1) cospace Pattern Filter:

The original EEG signals of three generic tasks is the matrix of 64 × 384 dimensions: be made as

,

with

, wherein 64 is the port number that leads, and 384 is the number of data points of first 3 second of each passage under stimulation task, and the normalized covariance matrix of each class signal is respectively:

Here it is right to represent

transposition, tracethe mark of representing matrix;

Constructing synthetic space covariance matrix is:

can be decomposed into:

with be respectively its feature matrix and eigenvalue diagonal matrix, whitening transformation can make variance homogenization, and albefaction matrix is:

In order to set forth better this algorithm, first consider how to obtain the spatial filter of the evoked brain potential that joyful degree grade is 1, establish:

order:

here

corresponding albefaction matrix;

If

can resolve into:

can be broken down into:

And have:

By formula

,

, ,

,

comprehensively, obtain:

Here

column vector be matrix

characteristic vector, can find out, after conversion, the characteristic vector characteristic of correspondence value sum of signal covariance matrix is 1, so remaining signal in the direction of variance yields maximum with

variance yields just very little, so choose

in with maximum eigenvalue characteristic of correspondence Definition of Vector be

, corresponding to

spatial filter, the projecting direction of signal is:

here corresponding

try to achieve

albefaction matrix,

in with maximum eigenvalue characteristic of correspondence Definition of Vector be

;

Projection under corresponding pattern is:

In like manner can obtain

,

spatial filter be respectively:

here

corresponding

try to achieve

albefaction matrix,

in with maximum eigenvalue characteristic of correspondence Definition of Vector be ;

here

corresponding try to achieve

albefaction matrix,

in with maximum eigenvalue characteristic of correspondence Definition of Vector be

;

The spatial filter more than obtaining is 64 × 64 dimensions, each row vector is called as space filtering, corresponding spatial filter obtains filtered signal with corresponding signal multiplication, the emotion EEG signals after cospace Pattern Filter, the interference of having removed background signal;

2) in order to carry out better follow-up Classification and Identification, according to filtered signal

do after following variation as brain electrical characteristic values,

(4) feature identification

After feature extraction, use support vector machine classifier to identify feature, different emotional states are distinguished.

2. the emotional state recognition methods based on brain electrical feature according to claim 1, it is characterized in that, change reference potential in described data preprocessing phase is that the lead former reference potential at place of CZ is become and is positioned at the M1 at mastoid process position, both sides, the current potential leading in M2 place.

3. the emotional state recognition methods based on brain electrical feature according to claim 1, is characterized in that, down-sampled described in the stage (2) is that the sample frequency of EEG signals is reduced to 128Hz by 1000Hz.

4. the emotional state recognition methods based on brain electrical feature according to claim 1, is characterized in that, the bandpass filtering scope described in the stage (2) is 1Hz～45Hz.

5. the emotional state recognition methods based on brain electrical feature according to claim 1, is characterized in that, the removal eye electricity described in the stage (2) is removed by the method for independent component analysis filtering.

Images