CN112215057A - Electroencephalogram signal classification method based on three-dimensional depth motion - Google Patents

Abstract

The invention relates to an electroencephalogram signal classification method based on three-dimensional depth motion, which comprises the following steps: making stereo images of dynamic random points at different depths as experimental materials; the tested person watches the moving stereo image and simultaneously acquires an EEG signal of the tested person; preprocessing an electroencephalogram signal; selecting an electroencephalogram channel by utilizing an improved CSP-rank method; EEG signal generation

Dividing the signal into signals containing k overlapped frequency sub-bands in a frequency domain; extracting the characteristics of the electroencephalogram signals by using the constructed common space mode group; performing weight calculation on the extracted signal features by using a sparse regression algorithm, and selecting one feature as a final signal feature to finally obtain an optimized EEG signal; signal processed EEG signal

And sending the signals into a Support Vector Machine (SVM) for signal classification.

Description

Electroencephalogram signal classification method based on three-dimensional depth motion

The technical field is as follows:

the invention relates to the technical field of electroencephalogram signal classification based on three-dimensional depth motion.

Background art:

with the further development of the 3D technology, the method has a great deal of application in various fields such as industry, military, entertainment and the like, and brings great significance to peopleIt is convenient. However, people can have some uncomfortable symptoms such as aching eyes, blurred vision, dizziness and discomfort when watching 3D images. There are documents that indicate that factors affecting 3D viewing comfort and fatigue are crosstalk, distortion, convergence accommodation conflict, and the like. Filippo and the like research the influence of the size, the speed and the parallax of a moving object in a stereoscopic scene on the comfort level, and the fact that the speed of the moving object can cause discomfort in watching compared with the parallax is obtained^[1]. Lambooij et al have concluded that the rate of change of parallax and the frequency of a three-dimensional moving object passing through a zero-parallax plane are key factors affecting discomfort^[2]. Yano has pointed out that stereoscopic depth motion has an important influence on stereoscopic comfort after research using stereoscopic images. For most viewers, the comfortable parallax range of the still image is + -1 °^[3]. For the moving picture, mismatching of convergence adjustment caused by rapid depth movement causes discomfort to the person even if the parallax is within the comfortable range. Although depth motion has a significant impact on visual perception, there has been no adequate and comprehensive study.

Research has indicated that stereo motion is one of the main causes of visual discomfort in video scenes^[4]. Currently, in the field of stereoscopic vision research, two methods, namely subjective evaluation and objective evaluation, are mainly used for relevant research. The traditional research method is mainly based on subjective evaluation, but the method is time-consuming, labor-consuming and susceptible to the test. The objective evaluation method mainly analyzes the acquisition of bioelectrical signals such as electroencephalogram (EEG), Electrocardiogram (ECG), nuclear magnetic resonance (FMRI), and the like. The EEG signal can reflect the change of the nervous system and contain rich physiological and psychological information, and the acquisition process is non-invasive and real-time, so that the EEG signal is a three-dimensional fatigue and comfort detection index with wide development prospect. Hyunmi et al indicate that viewing a flickering motion video can simultaneously induce steady-state visual evoked potentials (SSVEP) and mirror neuron systems, activating different EEG signals^[5]. Frey et al found that differences in the amount of convergence modulation conflict could lead to differences in average brain activity^[6]. Therefore, the EEG signal can reflect the brain nerve change in the stereoscopic vision cognition, and lays a data foundation for the stereoscopic vision cognition research。

Reference documents:

[1]Speranza F,Tam W J,Hur N.Effect of disparity and motion on visual comfort of stereoscopic images[J]. Electronic Imaging,2006,6055:94-103.

[2]Lambooij M T M,Ijsselsteijn W A,Bouwhuis D G,et al.Evaluation of Stereoscopic Images:Beyond 2D Quality[J].IEEE Transactions on Broadcasting,2011,57(2):432-444.

[3]OSTBERG.Accommodation and visual fatigue in display work[J].Displays,2015,2(2):81-85.

[4]Bahill A T,Stark L.Overlapping saccades and glissades are produced by fatigue in the saccadic eye movement system[J].Experimental Neurology,1975,48(1):95-106.

[5]H.Lim,J.Ku,Multiple-command single-frequency ssvep-based bci system using ickering action video[J]. Journal of neuroscience methods 2019,314,21-27.

[6]J.Frey,A.Appriou,F.Lotte,M.Hachet,Estimating visual comfort in stereoscopic displays using electroencephalography:A proof-of-concept[J].IFIP Conference on Human-Computer Interaction,Springer,2015, 354-362.

the invention content is as follows:

the invention provides an electroencephalogram signal classification method aiming at EEG signals induced by different depth motions. The method classifies the signals by the steps of preprocessing, channel selection, signal segmentation, feature extraction, feature optimization and the like, has strong anti-interference capability and universality; the visual cognition condition can be objectively analyzed through the electroencephalogram signals, and errors caused by subjective behaviors of people are avoided. The technical scheme of the invention is as follows:

an electroencephalogram signal classification method based on stereoscopic depth motion comprises the following steps:

(1) making dynamic random point stereogram videos at different depths as experimental materials for exciting EEG signals;

(2) the tested person watches the moving stereo image and simultaneously acquires an EEG signal of the tested person;

(3) preprocessing an electroencephalogram signal: removing redundant information by adopting a band-pass filter, separating an EEG signal by utilizing an independent component analysis method, correcting a base line to solve the problem of base line drift, and marking the category of the extracted EEG signal;

(4) selecting an electroencephalogram channel by utilizing an improved CSP-rank method: calculating the weight of the electroencephalogram channel by using a CSP-rank method based on CSP filtering coefficient sorting; then, according to the calculation result, all the obtained filter coefficients, namely absolute values of channel weight values are sequenced, signals are preliminarily classified at the same time, the number of channels reaching the highest classification rate is selected, the channels are determined, and the electroencephalogram channel selection process is completed; when the data of n electroencephalogram channels are selected for subsequent calculation, EEG signals after channel screening are obtained;

(5) dividing the EEG signal after channel screening into signals containing k overlapped frequency sub-bands on a frequency domain; decomposing each overlapping frequency sub-band into l overlapping time sub-segments in the time domain; at this time, the EEG signal is decomposed into k × l correlated time-frequency signals;

(6) and (3) extracting the characteristics of the electroencephalogram signals by using the constructed common spatial mode group: extracting signal features by a CSP method, and extracting M features from each specific time frequency band respectively, wherein the dimension of the feature is 1 × L, wherein L is M × M, and M is k × L;

(7) performing weight calculation on the signal features extracted in the step (6) by using a sparse regression algorithm, selecting one feature as a final signal feature, and finally obtaining an optimized EEG signal

The dimension of the glass is 1 × l;

(8) signal processed EEG signal

And sending the signals into a Support Vector Machine (SVM) for signal classification.

Description of the drawings:

the implementation steps and advantages of the present invention can be more prominent, and the flow and operation of the present invention can be more easily understood through the attached drawings.

FIG. 1 is a display of an experimental set-up;

FIG. 2 is a flow chart of an overall electroencephalogram experiment;

FIG. 3 is a general framework of a depth-motion-based electroencephalogram signal classification method;

FIG. 4 is a depth dynamic random point stereogram rendering effect;

FIG. 5 electroencephalogram channel screening results;

FIG. 6 is a graph of Oz channel ERSP of tested S1;

FIG. 7 is a schematic diagram of a time-frequency signal screening result;

FIG. 8 shows the classification accuracy of five subjects;

the specific implementation mode is as follows:

the embodiments are described and illustrated in detail to make the aspects of the invention more apparent and convenient to practice, so as to further highlight the advantages and objects of the invention. The method mainly comprises three parts, namely experiment material preparation, EEG signal acquisition and EEG signal classification processing, wherein the EEG signal classification processing method comprises the following steps: preprocessing of electroencephalogram signals, channel selection, signal segmentation, feature extraction, feature optimization and the like, and an overall block diagram of the electroencephalogram signal method is shown in fig. 2.

Firstly, making experimental materials

The depth dynamic random point stereogram (DRDS) in the experimental material is made by Matlab software, and there are 6 kinds of stereo videos with different parallaxes as the experimental material in the experiment, the parallaxes are 20 ", 25", 60 ", 65", 100 "and 200" (i.e. three states of watching is unclear, fuzzy and clear), and the video presentation time is 4 s. The background of the experimental motion scene is a depth dynamic random point diagram without a three-dimensional graph, the visual stimulus is a three-dimensional graph (respectively: square, rectangle, circle and ellipse) in the depth dynamic random point three-dimensional graph, the three-dimensional graph and the visual stimulus continuously make back and forth periodic motion on a display screen, and the experimental stimulus presenting effect is shown in fig. 3.

Acquisition of electroencephalogram signals

EEG signals were acquired using the International 10-20 System, using an embedded 32-lead electroencephalogram cap and a neural scanning system. The sampling frequency of the electroencephalogram signal is 1000Hz, and the experimental device is shown in figure 1. The whole EEG signal experiment flow is shown in fig. 4. The experiment was conducted in a dark room, and the subject was asked to sit at 102cm from the display screen to view moving stereoscopic images. First, at the beginning of each trial, the screen displays a planar cross image for the subject to focus on, which lasts 0.5 s. Then, in the next 4s, a DRDS video appears on the screen, requiring the subject to view and mentally recognize and imagine the shape. In the whole experiment process, the tested person keeps stable emotion and does not make any sound, and the limbs, the lips or the tongue cannot move. Finally, at the end of each experimental subsection, the subject can rest on his own needs. The overall experiment consisted of six sub-sections, each containing 72 trials (4 stereograms, 3 random displays of 6 different sizes of parallax), for a total of 432 trials. There was a 1 second interval between each two trials, and the trial was asked to rest for at least 2min after the end of each sub-segment experiment.

Method for processing electroencephalogram signals

1. Electroencephalogram signal preprocessing

Firstly, a band-pass filter of 1-40Hz is utilized to remove power frequency interference and redundant information of 50 Hz; then, an Independent Component Analysis (ICA) is adopted to separate EEG signals, the interference of artifacts such as myoelectricity and electrooculogram is removed, and baseline correction is carried out to solve the problem of baseline drift; and finally, preliminarily extracting the electroencephalogram signal with the duration of 4s generated when the stimulated image is identified. The labels of the EEG signals extracted after the pre-processing are respectively: and the state of unclear, fuzzy and clear viewing.

2. Channel selection

Firstly, respectively calculating the weight of the brain electric channel under any two modes by using a CSP-rank method based on CSP filter coefficient sorting; the absolute values of the three filter coefficients (i.e., channel weights) are then sorted and the channel with the simultaneously larger coefficient in each mode is selected. After the first half coefficients of all the electroencephalogram channels are searched, the number of the channels with the highest classification accuracy is adopted, the channels are determined, and the electroencephalogram is finishedA channel selection process. By x_iI e {1,2,3} is a representation of the preprocessed EEG signal, all of dimensions N T_sX M, where N is the number of EEG signal acquisition channels, T_sThe number of sampling points of each EEG signal acquisition channel is shown, and M is the total test times contained in the EEG experiment. Then the signal x_iThe normalized covariance matrix in any one mode is:

wherein trace (·) represents the trace of the matrix,

the electroencephalogram data, M, representing the mth trial in the case where the experimental stimulus is the pattern i_iExpressed as the number of experimental trials performed with experimental stimulus in mode i, function (-)^TRepresenting a matrix transposition operator. The results of the EEG channel screening are shown in FIG. 5

3. Signal splitting

Fig. 6 is a graph of ERSP in Oz channel of tested S1 in three modes. As can be seen from fig. 5, the energy of the signal in different modes is different in the same time period, and the energy peak occurrence time is different in the three modes, thereby determining the signal division range. The signal segmentation mainly comprises two steps, firstly, original brain electrical signals are segmented into 7 overlapped frequency sub-bands of 0-8Hz, 4-12Hz, 8-16Hz, 12-24Hz, 16-30Hz, 20-28Hz and 24-32Hz by utilizing discrete wavelet transform, and then each overlapped frequency sub-band is decomposed into 4 overlapped time sub-segments of 0-1.6s, 0.8-2.4s, 1.6-3.2s and 2.4-4s by a sliding time window. The signal segmentation process decomposes a segment of the EEG signal into 28 interrelated time-frequency signals.

4. Feature extraction

The method adopts OVO method to realize multi-classification, and carries out EEG signal feature extraction by constructing a common space pattern group (CSG), thereby realizing EEG signal classification and solving the problem. Sending the EEG signals subjected to channel selection in any two modes into a CSP model to extract m pairs of feature vectors which correspond to each otherThe maximum and minimum of the generalized eigenvalues. For three mode signals, a common spatial filter W is used_jJ belongs to {1,2,3}, and a general common space mode group W is constructed_G。

W_G＝{W₁,W₂,W₃,} (2)

Wherein W_GIs a parallel connection of three spatial filters.

Electroencephalogram signal x in any mode_iThe process of extracting the signal characteristics after channel selection is as follows:

Z_i＝x_i×W_G (3)

wherein Z is_iRepresenting an electroencephalogram signal x_iThe characteristics of (1). After taking the logarithm of the electroencephalogram signal and standardizing the variance, the characteristics of the electroencephalogram signal obtained finally are as follows:

where var (·) represents the variance.

Each channel-selected EEG signal is divided into 28 specific overlapping time-frequency sub-segments and then 2 features are extracted from each specific time-frequency segment using the CSP method. Thus, after the co-spatial mode group, 168 signal features were co-extracted in the EEG signals in 3 different modes.

5. Feature optimization

The sparse regression algorithm is used for automatically optimizing the mixed characteristics with the time-frequency information, and the algorithm formula is as follows:

wherein, beta₁Coefficient regression penalty coefficients representing the signal features to be selected on the control vector β, y representing the labels of the different classes (i.e. class 1, class 2 or class 3), and λ representing a sparse parameter of the control vector β. n represents the number of all the characteristics of the electroencephalogram signals after passing through the common space group. p is sparseNumber of features after regression. The control vector beta obtained by sparse regression often has a sparse characteristic, wherein control parameters corresponding to a plurality of features are zero, electroencephalogram signals are selected through the control parameters, and the feature optimization result is shown in fig. 7. 168 signal features are screened by using a sparse regression algorithm, and finally 36 EEG signal features are selected as classification features.

6 Signal Classification

Each EEG signal after channel selection is divided into specific overlapping time-frequency subsections, signal characteristics are extracted from each specific time-frequency subsection by utilizing a CSP method, all signal characteristics are screened by using a sparse regression algorithm, and finally, partial EEG signal characteristics are selected as classification characteristics. Among many kernel functions of the SVM classifier, the RBF kernel has higher classification performance on nonlinear features and is widely applied to EEG signal classification, so that the SVM with the RBF kernel is selected to classify the features, and the classification result of the method is shown in fig. 8.

Claims (1)

1. An electroencephalogram signal classification method based on stereoscopic depth motion comprises the following steps:

(1) making dynamic random point stereogram videos at different depths as experimental materials for exciting EEG signals;

(2) the tested person watches the moving stereo image and simultaneously acquires an EEG signal of the tested person;

(3) preprocessing an electroencephalogram signal: removing redundant information by adopting a band-pass filter, separating an EEG signal by utilizing an independent component analysis method, correcting a base line to solve the problem of base line drift, and marking the category of the extracted EEG signal;

(4) selecting an electroencephalogram channel by utilizing an improved CSP-rank method: calculating the weight of the electroencephalogram channel by using a CSP-rank method based on CSP filtering coefficient sorting; then, according to the calculation result, all the obtained filter coefficients, namely absolute values of channel weight values are sequenced, signals are preliminarily classified at the same time, the number of channels reaching the highest classification rate is selected, the channels are determined, and the electroencephalogram channel selection process is completed; when the data of n electroencephalogram channels are selected for subsequent calculation, EEG signals after channel screening are obtained;

(5) dividing the EEG signal after channel screening into signals containing k overlapped frequency sub-bands on a frequency domain; decomposing each overlapping frequency sub-band into l overlapping time sub-segments in the time domain; at this time, the EEG signal is decomposed into k × l correlated time-frequency signals;

(6) and (3) extracting the characteristics of the electroencephalogram signals by using the constructed common spatial mode group: extracting signal features by a CSP method, and extracting M features from each specific time frequency band respectively, wherein the dimension of the feature is 1 × L, wherein L is M × M, and M is k × L;

(7) performing weight calculation on the signal features extracted in the step (6) by using a sparse regression algorithm, selecting one feature as a final signal feature, and finally obtaining an optimized EEG signal

The dimension of the glass is 1 × l;

(8) signal processed EEG signal

And sending the signals into a Support Vector Machine (SVM) for signal classification.

Images