CN115778390A - Mixed modal fatigue detection method based on linear prediction analysis and stacking fusion - Google Patents

Abstract

The invention discloses a mixed-mode fatigue detection method based on linear predictive analysis and stacking fusion, which comprises the steps of: collecting EEG signals and EOG signals; performing blind source separation based on a fast independent principal component analysis algorithm to obtain multiple channel forehead EOG signal and multi-channel pure EOG signal; perform linear predictive analysis to solve the linear predictive cepstral coefficient; perform Mexican hat continuous wavelet transform to detect peaks, encode positive and negative peaks, and extract the information reflected by electrooculogram signals from the coded sequence Statistical characteristics of gaze, saccade, and blink; perform regression training to obtain fatigue prediction results. On the basis of exploring and using EEG and EEG physiological signals to detect the laws of fatigue state, the present invention carries out research on EEG signal processing, feature extraction and regression training methods for fatigue detection, and makes full use of the fatigue-related features reflected in EEG and EEG information, which effectively improves the accuracy of fatigue detection based on EEG mixed modalities.

Description

Hybrid-modal fatigue detection method based on linear predictive analysis and stack fusion

technical field

The invention relates to the technical field of EEG signal processing, in particular to a mixed-mode fatigue detection method based on linear predictive analysis and stacking fusion.

Background technique

Mental fatigue is generally a state of decreased brain activity or efficiency resulting from prolonged or intense mental activity. Mental fatigue is one of the main causes of many safety accidents, especially for practitioners who need to maintain a constant state of vigilance, such as car and airplane drivers. Uncontrollable hazardous events reduce ability to increase potential driving risk. How to effectively detect the fatigue state of workers and give early warning to overloaded dangerous work behavior is a very meaningful research work.

The current methods used to assess the state of mental fatigue can be mainly divided into two categories: detection methods based on subjective surveys and objective detection methods based on physiological signals. The detection method of subjective survey is mainly through self-judgment or other people's judgment, in the form of filling in questionnaires (such as Chalder fatigue scale, Fatigue Scale-14 fatigue scale) to evaluate the fatigue degree of the subject. Although this form is simple to implement, it is too subjective and not real-time enough to accurately know the fatigue state level of the subject. The objective detection method based on physiological signals mainly collects electroencephalogram (Electroencephalogram, EEG), electrooculogram (EOG), electrocardiogram (ECG) and electromyogram (EMG) signals from the judge body. From these physiological signals, the law of signal changes when fatigue occurs is studied, and the characteristics of fatigue-related signals are extracted to reflect the level of mental fatigue. Although the detection method based on physiological signal analysis has specific requirements for signal acquisition equipment and complex processing, the indicators of fatigue detection are objective, accurate, and real-time, and have wider application prospects. EEG signals are collected from the scalp surface of the human brain and directly record the neurophysiological signals related to alertness. It has the advantages of strong objectivity and high real-time performance for mental state indication, and is generally considered to be a reliable alertness estimation method. The EOG signal contains the eyelid and eye movement information, and when people are tired, they usually unconsciously reduce the blink frequency and eye movement. Therefore, analyzing the EOG signal to extract eye movement feature information is one of the most commonly used and effective methods in fatigue detection. .

Although the fatigue detection index based on EEG is objective and has high time resolution, the subject of EEG signals is very different. The EEG signals collected from different subjects reflect the signal characteristics of the same fatigue level. The ability to transform is not strong, and it cannot be widely used in the crowd. The disadvantage of EOG-based fatigue detection is poor time resolution, because the characteristics reflected by EOG have statistical significance only on a long data segment, which determines the effectiveness of EOG-based fatigue detection in the process of subject fatigue. Early stage detection is poor. The accuracy of existing fatigue detection algorithms based on physiological signals needs to be improved urgently.

Contents of the invention

In order to solve the above technical problems, the present invention proposes a mixed-modal fatigue detection method based on linear prediction cepstral coefficients (LPCC) and stack fusion, and combines the stack fusion algorithm with the EEG mode and the oculoelectric mode , make full use of the information reflecting physiological fatigue in EEG and EEG, and improve the accuracy of fatigue detection based on the mixed mode signals of EEG and EEG.

In order to achieve the above purpose, a mixed-mode fatigue detection method based on linear predictive analysis and stacking fusion designed by the present invention is special in that the method includes the following steps:

S1: collect the EEG signal and EOG signal of the subject;

S2: Process the EEG signal and EOG signal through a band-pass filter to obtain a multi-channel EEG signal and EOG signal;

S3: Perform blind source separation based on the fast independent component analysis (FastICA) algorithm on the multi-channel EOG signal, and decompose the original multi-channel electro-oculogram signal into a multi-channel frontal EOG signal and a multi-channel pure EOG signal;

S4: Merge the multi-channel EEG signal and the multi-channel forehead EOG signal, perform linear predictive analysis on the entire merged multi-channel EEG signal, solve the linear predictive cepstral coefficient, and adopt a moving average (MA) algorithm Feature smoothing of the linear predictive cepstral coefficients;

S5: Perform Mexican hat wavelet transform (MHWT) on the multi-channel pure EOG signal to detect the peak value, encode the positive and negative peak values, and extract the fixation, saccade, and blinking reflected by the electrooculogram signal from the encoded sequence Statistical Features;

S6: Input the features extracted from the EEG signal and the EOG signal in step S5 into the stacking fusion algorithm model for regression training, and obtain the fatigue degree prediction result.

Preferably, in step S1), the EEG signal is collected from the temporal lobe and the occipital lobe of the subject's brain, and the EOG signal is collected from the forehead above the eyes of the subject.

Preferably, in step S2), the band-pass filter process uses a rectangular window function to divide the EEG signal and the EOG signal into non-overlapping data frame segments.

Preferably, in step S3), the specific steps of blind source separation include:

S3.1: Perform centralized processing on the data of each EOG signal acquisition channel, record the original multi-channel EOG data matrix collected as S _M×L , where M is the number of channels, L is the signal length, let S _M×L Subtract the mean value of the row from each element in , and get the central processing matrix S _C ;

S3.2: Whiten the central processing matrix S _C with multi-channel de-meaning, X = ED ^-1/2 E ^T S _C , where X is the data matrix after whitening, E=[c ₁ ,c ₂ ,… ,c _L ] is the eigenvector matrix of M×L, D ^-1/2 =diag[d ^-1/2 d ₁ , d ^-1/2 d ₂ ,…, d ^-1/2 d _L ] is L× The diagonal eigenvalue matrix of L, d _i is the ith eigenvalue of the covariance matrix of the central processing matrix S _C , and _ci is the corresponding eigenvector, i=1,2,...,L;

S3.3: Let the unmixing matrix be W, W is an M-dimensional vector, decompose the multi-channel data S _M×L into a series of independent component variables: U=W×X, use negative entropy to calculate the non- Gaussian, J _G (W) = [E{G(W ^T X)}-E{G(V)}] ² , where J _G (W) is the calculated signal component when using the nonlinear function G Non-Gaussian results, V is a Gaussian variable with the same mean and covariance matrix as X, and the unmixing matrix W of the maximum non-Gaussian components is solved using Newton's method iteratively;

，其中

为满足各分量非高斯性最大时的独立成分变量构成的矩阵，

为分离出的前额EOG信号，

为解混矩阵W的逆矩阵，原始EOG信号与分离出的前额EEG的差值信号为纯净的EOG信号。S3.4: Reconstruct the forehead EEG signal from the unmixing matrix,

,in

In order to satisfy the matrix composed of independent component variables when the non-Gaussian property of each component is the largest,

is the isolated forehead EOG signal,

is the inverse matrix of the unmixing matrix W, and the difference signal between the original EOG signal and the separated forehead EEG is a pure EOG signal.

Preferably, the specific steps of step S4) include:

S4.1: Merge the EEG EEG signal E1=[p ₁ ,p ₂ ,…,p _m1 ] and the separated forehead EOG signal E2=[q ₁ ,q ₂ ,…,q _m2 ] into a complete brain electric signal:

E3=[p ₁ ,p ₂ ,…,p _m1 , q ₁ ,q ₂ ,…,q _m2 ];

Among them, p _i is the EEG channel signal of the head, m1 is the channel number corresponding to the head electrode, q _i is the forehead EEG channel signal, and m2 is the channel number corresponding to the forehead electrode;

来估计：S4.2: Assume that a p-order linear prediction system is adopted, that is, the nth signal sample s(n) is linearly combined through its previous p samples

to estimate:

；

;

为信号样本数据帧片段上的常数，称为p阶线性预测系数，最小化s(n)与

之间的预测误差，求出此时的线性预测系数；in

is a constant on the segment of the signal sample data frame, which is called the p-order linear prediction coefficient, and minimizes s(n) and

Between the prediction error, find the linear prediction coefficient at this time;

：S4.3: According to the defined relationship between the linear prediction coefficient and the linear prediction cepstral coefficient, the following recursive formula is used to iteratively convert the linear prediction coefficient a _i into the linear prediction cepstral coefficient

:

；

;

表示第n个线性预测倒谱系数，

为p阶线性预测系数，由有限个线性预测系数可以得到无限个线性预测倒谱系数；in

Indicates the nth linear predictor cepstral coefficient,

is the p-order linear predictive coefficient, and an infinite number of linear predictive cepstral coefficients can be obtained from a finite number of linear predictive coefficients;

S4.4: Use the moving average algorithm to perform feature smoothing on the extracted linear predictive cepstral coefficients, that is, the smooth linear predictive cepstral _coefficients of the xth data frame The linear predictive cepstral coefficients of are computed by averaging over the time frame dimension.

Preferably, the specific steps in step S4.2 include:

S4.2.1: The difference between the actual sample and the predicted sample is called the prediction error, expressed as:

；

;

Based on the minimum mean square error criterion, the average forecast error is:

；

;

In order to minimize E{e ² (n)}, take the partial derivative of a _i and set the partial derivative to 0, and obtain the equation system:

；

;

S4.2.2: Rewrite the linear prediction equation system to its equivalent form according to the autocorrelation function:

；

;

；

;

Where R _n (j) represents the autocorrelation value calculated between the nth data frame segment and the data frame segment after j data points delay, m is the index of the nth data frame segment, and N is the data frame segment length;

The matrix form is:

；

;

S4.2.3: Use the Levinson-Durbin algorithm to solve the linear prediction equations to calculate the linear prediction coefficient a _i .

Preferably, the specific steps of step S5) include:

S5.1: Perform Mexican hat continuous wavelet transform on the multi-channel pure EOG signal:

；

;

为小波变换结果，t为时间点数，e为自然对数的底数，

为数据的标准差；in,

is the result of wavelet transformation, t is the number of time points, e is the base of natural logarithm,

is the standard deviation of the data;

S5.2: Using a sliding window with a fixed length D, use the peak-seeking algorithm to detect the peak value of the moving window without overlapping, encode the positive peak value detected by the window as 1, and encode the negative peak value as 0, and identify the window without peak value as gaze feature, segments with 01 or 10 were identified as candidates for a glance feature, and occurrences of 010 were identified as a blink feature;

S5.3: Count the number of glances on each data frame, the variance of the number of glances, the maximum value, minimum value, average value, power and average power of the sweep amplitude as the characteristics of the eye-saccade; the blinking on each data frame continues to total Duration, the average value, maximum value, minimum value, number of blinks, maximum value, minimum value, average value, power and average power of the duration of the blink as eye movement features; the total duration of fixation on each data frame, The mean, maximum and minimum values of duration were used as eye movement characteristics of fixation.

Preferably, the specific steps of step S6) include:

S6.1: Input the EEG features into the Bayesian ridge regression model for training, and record the training result as Y1; input the EEG features into the light gradient boosting machine (light gradient boosting machine, lightGBM) regression Training in the model, the training results are recorded as Y2; the EEG features and oculoelectric features are serially fused together, and input into the Bayesian Ridge regression model and the lightweight gradient boosting machine regression model respectively, and the training results are recorded as Y3 and Y3 respectively. Y4, complete the basic regression layer training of the stacking fusion algorithm;

S6.2: Connect the training results of the basic regression layer together as the input NewX=[Y1, Y2, Y3,Y4] of the secondary regression layer of the stacking fusion algorithm, where NewX is the input of the secondary regression layer, Y1 , Y2, Y3, Y4 are the training results of the primary regression layer. The secondary regression layer is the meta-regression layer, which uses a simple linear regression model to avoid overfitting of the fusion model;

S6.3: Evaluate the predicted value output by the model. If the evaluation is passed, the training is completed. If the evaluation fails, return to step S6.1.

与实际PERCLOS疲劳指数标签值Y之间的相关系数来评估模型对于疲劳检测的表现，相关系数公式为：Preferably, the method of evaluating the predicted value output by the model is to use the predicted value output by the model

The correlation coefficient between the actual PERCLOS fatigue index label value Y is used to evaluate the performance of the model for fatigue detection. The correlation coefficient formula is:

；

;

与

分别为第i个数据帧片段的实际PERCLOS疲劳指数标签值与模型对于疲劳程度预测值，i=1,2,…,N；

与

分别为实际PERCLOS疲劳指数标签值Y的平均值与模型输出的预测值

的平均值；where N is the length of the actual PERCLOS fatigue index label value Y;

and

are the actual PERCLOS fatigue index label value of the i-th data frame segment and the predicted value of the model for the fatigue degree, i=1,2,...,N;

and

are the mean value of the actual PERCLOS fatigue index label value Y and the predicted value output by the model

average of;

If the correlation coefficient reaches the preset value, the evaluation is passed; otherwise, the evaluation is not passed.

The present invention further proposes a computer-readable storage medium, which stores a computer program, which is characterized in that, when the computer program is executed by a processor, the above-mentioned mixed-mode fatigue detection method based on linear predictive analysis and stacking fusion is implemented. A Mixed-Modal Fatigue Detection Approach by Fusion of Predictive Analytics and Stacking.

With the rise of brain computer interface (BCI) technology, the use of EEG signals to detect mental fatigue has received continuous attention from researchers. EEG signals are directly collected from the surface of the scalp of the human brain, which is objective for mental state indicators. Strong, high real-time advantages. Because EEG directly records the neurophysiological signals associated with alertness, it is generally accepted as a reliable method for estimating alertness. EEG-based fatigue detection reflects the change of brain activity when fatigue occurs, and has a high time resolution. If EEG and EOG are used in combination of the two signal modalities, the advantages of a single modality can be combined to achieve better detection results. The present invention conducts research accordingly, and proposes an EEG hybrid fatigue detection method based on linear prediction cepstral coefficient and stacking fusion algorithm.

The mixed-mode fatigue detection method based on linear predictive analysis and stacking fusion proposed by the present invention firstly filters and processes the EEG signals and EOG signals collected based on fatigue detection through a band-pass filter, and uses a rectangular window function to divide the signals into non-overlapping Data frame; then use the FastICA blind source separation algorithm to decompose the original EOG into forehead EEG and pure EOG, merge the separated EEG and the EEG collected from the head into a complete EEG signal; use linear predictive analysis method for all EEG signals, The Levinson-Durbin algorithm based on the idea of autocorrelation obtains the linear prediction cepstral coefficient as the feature extracted by EEG; performs MHWT peak-finding on the EOG signal, encodes the positive and negative peaks, and identifies the saccade, blink and gaze reflected by the EOG according to the coding sequence The statistical parameters of each eye movement feature are calculated as the features extracted by EOG; the obtained features are input into the stacking fusion algorithm model, and in the basic regression layer, Bayesian Ridge regression model is used for EEG features; light weight is used for EOG features Quantitative gradient boosting machine regression model; Bayesian Ridge regression model and lightweight gradient boosting machine regression model are used for EEG and EOG serial mixed features respectively, and through the secondary regression layer, that is, meta-regression layer training, the results of each model in the basic regression layer are obtained. weight factor. The meta-regression layer adopts the linear regression model, inputs the prediction results from the basic regression layer, and outputs the final fatigue index prediction value of the stacked hybrid model through training.

The present invention has following beneficial effect:

1) The linear prediction method used in the present invention can well reflect the relationship between the signal before and after, and fatigue, as a gradually changing physiological state, is dynamically accumulated over time, so the idea of linear prediction can be very good It can accurately capture the fatigue characteristics of contextual correlation.

2) The present invention uses the idea of linear prediction to calculate LPC and then convert it to LPCC, because the common features in the field of EEG fatigue detection, such as power spectral density PSD, differential entropy DE, etc., are often measured by Log logarithm, using dB as the unit . According to the Wiener-Hinchin theorem, the autocorrelation sequence of the signal can be obtained by inverting the Fourier transform of the power spectrum of the signal. If we take the Log logarithm of the power spectrum before performing inverse Fourier transform on the power spectrum of the signal, the cepstrum of the signal can be obtained. Therefore, the cepstral coefficient LPCC feature is the corresponding linear prediction feature of the Log logarithmic measure, and the representation of the feature is more accurate.

3) The present invention uses the MA smoothing algorithm to smooth the extracted EEG features. Since the EEG is less stable than the EOG and changes quickly, and the EEG features reflecting fatigue are dynamic and gradual, the fatigue-related features can be filtered out by smoothing. EEG features.

4) The present invention adopts a stacked fusion algorithm model. In the basic regression layer, a Bayesian ridge regression model suitable for EEG fatigue detection is used for EEG mode; a lightweight gradient hoisting machine regression suitable for EOG fatigue detection is used for EOG mode Model; Bayesian Ridge regression model and lightweight gradient boosting machine regression model are used for EEG and EOG serial mixed features respectively, and the task of model weight training is handed over to the meta-regression layer, which can fully and fully utilize EEG and EOG to reduce fatigue related feature information.

5) The stacking fusion algorithm model used in the present invention selects a regressor with large model differences and complex models for training in the basic regression layer, which ensures the depth and breadth of the utilization of EEG features and EOG features, and uses a simple linear regression model for training in the meta-regression layer Learning the model weights of each base regressor prevents overfitting, and the final fusion model is compatible with the advantages of the models obtained from each modality, and has better generalization ability.

Description of drawings

Figure 1 is a diagram illustrating the position of electrodes for EEG signal acquisition.

Figure 2 is an explanatory diagram of the position of the EOG signal acquisition electrodes.

Fig. 3 is a diagram of the implementation steps of the method proposed by the present invention.

Fig. 4 is a demonstration diagram of the processing flow of the method proposed by the present invention.

Figure 5 is a comparison chart of linear prediction power spectrum and fatigue index label.

Fig. 6 is a line graph comparing the predicted value of the fatigue index with the actual label value using the single-mode and mixed-mode methods in an example.

Figure 7 is a line chart of the correlation coefficient of the model obtained on the data of each subject using the single-modal and mixed-modal methods.

Detailed ways

In order to clarify the technical problems, technical solutions and advantages to be solved by the present invention, the processing process of the method proposed by the present invention in specific implementation cases will be described in detail below in conjunction with the accompanying drawings.

It should be noted that the following examples are only illustrative, and the protection scope of the present invention is not limited by these examples.

In this embodiment, an EEG mixed data set composed of 23 subjects' data is used. The data set uses an eye tracker to record eye movements while collecting EEG, and is used to reflect the collected EEG and EEG. Mark the level of fatigue. According to the record of the eye tracker, the percentage of eyelid closure (the Percentage of Eye Closure, PERCLOS) of the subject is calculated every 8s as the label value of the subject's true fatigue level within the 8s, that is, every 8s constitutes a test of fatigue detection Data frame, when applying the proposed method below, feature extraction and fatigue level prediction will be performed on the 8s long data frame.

The embodiment of the present invention provides a mixed-mode fatigue detection method based on linear predictive analysis and stack fusion. By extracting the linear predictive cepstral coefficient features from the EEG signals from the temporal lobe, occipital lobe and forehead of the brain, the EOG from the forehead is extracted. Statistical characteristics of eye movement in the signal can analyze and predict the fatigue degree of the subject, which can achieve the technical effect of improving the accuracy of fatigue detection based on EEG and EOG mixed signals.

In order to achieve the above-mentioned technical effects, the general idea of the present invention is as follows:

Collect EEG signal data from the temporal lobe and occipital lobe of the brain, and EOG signal from the forehead for fatigue detection and regression prediction tasks; filter the collected EEG and EOG data with a band-pass filter and use a rectangular window function to divide them into different Overlapping data frames; use fast independent principal component analysis algorithm on the EOG signal collected from the forehead, separate the original EOG into forehead EEG and pure EOG; merge the EEG collected from the head and the separated forehead EOG, and extract the linear prediction inverted Spectral coefficient features and feature smoothing; the separated EOG signal without EEG interference is subjected to Mexican hat continuous wavelet transform to find peaks, and the positive and negative peaks are coded, and the statistical parameter values of saccade, blink, and fixation characteristics are extracted as the oculograph. The dynamic feature value; the EEG feature and EOG feature are input into the stacked fusion regression model for training, and the output of the meta-regression layer of the stacked fusion model is the predicted value of fatigue.

As shown in Figure 3 and Figure 4, the EEG hybrid fatigue detection method based on linear predictive cepstral coefficient and stacking fusion algorithm proposed by the present invention includes the following steps:

S1: Acquisition of EEG and EOG signals based on fatigue detection. EEG is collected from the temporal lobe and occipital lobe of the brain, and EOG is collected from the forehead above the eyes. The specific location of EEG electrodes can be seen in Figure 1, and the specific location of EOG electrodes can be seen in Figure 1. 2.

S2: Process the collected EEG signal through a band-pass filter of 0.5Hz~50.5Hz, and process the EOG signal through a bandpass filter of 0.5Hz~30.5Hz, and divide the EEG signal and EOG signal into 885 dataframes that don't overlap;

S3: Perform blind source separation on the EOG signal based on the fast independent principal component analysis algorithm FastICA, and decompose the original multi-channel EOG signal into a multi-channel forehead EEG signal and a multi-channel pure EOG signal;

Specific steps include:

S3.1: Centralize the data collected by each EOG channel, record the original multi-channel EOG data matrix collected as S _M×L , where M is the number of channels, L is the signal length, let S _M×L Subtract the mean value of the row from each element to obtain the center processing matrix S _C .

S3.2: Whiten the central processing matrix S _C with multi-channel de-meaning, X = ED ^-1/2 E ^T S _C , where X is the data matrix after whitening, E=[c ₁ ,c ₂ ,… ,c _L ] is the eigenvector matrix of M×L, D ^-1/2 =diag[d ^-1/2 d ₁ , d ^-1/2 d ₂ ,…, d ^-1/2 d _L ] is L× The diagonal eigenvalue matrix of L, d _i is the i-th eigenvalue of the covariance matrix of the central processing matrix S _C , and _ci is the corresponding eigenvector.

S3.3: Let the unmixing matrix be W, W is an M-dimensional vector, decompose the multi-channel data S _M×L into a series of independent component variables: U=W×X, use negative entropy to calculate the non- Gaussian, J _G (W) = [E{G(W ^T X)}-E{G(V)}] ² , where J _G (W) is the calculated signal component when using the nonlinear function G Non-Gaussian result, the present invention selects function G(x)=-exp(-x2/2), V is the Gaussian variable that has identical mean value and covariance matrix with X, uses Newton's method to iteratively solve and makes each component non-Gaussian The unmixing matrix W at the maximum.

，其中

为满足各分量非高斯性最大时的独立成分变量构成的矩阵，

为分离出的前额脑电，

为解混矩阵W的逆矩阵。原始EOG与分离出的前额EEG的差值信号为纯净的EOG信号。S3.4: Reconstruct the forehead EEG signal from the unmixing matrix,

,in

In order to satisfy the matrix composed of independent component variables when the non-Gaussian property of each component is the largest,

For the isolated forehead EEG,

is the inverse matrix of the unmixing matrix W. The difference signal between the original EOG and the separated forehead EEG is a pure EOG signal.

S4: Merge the multi-channel EEG signal and the separated multi-channel forehead EEG signal, perform linear prediction analysis on the merged entire multi-channel EEG signal, solve the LPCC features, and use the MA smoothing algorithm to smooth the LPCC features;

Specific steps include:

S4.1: Merge the EEG signal E1=[p ₁ ,p ₂ ,…,p _m1 ] and the separated frontal EEG signal E2=[q ₁ ,q ₂ ,…,q _m2 ] into a complete brain electric signal:

E3=[p ₁ ,p ₂ ,…,p _m1 , q ₁ ,q ₂ ,…,q _m2 ]

Among them, p _i is the EEG channel signal of the head, m1 is the channel number corresponding to the head electrode, q _i is the forehead EEG channel signal, and m2 is the channel number corresponding to the forehead electrode.

来估计：S4.2: Assume that a p-order linear prediction system is adopted, that is, the nth signal sample s(n) can be linearly combined through its previous p samples

to estimate:

被假定为信号样本数据帧片段上的常数，这些常数即为线性预测系数。基于最小均方误差准则，最小化s(n)与

之间的预测误差，求出此时的线性预测系数。在该实施案例中，采用的是14阶的线性预测系统。in,

are assumed to be constants over the slices of the signal sample data frame, these constants are the linear prediction coefficients. Based on the minimum mean square error criterion, minimize s(n) and

Between the prediction error, find the linear prediction coefficient at this time. In this implementation case, a 14th-order linear prediction system is used.

The specific steps of S4.2 include:

S4.2.1: The difference between the actual sample and the predicted sample is called the forecast error, which can be expressed as:

；

;

Based on the minimum mean square error criterion, the average forecast error is:

；

;

In order to minimize E{e ² (n)}, take the partial derivative of a _i and set the partial derivative to 0, and obtain the equation system:

；

;

S4.2.2: Rewrite the linear prediction equations to their equivalent form according to the autocorrelation function. From the transposition of the linear prediction equation obtained in S4.2.1:

；

;

For the convenience of expression, remember:

；

;

Then the equation can be rewritten as:

；

;

The autocorrelation function of a sample signal s(n) is defined as:

；

;

Combined with S4.2.1, the obtained equations are:

；

;

Since R _n (j) is an even function, and R _n (j) is only related to the relative size of i and j, then:

；

;

The solution to the linear prediction equations is rewritten as:

；

;

Written in matrix form as:

；

;

S4.2.3: Use the Levinson-Durbin algorithm to solve the linear prediction equations obtained in S4.2.2 to calculate the linear prediction coefficient a _i .

：S4.3: According to the defined relationship between the linear prediction coefficient and the linear prediction cepstral coefficient, the following recursive formula is used to iteratively convert the linear prediction coefficient a _i into the linear prediction cepstral coefficient

:

；

;

表示第n个线性预测倒谱系数，

为p阶线性预测系数，由有限个线性预测系数可以得到无限个线性预测倒谱系数，但一般而言，n取到12~20已经足够，在本实施例中，n取到14。in

Indicates the nth linear predictor cepstral coefficient,

is the p-order linear predictive coefficient, and an infinite number of linear predictive cepstrum coefficients can be obtained from the finite linear predictive coefficients, but generally speaking, it is enough for n to be 12~20, and n is taken to be 14 in this embodiment.

根据以x为中心，长为win的平滑窗口内的所有的线性预测倒谱系数在时间帧维度上求取平均值计算得出，

，其中

表示第i个数据帧上计算得到的未平滑的线性预测倒谱系数，平滑窗口长度win必须设置为奇数，在本实施案例中，平滑窗口长设置为29。S4.4: Use the moving average algorithm to perform feature smoothing on the extracted linear predictive cepstral coefficients, that is, the smoothed linear predictive cepstral coefficients of the xth data frame

Calculated by calculating the average value of all linear predictive cepstral coefficients in the smoothing window with length win in the time frame dimension centered on x,

,in

Indicates the unsmoothed linear prediction cepstral coefficient calculated on the i-th data frame. The smoothing window length win must be set to an odd number. In this implementation case, the smoothing window length is set to 29.

As shown in Figure 5, from top to bottom is the PERCLOS fatigue index of a subject's data, and the corresponding data frame is calculated to obtain the power spectrum waterfall diagram, the linear prediction power spectrum diagram and the linear prediction power spectrum diagram smoothed by the MA algorithm. It can be seen that compared to the traditionally used power spectrum features, the changes in the alpha (8-14Hz) band of the spectrograms obtained by the linear prediction method proposed by the present invention are more similar to the changes in the PERCLOS fatigue index label, and the smoothing feature reflects more than the smoothing front. The change law of is more correlated with the change law of the real label.

S5: Use MHWT peak detection with a window length of 8 for the de-interferenced multi-channel EOG signal, encode the detected positive and negative peaks, and extract the statistical features of gaze, saccade, and blink reflected by the EOG signal;

Specific steps include:

；S5.1: Perform the Mexican hat continuous wavelet transform on the multi-channel clean electro-oculogram signal:

;

为小波变换结果，t为时间点数，e为自然对数的底数，

为数据的标准差；经过变换后的眼动信号峰值会更加明显。in,

is the result of wavelet transformation, t is the number of time points, e is the base of natural logarithm,

is the standard deviation of the data; the peak of the transformed eye movement signal will be more obvious.

S5.2: Using a sliding window with a fixed length of 8, the peak-seeking algorithm is used to detect the signal peak value in a non-overlapping moving window. The positive peak value detected by the window is coded as 1, the negative peak value is coded as 0, and the window without peak value is directly identified as Look at the features. Segments with "01" or "10" were identified as candidates for a glance feature, and a blink feature if "010" was present.

S5.3: Count the number of glances on each data frame, the variance of the number of glances, the maximum value, minimum value, average value, power and average power of the sweep amplitude as the characteristics of the eye-saccade; the blinking on each data frame continues to total Duration, the average value, maximum value, minimum value, number of blinks, maximum value, minimum value, average value, power and average power of the duration of the blink as eye movement features; the total duration of fixation on each data frame, The mean, maximum and minimum values of duration were used as eye movement characteristics of fixation. All eye movement features constitute the characteristic sequence of electrooculogram signal.

S6: Input the extracted features of the EEG signal and the electrooculogram signal into the stacking fusion algorithm model for regression training, and obtain the fatigue degree prediction result;

Specific steps include:

S6.1: Input the EEG features into the Bayesian Ridge regression model for training, and record the training results as Y1; input the EEG features into the lightweight gradient boosting machine regression model for training, and record the training results as Y2; The electrical features and oculoelectric features were fused together in series, and were respectively input into the Bayesian Ridge regression model and the lightweight gradient boosting machine regression model, and the training results were recorded as Y3 and Y4 respectively. The above operations complete the basic regression layer training of the stack fusion algorithm.

S6.2: Connect the training results of the basic regression layer together as the input of the secondary regression layer of the stacking fusion algorithm. NewX=[Y1, Y2, Y3, Y4], where NewX is the input of the secondary regression layer, and Y1, Y2, Y3, Y4 are the training results of the primary regression layer. The secondary regression layer is the meta-regression layer, which uses a simple linear regression model to avoid over-fitting of the fusion model. The output result of the meta-regression layer is the final prediction value of the stacked fusion model for the fatigue degree of the subject.

与实际PERCLOS疲劳指数标签值Y之间的相关系数来评估模型对于疲劳检测的表现，相关系数公式如下：

；Take the predicted value of the model output

The correlation coefficient between the actual PERCLOS fatigue index label value Y is used to evaluate the performance of the model for fatigue detection. The correlation coefficient formula is as follows:

;

与

分别为第i个数据帧片段的实际PERCLOS疲劳指数标签值与模型对于疲劳程度预测值，i=1,2,…,N；

与

分别为实际PERCLOS疲劳指数标签值Y的平均值与模型输出的预测值

的平均值；相关系数达到预设值则评估通过，否则评估未通过。where N is the length of the actual PERCLOS fatigue index label value Y;

and

are the actual PERCLOS fatigue index label value of the i-th data frame segment and the predicted value of the model for the fatigue degree, i=1,2,...,N;

and

are the mean value of the actual PERCLOS fatigue index label value Y and the predicted value output by the model

The average value of ; if the correlation coefficient reaches the preset value, the evaluation is passed, otherwise the evaluation is not passed.

Figure 6 shows the line graph of the comparison between the predicted value of the fatigue index and the actual label value using the EEG unimodal, EOG unimodal and EEG-EOG mixed modal methods on a subject. It can be seen that compared with the unimodal, this The fatigue level prediction result obtained by the hybrid mode adopted by the invention is closer to the real label value. Figure 7 shows the results of the correlation coefficient between the predicted value and the actual value of the fatigue level of the data of the 23 subjects in the case obtained when the EEG single modality, EOG single modality and EEG-EOG mixed modality are used. It can be seen that the mixed modality performs better on almost all the subject data, indicating that the method proposed by the present invention can indeed improve the accuracy of fatigue detection, and the generalization ability of the obtained model is stronger.

The above application of specific case data sets to illustrate the present invention is only used to help understand the present invention, and is not intended to limit the present invention. For those skilled in the technical field to which the present invention belongs, some simple deduction, deformation or replacement can also be made according to the idea of the present invention. Those skilled in the art can easily understand that the above description is only an implementation case of the patent of the present invention, and is not intended to limit the patent of the present invention. Any modification, equivalent replacement and Improvements, etc., should be included within the protection scope of the patent for the present invention.

The content not described in detail in this specification belongs to the prior art known to those skilled in the art.

Claims (10)

1. A mixed-mode fatigue detection method based on linear predictive analysis and stack fusion, characterized in that: the method comprises the steps of: S1: collect the EEG signal and EOG signal of the subject; S2: Process the EEG signal and EOG signal through a band-pass filter to obtain a multi-channel EEG signal and EOG signal; S3: performing blind source separation based on a fast independent principal component analysis algorithm on the multi-channel EOG signal, decomposing the original multi-channel electro-oculogram signal into a multi-channel forehead EOG signal and a multi-channel pure EOG signal; S4: Merge the multi-channel EEG signal and the multi-channel forehead EOG signal, perform linear predictive analysis on the entire merged multi-channel EEG signal, solve the linear predictive cepstral coefficient, and use the moving average algorithm to predict the linear predictive cepstral coefficient Perform feature smoothing; S5: performing Mexican hat continuous wavelet transform on the multi-channel pure EOG signal to detect the peak value, encoding the positive and negative peak values, and extracting the statistical characteristics of gaze, saccade, and blink reflected by the electrooculogram signal from the encoding sequence; S6: Input the features extracted from the EEG signal and the EOG signal in step S5 into the stacking fusion algorithm model for regression training, and obtain the fatigue degree prediction result. 2. A kind of mixed mode fatigue detection method based on linear predictive analysis and stacking fusion according to claim 1, characterized in that: in step S1, the EEG signal is collected from the brain temporal lobe of the subject, Occipital lobe, the EOG signal is collected at the forehead position above the eyes of the subject. 3. A kind of mixed mode fatigue detection method based on linear predictive analysis and stacking fusion according to claim 1, characterized in that: in step S2, bandpass filter processing adopts rectangular window function to convert the EEG signal and EOG signals are divided into non-overlapping data frame segments. 4. A kind of mixed mode fatigue detection method based on linear predictive analysis and stacking fusion according to claim 1, characterized in that: in step S3, the specific steps of blind source separation include: S3.1: Perform centralized processing on the data of each EOG signal acquisition channel, record the original multi-channel EOG data matrix collected as S _M×L , where M is the number of channels, L is the signal length, let S _M×L Subtract the mean value of the row from each element in , and get the central processing matrix S _C ; S3.2: Whiten the central processing matrix S _C with multi-channel de-meaning, X = ED ^-1/2 E ^T S _C , where X is the data matrix after whitening, E=[c ₁ ,c ₂ ,… ,c _L ] is the eigenvector matrix of M×L, D ^-1/2 =diag[d ^-1/2 d ₁ , d ^-1/2 d ₂ ,…,d ^-1/2 d _L ] is L× The diagonal eigenvalue matrix of L, d _i is the ith eigenvalue of the covariance matrix of the central processing matrix S _C , and _ci is the corresponding eigenvector, i=1,2,...,L; S3.3: Let the unmixing matrix be W, W is an M-dimensional vector, decompose the multi-channel data S _M×L into a series of independent component variables: U=W×X, use negative entropy to calculate the non- Gaussian, J _G (W) = [E{G(W ^T X)}-E{G(V)}] ² , where J _G (W) is the calculated signal component when using the nonlinear function G Non-Gaussian results, V is a Gaussian variable with the same mean and covariance matrix as X, and the unmixing matrix W of the maximum non-Gaussian components is solved using Newton's method iteratively; S3.4: Reconstruct the forehead EEG signal from the unmixing matrix,

,in

In order to satisfy the matrix composed of independent component variables when the non-Gaussian property of each component is the largest,

is the isolated forehead EOG signal,

is the inverse matrix of the unmixing matrix W, and the difference signal between the original EOG signal and the separated forehead EEG is a pure EOG signal. 5. A kind of mixed mode fatigue detection method based on linear predictive analysis and stack fusion according to claim 1, characterized in that: the specific steps of step S4 include: S4.1: Merge the EEG EEG signal E1=[p ₁ ,p ₂ ,…,p _m1 ] and the separated forehead EOG signal E2=[q ₁ ,q ₂ ,…,q _m2 ] into a complete brain electric signal: E3=[p ₁ ,p ₂ ,…,p _m1 , q ₁ ,q ₂ ,…,q _m2 ] Among them, p _i is the EEG channel signal of the head, m1 is the channel number corresponding to the head electrode, q _i is the forehead EEG channel signal, and m2 is the channel number corresponding to the forehead electrode; S4.2: Assume that a p-order linear prediction system is adopted, that is, the nth signal sample s(n) is linearly combined through its previous p samples

to estimate:

; in

is a constant on the segment of the signal sample data frame, which is called the p-order linear prediction coefficient, and minimizes s(n) and

Between the prediction error, find the linear prediction coefficient at this time; S4.3: According to the defined relationship between the linear prediction coefficient and the linear prediction cepstral coefficient, the following recursive formula is used to iteratively convert the linear prediction coefficient a _i into the linear prediction cepstral coefficient

:

; in

Indicates the nth linear predictor cepstral coefficient,

is the p-order linear predictive coefficient, and an infinite number of linear predictive cepstral coefficients can be obtained from a finite number of linear predictive coefficients; S4.4: Use the moving average algorithm to perform feature smoothing on the extracted linear predictive cepstral coefficients, that is, the smooth linear predictive cepstral _coefficients of the xth data frame The linear predictive cepstral coefficients of are computed by averaging over the time frame dimension. 6. A kind of mixed mode fatigue detection method based on linear predictive analysis and stack fusion according to claim 5, characterized in that: the specific steps in step S4.2 include:

; Based on the minimum mean square error criterion, the average forecast error is:

; In order to minimize E{e ² (n)}, take the partial derivative of a _i and set the partial derivative to 0, and obtain the equation system:

; S4.2.2: Rewrite the linear prediction equation system to its equivalent form according to the autocorrelation function:

;

; Where R _n (j) represents the autocorrelation value calculated between the nth data frame segment and the data frame segment after j data points delay, m is the index of the nth data frame segment, and N is the data frame segment length; The matrix form is:

; S4.2.3: Use the Levinson-Durbin algorithm to solve the linear prediction equations to calculate the linear prediction coefficient a _i . 7. A kind of mixed mode fatigue detection method based on linear predictive analysis and stack fusion according to claim 1, characterized in that: the specific steps of step S5 include: S5.1: Perform Mexican hat continuous wavelet transform on the multi-channel pure EOG signal:

; in,

is the result of wavelet transformation, t is the number of time points, e is the base of natural logarithm,

is the standard deviation of the data; S5.2: Using a sliding window with a fixed length D, use the peak-seeking algorithm to detect the peak value of the moving window without overlapping, encode the positive peak value detected by the window as 1, and encode the negative peak value as 0, and identify the window without peak value as gaze feature, segments with 01 or 10 were identified as candidates for a glance feature, and occurrences of 010 were identified as a blink feature; S5.3: Count the number of glances on each data frame, the variance of the number of glances, the maximum value, minimum value, average value, power and average power of the sweep amplitude as the characteristics of the eye-saccade; the blinking on each data frame continues to total Duration, the average value, maximum value, minimum value, number of blinks, maximum value, minimum value, average value, power and average power of the duration of the blink as eye movement features; the total duration of fixation on each data frame, The mean, maximum and minimum values of duration were used as eye movement characteristics of fixation. 8. A kind of mixed mode fatigue detection method based on linear predictive analysis and stack fusion according to claim 1, characterized in that: the specific steps of step S6 include: S6.1: Input the EEG features into the Bayesian Ridge regression model for training, and record the training results as Y1; input the EEG features into the lightweight gradient boosting machine regression model for training, and record the training results as Y2; The electrical features and oculoelectric features are fused together in series, and are respectively input into the Bayesian Ridge regression model and the lightweight gradient boosting machine regression model. The training results are recorded as Y3 and Y4 respectively, and the basic regression layer training of the stacking fusion algorithm is completed; S6.2: Connect the training results of the basic regression layer together as the input NewX=[Y1, Y2, Y3,Y4] of the secondary regression layer of the stacking fusion algorithm, where NewX is the input of the secondary regression layer, Y1 , Y2, Y3, Y4 are the training results of the primary regression layer; S6.3: Evaluate the predicted value output by the model. If the evaluation is passed, the training is completed. If the evaluation fails, return to step S6.1. 9. A mixed-mode fatigue detection method based on linear predictive analysis and stack fusion according to claim 8, characterized in that: the method of evaluating the predicted value output by the model is to use the predicted value output by the model

The correlation coefficient between the actual PERCLOS fatigue index label value Y is used to evaluate the performance of the model for fatigue detection. The correlation coefficient formula is:

; where N is the length of the actual PERCLOS fatigue index label value Y;

and

are the actual PERCLOS fatigue index label value of the i-th data frame segment and the predicted value of the model for the fatigue degree, i=1,2,...,N;

and

are the mean value of the actual PERCLOS fatigue index label value Y and the predicted value output by the model

average of; If the correlation coefficient reaches the preset value, the evaluation is passed; otherwise, the evaluation is not passed. 10. A computer-readable storage medium storing a computer program, wherein the method according to any one of claims 1 to 9 is implemented when the computer program is executed by a processor.

Images