CN105105774A - Driver alertness monitoring method and system based on electroencephalogram information - Google Patents

Abstract

The invention discloses a driver alertness monitoring method based on electroencephalogram information. The method comprises the steps that 1, an electroencephalogram sensor is used for collecting an original electroencephalogram signal of a driver; 2, noise-reduction and artifact removing treatment is conducted on the original electroencephalogram signal to improve the reliability of the electroencephalogram signal; 3, correlated alertness characters including the time-space character, the spectral character and the complexity character are extracted; 4, the correlated alertness characters are filtered to remove signals which have nothing to do with alertness states; 5, the relation between the extracted electroencephalogram characters and the alertness states of the driver is found out, and estimation on the alertness states of the driver is eventually achieved by utilizing the extracted electroencephalogram characters. By means of the driver alertness monitoring method based on the electroencephalogram information, the alertness states of the driver can be obtained most objectively and accurately, and enormous potentiality is shown in the field of driving fatigue monitoring. The invention further provides a driver alertness monitoring system based on the electroencephalogram information.

Description

Based on driver's Alertness monitoring method and the monitoring system of brain electric information

Technical field

The present invention relates to driver's Alertness Condition Monitoring Technology field, particularly a kind of driver's Alertness state monitoring method based on brain electric information and detection system.

Background technology

The nineties in 20th century, scientific research institution and company is had to set foot in the middle of the research of driver's Alertness successively.The technology commonly used in present stage is included in the Alertness monitoring technology based on expressive features, the Alertness monitoring technology based on driving behavior, Alertness monitoring technology based on physiological feature parameter.

The advantages such as the Alertness monitoring method based on driver's eye feature is still most widely used, and it relies on without the need to contacting human body, easy to use, obtain good popularization.But this technology is subject to the brightness of environment, the position of tested person head, facial movement problem because exist, and Alertness estimated accuracy is lower.In addition, myopia population increases year by year in the ratio of driver, and many non-near-sighted drivers also have the custom wearing polaroid glasses, and due to the impact that glasses block light, the Alertness precision that this technology is recorded reduces greatly.

Based on the technology of driving behavior, simple, convenient, do not need more data analysis, more intuitively, but along with expanding economy, the demand of automobile is increasing, and increasing people takes driving test, and driving license just can take hands at short notice.The good driving behavior of driver can not be formed at short notice, causes the technology based on driving behavior can produce the judgement information of mistake because of the bad steering behavior of driver thus.

Technology based on brain electricity EEG can obtain the Alertness state of driver the most objective, accurately and real-time, has shown huge potentiality, effectively can reduce the generation of vehicle accident in driving fatigue monitoring field.But, some are had by the driver of the abnormal disease caused of brain electric information, will have a huge impact data, may vehicle accident be caused.

Therefore need to find external action minimum, monitoring method the most accurately, carries out the supervision and analysis of driver's Alertness.

Summary of the invention

The object of the invention is to overcome the defect that in prior art, brain electric information is easily interfered, provide a kind of driver's Alertness state monitoring method based on brain electric information and monitoring system, driver's Alertness state can be monitored out accurately.

Technical scheme provided by the invention is:

Based on driver's Alertness monitoring method of brain electric information, comprise the following steps:

Step one, make require mental skill the original EEG signals of electric transducer collection driver and be converted to brain piezoelectric voltage signal;

Step 2, noise reduction is carried out to described EEG signals go artefact process, improve the reliability of EEG signals;

Step 3, temporal signatures, spectral characteristic and complexity characteristics that extraction Alertness is relevant;

Step 4, filtration Alertness correlated characteristic, remove the signal irrelevant with Alertness state;

Step 5, by experiment Alertness state to be marked, and then determine the relation between described EEG signals and driver's Alertness state, driver's Alertness state is made an estimate.

Preferably, utilize following formula that the original EEG signals of brain electric transducer collection is converted into voltage signal in step one:

$Voltage=\[rawValue\·\left(\frac{1.8}{4096}\right)\]/2000$

Wherein, Voltage is voltage signal, and rawValue is original EEG signals.

Preferably, in step 2, the brain gathered electricity clock signal b (t, x) is transformed to frequency domain B (f, x), and carries out empirical mode decomposition to B (f, x), result is:

$B(f,x)=\underset{i=1}{\overset{n}{\Σ}}{\mathrm{IMF}}_i(f,x)+R(f,x)$

Wherein, IMF _i(f, x) is i-th intrinsic mode function, and R (f, x) is surplus; F is frequency; X is EEG signals variable, and n is the number of intrinsic mode function;

First and second intrinsic mode function component are removed, reaches noise reduction and go artefact object.

Preferably, in step 3, going the EEG signals after artefact to carry out power spectral density calculating to noise reduction, is the EEG signals x (0) of N by length ... x (N-1) sees finite energy signal as, gets its Fourier transformation and obtains

$X_N\left(e^{jw}\right)=\underset{n=0}{\overset{N-1}{\Σ}}x\left(n\right)e^{-jwn},$ Wherein j is complex unit;

Then X is got _n(e ^jw) amplitude square, and divided by the real power spectrum P (e of N as x (n) ^jw) estimated value, namely

$\hat{P}\left(e^{jw}\right)=\frac{1}{N}{\left|\underset{l=1}{\overset{N}{\Σ}}{x}_l{e}^{-jwt}\right|}^2.$

Preferably, the method of filtering Alertness correlated characteristic in step 4 is do on average to the Alertness feature extracted in window when each 2 seconds and several adjacent Alertness features, if the width of sliding window is L, after slip, L data are carried out arithmetic average at every turn, just can obtain one group of electricity Alertness characteristic sequence y of neencephalon through moving average filter (n), its expression formula is:

$y\left(n\right)=\frac{1}{L}\underset{k=0}{\overset{L-1}{\Σ}}(n-k).$

Preferably, step 5, is tested human pilot display traffic signs picture, tested human pilot is judged fast, and uses the correct responsiveness e (t) of following formulae discovery

$e\left(t\right)=\frac{numberT(ST+t-60,ST+t-1+60)}{numberSum(ST+t-60,ST+t-1+60)}$

Wherein, ST is initial time; NumberT (i, j) is the correct number of responses in time window (i, j); NumberSun (i, j) is the total quantity Showed Picture in time window (i, j);

Tested Alertness state is marked by response accuracy.

Based on driver's Alertness monitoring system of brain electric information,

Brain wave sensor, it is for gathering the original EEG signals of driver;

Mobile terminal, its original EEG signals gathered for receiving described brain wave sensor;

Detecting and controlling system, it is integrated on described mobile terminal, according to the eeg data that brain wave sensor gathers, the Alertness of analytical calculation driver.

Preferably, described brain wave sensor and mobile terminal wirelessly carry out data transmission.

Preferably, described mobile terminal adopts Android system or ios system.

The invention has the beneficial effects as follows:

Driver's Alertness state monitoring method based on brain electric information provided by the invention can obtain the Alertness state of driver the most objective, exactly, has shown huge potentiality in driving fatigue monitoring field.Driver's Alertness condition monitoring system based on brain electric information provided by the invention, directly can utilize mobile phone to create the direct control appliance of system, need independent monitoring device more convenient to use more in the past.Equipment exchanges information by wireless modes such as bluetooths, easy to use.Equipment is wearable simultaneously, uses and can directly wear in head during equipment, easy to use.

Accompanying drawing explanation

Fig. 1 is the driver's Alertness state monitoring method flow chart based on brain electric information of the present invention.

Fig. 2 is that the brain electricity based on empirical mode decomposition of the present invention removes artefact method flow diagram

Fig. 3 is the driver's Alertness condition monitoring system structural representation based on brain electric information of the present invention.

Detailed description of the invention

Below in conjunction with accompanying drawing, the present invention is described in further detail, can implement according to this with reference to description word to make those skilled in the art.

As shown in Figure 1, the invention provides a kind of driver's Alertness state monitoring method based on brain electric information, comprise the following steps:

Step one S110: use brain wave sensor to measure the original EEG signals of human pilot in whole driving procedure.

Step 2 S120: noise reduction is carried out to the original EEG signals collected in step one and goes artefact process.

Because EEG signals is very faint, be easily subject in collection and transmission from inside of human body and outside interference.Interfering signal seriously may reduce the signal to noise ratio of collected EEG signals, priming signal distortion, makes the signal that collects unreliable.Go artefact process can improve the reliability of EEG signals by noise reduction.

Step 3 S130: extract Alertness state correlated characteristic.

Principal character relevant with Alertness state in brain electric information has temporal signatures, spectral characteristic, complexity characteristics.Different characteristic is different for the sign ability of Alertness state, and computation complexity is different, and capacity of resisting disturbance is different.

Step 4 S140: filter Alertness correlated characteristic.

In EEG signals, the change of not all current potential is all caused by the change of Alertness state, namely there is a lot of and that Alertness state is irrelevant signal.The original brain electricity Alertness feature of extracting directly affects because be subject to the EEG signals irrelevant with Alertness state, fluctuates larger.Even if driver is in same Alertness state, original brain electricity Alertness feature is also very unstable.Therefore necessary smoothing processing to be carried out to the original brain electricity Alertness feature extracted.

Step 5 S150: the state estimation of vigilance.

Existing Alertness method for estimating state has: linear discriminant model, linear regression model (LRM), artificial nerve network model, support vector machine etc.Wherein linear model training process is simple, and result is stablized, but the precision that Alertness is estimated is poor.The precision that nonlinear method of estimation Alertness is estimated is higher, but also there is deficiency.Artificial nerve network model model parameter in different training is unstable, causes the Alertness obtained to be estimated also unstable, and the setting of neural network structure is also a very difficult problem; And supporting vector machine model training speed when data scale is larger is slower.

In addition, in the estimation of Alertness state, model training process need has the eeg data of Alertness state markup information in a large number.

In step one S110, brain wave sensor is used to measure the original eeg data of human pilot.

Brain wave sensor calculates original EEG signals value by built-in chip on sensor, is transmitted by blue tooth serial agreement with the form of serial data stream, and transmission speed is 9600bps.The original EEG signals value that sensor built-in chip calculates is the integer comprising two bytes (byte), so the magnitude range of EEG signals value is from-32768 to 32767.Being converted into by original EEG signals value can according to following formula with volt (V) magnitude of voltage that is unit:

$Voltage=\[rawValue\·\left(\frac{1.8}{4096}\right)\]/2000$

Wherein, Voltage is the EEG signals being converted into voltage, and rawValue is original eeg data.

In step 2 S120, adopt the brain electricity based on empirical mode decomposition to go artefact method, the process of noise reduction artefact is gone to the brain electric information gathered in step one.First several intrinsic mode functions IMF (IntrinsicModeFunction) is decomposed into brain electric information clock signal, again Hilbert transform is carried out to each IMF, obtain the energy profile on time-frequency plane, thus the time-frequency characteristics of analytic signal.

The artefact stage is gone at the noise reduction of EEG signals, empirical mode decomposition can be utilized, can to non-linear, that non-stationary signal carries out Time-frequency Decomposition advantage, at frequency domain, the EEG signals be recorded to is decomposed into different IMF, shared by signal and noise, the difference of IMF carries out noise reduction.

Empirical mode decomposition can by several IMF sums of the data decomposition of any sophisticated signal.Have following two constraintss for IMF: 1) in whole data sequence, the number of extreme point is equal with the number of zero crossing or differ 1 at the most; 2) in whole wavy curve, the envelope average defined respectively by Local modulus maxima and local minizing point is 0.

The brain electricity clock signal gathered is b (t, x), and it can be decomposed into:

b(t,x)＝s(t，x)+c(t，x)+n(t，x)

Wherein: s (t, x) is noiseless and EEG signals that is artefact; C (t, x) is coherent interference; N (t, x) is random noise.

Transforming to frequency domain is:

B(f，x)＝S(f，x)+C(f，x)+N(f，x)。

As shown in Figure 2, in f-x territory, the key step of empirical mode decomposition EMD denoising carried out to brain wave signal B (f, x) as follows:

A, S121: set handling frequency domain [f ₁, f ₂], original frequency f=f ₁, frequency step Δ f.

B, S122: for frequency f, arrange initial space sampled data B ₁(f, x)=B (f, x).

C, S123: calculate B ₁the all maximum of (f, x) volume and minimum, calculate maximum envelope E _max(f, x) minimum envelope E _min(f, x).

D, S124: calculate average packet winding thread E (f, x)=[E _max(f, x)+E _min(f, x)]/2.

E, S125: deduct average packet winding thread data with initial data and obtain B ₁(f, x)=B ₁(f, x)-E (f, x).

F, S126: judge B ₁whether (f, x) is intrinsic mode function IMF, if not carry out step cS123.

G, S127: record this IMF component IMF _i(f, x)=B ₁(f, x), resets B ₁(f, x)=B (f, x)-B ₁(f, x), and go to step cS123, calculate next IMF

H, S128: put f=f+ Δ f, if f=f ₂, go to step step I S129, otherwise go to step bS122.

I, S129: terminate.

The result of B (f, x) being carried out to empirical mode decomposition is:

$B(f,x)=\underset{i=1}{\overset{n}{\Σ}}{\mathrm{IMF}}_i(f,x)+R(f,x)$

Wherein, R (f, x) is surplus.

On frequency domain, the high-frequency noise in usual EEG signals is distributed in first and second IMF component, therefore during process, first and second IMF component removal can be reached noise reduction object.

In step 3 S130, for the extraction of EEG signals feature, the present invention extracts EEG spectrum feature.

When utilizing EEG signals to carry out driver's Alertness state estimation, need within 2 seconds, extract an EEG spectrum feature, to ensure the reliability of EEG spectrum feature.Therefore the overall spectral distribution of EEG signals in the time window needing to know 2 seconds.In characteristic extraction procedure, the present invention adopts direct fast Fourier transform (FastFourierTransformation, FFT) to ask spectral method.

Be the EEG signals x (0) of N by the segment length in electrode channel ... x (N-1) sees finite energy signal as, goes its Fourier transformation to obtain

$X_N\left(e^{jw}\right)=\underset{n=0}{\overset{N-1}{\Σ}}x\left(n\right)e^{-jwn}$

Then X is got _n(e ^jw) amplitude square, and divided by the true P (e of N as x (n) ^jw) estimated value, namely

$\hat{P}\left(e^{jw}\right)=\frac{1}{N}{\left|\underset{l=1}{\overset{N}{\Σ}}{x}_l{e}^{-jwt}\right|}^2$

If to signal x (n) by window [w ₁, w ₂..., w _n] be weighted, the real power spectrum P (e of the x (n) so after weighting ^jw) volume is estimated as:

$\hat{P}\left(e^{jw}\right)=\frac{\frac{1}{N}{\left|\underset{l=1}{\overset{N}{\Σ}}{x}_l{e}^{-jwt}\right|}^2}{\frac{1}{N\underset{l=1}{\overset{N}{\Σ}}{\left|w_l\right|}^2}}.$

In step 3 S140, owing to there is a large amount of irrelevant signals in original brain wave signal, although go artefact can eliminate part irrelevant signal to the noise reduction of brain wave signal in step 2 S120, but have no idea to eliminate completely, between the brain electrical feature that the original brain electrical feature extracted is relevant to Alertness state, still there is relatively large deviation.Therefore extract after the stage terminates at brain electrical feature, need the filtration stage entering brain electrical feature, eliminate irrelevant signal in original EEG signals as far as possible to the impact of the brain electrical feature reliability extracted.

Research display, the change of Alertness state is the process of a slow gradual change, and the change of brain electrical feature that therefore Alertness state is correlated with also is the process of a slow gradual change, and the irrelevant characteristics of EEG of Alertness state changes greatly and irregularities usually.Utilize the gradually changeable of Alertness state correlated characteristic, i.e. sequential dependency, can algorithm for design filtering or reduce the impact of extraneous features.

Comparatively conventional solution is the brain electrical feature utilizing moving average filter smoothly original, namely does on average the brain electrical feature extracted in window when each 2 seconds and adjacent tens brain electrical features, reaches the effect of level and smooth brain electrical feature.

If the width of sliding window is L, after slip, L data are carried out arithmetic average, just can obtain one group of neencephalon electrical feature sequence through moving average filter, its expression formula is at every turn:

$y\left(n\right)=\frac{1}{L}\underset{k=0}{\overset{L-1}{\Σ}}(n-k).$

In step 5 S150, after removing artefact stage, brain electrical feature extraction stage, brain electrical feature filtration stage through the noise reduction of EEG signals, brain electrical characteristic data comparatively reliably can be obtained.Alertness state estimation model set up the last stage time Alertness state estimation, namely find out the relation between the brain electrical feature of extraction and driver's Alertness state by certain algorithm, final realization uses the brain electrical feature extracted to make an estimate to driver's Alertness state.

The algorithm for estimating of Alertness state is exactly utilize regression model to set up mapping relations between Alertness state related brain electrical feature and the change of Alertness state.Be usually used in the linear regression model of algorithm and support vector machine (SupportVectorMachine, the SVM) regression model of Alertness state estimation.

Assess the performance of the driver's Alertness state estimation model based on brain wave, while will knowing record EEG signals, the real Alertness state of driver how; Namely to mark tested Alertness state while recording tested EEG signals.

Carrying out in annotation process, the driving video that the display faced by tested human pilot will be driven under broadcasting one section of well-chosen monotonous environment, drive simulating people drives under monotonous environment.Except driving video, display also has a little square frame and occur various traffic signs, tested human pilot needs according to display occurring the color of traffic signs presses button corresponding in response panel.Experiment has the traffic signs of 4 kinds of colors, and often kind of Color pair answers a button on resonant plate.There are 160 marks under often kind of color, namely have 640 marks.Occur that stimulates a picture at every turn random in little square frame, two stimulate be spaced apart 7 seconds, the persistent period stimulating picture is 0.5 second, is blank screen in all the other times little square frame.

In this visual task, along with Alertness state is deteriorated, testedly easily push the wrong button, key response accuracy therefore can be utilized to mark tested Alertness state, namely based on tested behavior in Alertness inter-related task, its Alertness state is marked.

Shown by existing research, the Alertness state period of change of people is on average greater than 4 minutes, therefore can mark Alertness state based on average correct responsiveness tested in a period of time.When marking, tested current Alertness state is by the correct response of current time, and the correct responsiveness of local average around in 2 minutes marks, and often cross 7.5 seconds (test period) mark once.The higher Alertness of accuracy is higher, and Alertness state is better.The computing formula of the correct responsiveness e (t) of local average of any time t is as follows:

$e\left(t\right)=\frac{numberT(ST+t-60,ST+t-1+60)}{numberSum(ST+t-60,ST+t-1+60)}$

Wherein, ST is the initial time of Alertness experiment; NumberT (i, j) is the correct number of responses of button in time window (i, j); NumberSum (i, j) is the total quantity of time window (i, j) internal stimulus picture.

By said method, draw the preliminary labeled data of original eeg data in driving and Alertness, obtain driver's Alertness state labeling system.

Average correct responsiveness e (t) is divided into three sections, when e (t) is less than M1, is low Alertness; E (t) is more than or equal to M1, is middle Alertness when being less than M2; E (t) is high Alertness when being more than or equal to M2.

As one preferably, the value of M1 is the value of 0.33, M2 is 0.66.

Driver's Alertness condition monitoring system 200 based on brain electric information provided by the invention comprises brain electric transducer 210, mobile terminal device 220 and is integrated in monitor control system 230 on mobile terminal device.

Brain electric transducer 210 electrode of the present invention is positioned over the position of the left brain forehead of tested human pilot.Detecting and controlling system is integrated on mobile terminal device 220, described mobile terminal device 220 adopts Android system, brain electric transducer 210 and mobile terminal device 220 adopt wireless mode to communicate, namely the initial data collected is delivered on mobile terminal device by the mode such as bluetooth, GPRS by brain electric transducer, after the monitor control system 230 pairs of initial datas be integrated on mobile terminal device process, obtain the state of the vigilance of human pilot.

Although embodiment of the present invention are open as above, but it is not restricted to listed in description and embodiment utilization, it can be applied to various applicable the field of the invention completely, for those skilled in the art, can easily realize other amendment, therefore do not deviating under the general concept that claim and equivalency range limit, the present invention is not limited to specific details and illustrates here and the legend described.

Claims (9)

1., based on driver's Alertness monitoring method of brain electric information, it is characterized in that, comprise the following steps:

Step one, make the original EEG signals of electric transducer collection driver of requiring mental skill;

Step 2, noise reduction is carried out to described EEG signals go artefact process, improve the reliability of EEG signals;

Step 3, temporal signatures, spectral characteristic and complexity characteristics that extraction Alertness is relevant;

Step 4, filter described Alertness correlated characteristic, remove the signal irrelevant with Alertness state;

Step 5, Alertness state to be marked, determine the relation between described EEG signals and driver's Alertness state, driver's Alertness state is made an estimate.

2. the driver's Alertness monitoring method based on brain electric information according to claim 1, it is characterized in that, also comprise original EEG signals in step one and be converted to brain voltage signal, it utilizes following formula that the original EEG signals of brain electric transducer collection is converted into brain voltage signal:

$Voltage=\[rawValue\·\left(\frac{1.8}{4096}\right)\]/2000$

Wherein, Voltage is brain voltage signal, and rawValue is original EEG signals.

3. the driver's Alertness monitoring method based on brain electric information according to claim 2, it is characterized in that, in step 2, by the brain electricity clock signal b (t gathered, x) frequency domain B (f, x) is transformed to, and to B (f, x) carry out empirical mode decomposition, result is:

$B(f,x)=\underset{i=1}{\overset{n}{\Σ}}{\mathrm{IMF}}_i(f,x)+R(f,x)$

Wherein, IMF _i(f, x) is i-th intrinsic mode function, and R (f, x) is surplus; F is frequency; X is EEG signals variable, and n is the number of intrinsic mode function;

First and second intrinsic mode function component are removed, reaches noise reduction and go artefact object.

4. the driver's Alertness monitoring method based on brain electric information according to claim 3, it is characterized in that, in step 3, the EEG signals after artefact is gone to carry out power spectral density calculating to noise reduction, be the EEG signals x (0) of N by length, x (N-1) sees finite energy signal as, gets its Fourier transformation and obtains:

$X_N\left(e^{jw}\right)=\underset{n=0}{\overset{N-1}{\Σ}}x\left(n\right)e^{-jwn},$ Wherein j is complex unit;

Then X is got _n(e ^jw) amplitude square, and divided by the real power spectrum P (e of N as x (n) ^jw) estimated value, namely

$\hat{P}\left(e^{jw}\right)=\frac{1}{N}|\underset{l=1}{\overset{N}{\Σ}}{x}_l{e}^{-jwt}{|}^2.$

5. the driver's Alertness monitoring method based on brain electric information according to claim 4, it is characterized in that, the method of filtering Alertness correlated characteristic in step 4 is do on average to the Alertness feature extracted in window when each 2 seconds and several adjacent Alertness features, if the width of sliding window is L, after slip, L data are carried out arithmetic average at every turn, obtain one group of electricity Alertness characteristic sequence y of brain through moving average filter (n), its expression formula is:

$y\left(n\right)=\frac{1}{L}\underset{k=0}{\overset{L-1}{\Σ}}(n-k).$

6. the driver's Alertness monitoring method based on brain electric information according to claim 5, it is characterized in that, step 5, is human pilot display traffic signs picture, human pilot is judged fast, and uses the correct responsiveness e (t) of following formulae discovery

$e\left(t\right)=\frac{numberT(ST+t-60,ST+t-1+60)}{numberSum(ST+t-60,ST+t-1+60)}$

Wherein, ST is initial time; NumberT (i, j) is the correct number of responses in time window (i, j); NumberSun (i, j) is the total quantity Showed Picture in time window (i, j);

Tested Alertness state is marked by response accuracy.

7., based on driver's Alertness monitoring system of brain electric information, it is characterized in that,

Brain wave sensor, it is for gathering the original EEG signals of driver;

Mobile terminal, its original EEG signals gathered for receiving described brain wave sensor;

Detecting and controlling system, it is integrated on described mobile terminal, according to the eeg data that brain wave sensor gathers, extracts the feature that Alertness is relevant, marks Alertness state, analyze the Alertness of driver.

8. the driver's Alertness monitoring system based on brain electric information according to claim 7, it is characterized in that, described brain wave sensor and mobile terminal wirelessly carry out data transmission.

9. the driver's Alertness monitoring system based on brain electric information according to claim 7 or 8, is characterized in that, described mobile terminal adopts Android system or ios system.