WO2020147235A1

WO2020147235A1 - Method of detecting driving fatigue on basis of brain function network constructed using phase locking value

Info

Publication number: WO2020147235A1
Application number: PCT/CN2019/088445
Authority: WO
Inventors: 王洪涛; 刘旭程; 吴聪; 唐聪; 裴子安; 岳洪伟; 陈鹏
Original assignee: 五邑大学
Priority date: 2019-01-15
Filing date: 2019-05-25
Publication date: 2020-07-23
Also published as: CN109770923A

Abstract

Provided is a method of detecting driving fatigue on the basis of a brain function network constructed using a phase locking value, belonging to the field of driving fatigue detection; electroencephalogram signals are collected during driving in an awake time and a test time, and said signals are de-noised, decomposed, and reconstructed; then the phase locking value (PLV) within the awake time and the test time series for every two channels is calculated, and, according to the PLV, the functional connection matrix of the channels in the awake time and test time series is formed; a connection strength threshold is set and compared with each element value in the functional connection matrix so as to obtain a connection relationship between the channels in the awake time and the test time series and form a brain function network of the test subject during the awake time and the test time series; the differences between brain function network topology in three sub-bands between the awake time and the test time series are compared and analyzed to determine whether the test subject is in a state of driving fatigue during the test time series; the reliability and accuracy of detection are relatively high.

Description

A driving fatigue detection method based on phase lock value to construct brain function network

Technical field

The present invention relates to the field of driving fatigue detection, and more specifically, to a driving fatigue detection method for constructing a brain function network based on a phase lock value.

Background technique

Many countries attach great importance to the research of driving fatigue-related detection methods. The initial research mainly started from the medical aspect, using medical equipment to study the human mental state. In the early nineteenth century, the United States initiated the investigation of the rationality of the regulations on the service time of motor vehicle drivers. Afterwards, research on driving fatigue has been launched. After years of development, the research on driving fatigue detection methods can be roughly divided into three categories: detection methods based on facial features, detection methods based on driving behavior, and detection methods based on physiological characteristics.

The detection method based on facial features detects whether the driver is in a fatigued state by detecting eye activities such as the driver's blink amplitude, frequency, and average closing time, frequent nodding, long-term head immobility, and other facial features. Most of these detection methods are based on machine vision and have the advantages of easy placement of detection equipment and good timeliness of detection. They are the most commonly used methods for driving fatigue detection.

Detection method based on driving behavior, indirectly by monitoring the driving characteristics of the vehicle such as the steering wheel rotation angle, accelerator pedal, brake, etc., as well as the speed, acceleration, control stability, and deviation of the driving route during the driving process. Determine whether the driver has entered a state of fatigue. The advantage is that it does not need to touch the human body and is convenient to use, and the device occupies a small space inside the vehicle.

In the driving fatigue detection method based on physiological characteristics, the EEG signal can directly reflect the person's physical and mental activities, and is known as the "gold standard" for detecting driving fatigue. Among the methods for EEG signals, power spectrum or entropy methods are mostly used.

The power spectral density method is to transform the EEG signal from the time domain to the frequency domain for analysis. For each frequency band, the energy change from wakefulness to fatigue can be analyzed. When the human brain enters a state of fatigue, the energy of the δ and θ bands of the EEG signal will increase, while the energy of the α and β bands will decrease. The ratio of the energy in the corresponding frequency bands can amplify this trend to judge the driver Mental state.

Entropy can be used to measure the degree of chaos in the system. Entropy-based methods include approximate entropy, sample entropy, wavelet entropy, etc. Among them, wavelet transform has the characteristics of time-frequency localization, and wavelet entropy is an entropy value calculated from the signal sequence after wavelet decomposition, which can accurately reflect the complexity of brain waves. Using wavelet entropy to analyze the complexity of the brain waves of the driver's fatigue state, and analyze the brain waves before simulated driving, after simulated driving fatigue and after rest, driving fatigue can be judged.

However, the above-mentioned driving fatigue detection methods have the following shortcomings: the fatigue detection methods based on facial features are easily affected by the environment, brightness, angle, and some other uncontrollable factors still limit the performance of the algorithm to a certain extent. Facial features based on computer vision The extraction method is very easy to accept and be deceived by artificial forged signals; the driving behavior method is powerless on non-standard roads, the accuracy rate is not enough, and it is easy to produce false alarms; in the detection method based on physiological information, EEG can directly reflect For information such as human physical activity and mental state, the brain realizes information interaction through the interconnection and cluster work of different regions. The state of human consciousness and behavior is not determined by a single region, but is often determined by multiple parts of the whole brain. The regional cooperation is completed, but the driving fatigue detection method based on power spectrum and entropy does not involve brain regional information, and it is impossible to comprehensively and systematically study the mechanism of driving fatigue.

Summary of the invention

The present invention aims to overcome the above-mentioned defects of the prior art and provide a driving fatigue detection method based on the phase lock value to construct a brain function network, and the detection reliability and accuracy are high.

In order to achieve the above objective, the technical solution adopted by the present invention is to provide a driving fatigue detection method based on the phase lock value to construct a brain function network, which is characterized in that it includes the following steps:

S1. Use the EEG signal acquisition device to collect the EEG signals of the subject during the waking time and the test time respectively; the position of each electrode in the EEG signal acquisition device is used as a brain function network node, and the number of electrodes is Number of nodes N;

S2. Denoising the EEG signal to improve the signal-to-noise ratio of the EEG signal;

S3. Decompose and reconstruct the denoising EEG signal, and reconstruct three sub-band waveforms according to the frequency range. The frequency of theta wave is 4-8Hz, the frequency of the alpha wave is 8-13Hz, and the frequency of the beta wave is 13- 30Hz;

S4. In the reconstructed signal, each brain function network node is used as a channel; calculate the phase lock value PLV during the awake time for every two channels to obtain the coupling relationship between every two channels during the awake time; Divide into multiple test time series, calculate the phase lock value PLV in the test time series for each two channels to obtain the coupling relationship between every two channels in the test time series; the phase lock value PLV represents the connection between the two channels Intensity, the phase lock value PLV in the awake time and the test time series is used to form the functional connection matrix of the channel during the awake time and the test time series; the calculation of PLV uses the formula (1):

among them,

It is the phase difference between the two channels during the awake time and the test time series. The phase of each channel during the awake time and the test time series is obtained by Hilbert transform, i∈N is the brain function network node, and the PLV value Between [0, 1], 0 means no connection between channels, 1 means complete connection between channels;

S5. Set the connection strength threshold and compare it with the value of each element in the functional connection matrix to obtain the connection relationship between the channels in the awake time and test time series; the element value greater than or equal to the connection strength threshold is the connection between the two channels , Otherwise there is no connection between the two channels;

S6. According to the connection relationship between channels, form the brain function network during the waking time and the test time series; compare and analyze the difference in the three sub-bands of the brain function network topology between the waking time and the test time series to judge Test whether you are in a state of driving fatigue in the time series.

In the above scheme, the EEG signals of driving during the awake time and the test time are collected, denoising, decomposed and reconstructed, and then the awake time and test time series are calculated for each two channels. The phase lock value PLV in the awake time and the test time series are formed according to the PLV, and the connection strength threshold is set and compared with each element value in the function connection matrix to obtain the awake time and The connection between the channels in the test time series and the formation of the brain function network of the subject during the waking time and the test time series are compared and analyzed for the differences in the brain function network topology between the awake time and the test time series in the three sub-bands. To determine whether it is in a driving fatigue state in the test time series, the reliability and accuracy of the detection are high.

Preferably, the EEG acquisition device in step s1 is a wireless stem electrode EEG acquisition device, including 24 electrodes, and the frequency of collecting signals is 250 Hz; the functional connection matrix is 24*24; and the test time is 90 minutes. The improved international 10-20 placement standard is used to place the electrodes. The electrode names are: AFp3h, AFpz, AFp4h, AFF3, AFFz, AFF4, FFC5h, FFC3h, FFCz, FFC4h, FFC6h, CCP5h, CCP1, CCPz, CCP2, CCP6h, PO3, POz, PO4, PO7, O1h, Oz, O2h, PO8, the wireless stem electrode EEG acquisition equipment is published in the literature (Klem G H, Lüders H O, Jasper H H, et al. The ten-twenty electrode system of the International Federation[J].Electroencephalogr Clin Neurophysiol,1999,52(3):3-6.); the test time of each driving fatigue detection experiment is set at 90 minutes to ensure that the subjects are from the awake state to the fatigue state EEG signals throughout the process.

Preferably, in step s1, the subject uses the simulated driving system to drive, and the simulated driving system randomly sends out braking commands to record the time interval between seeing the braking command and responding to the subject; set the time interval threshold, if the response is correct If the time interval is greater than or equal to the time interval threshold, the EEG signal during the test period before this time interval will be used as the EEG signal during the awake period, and the EEG signal during the test period after this time interval will be used as the EEG signal The signal to be detected for fatigue. The time interval threshold is derived from the training experiment. Due to the individual differences of the subjects, the time interval threshold is not uniform. Therefore, it is necessary to obtain the time interval threshold for individual subjects through the training experiment before the test experiment, that is, before step S1. The calculation method It is the average value of the response time interval during the training experiment when the subject is externally fatigued (such as yawning) or the vehicle's driving path deviates from the normal running track; when driving fatigue, the subject is seeing The time interval between the brake command and the response will be longer. The test time is distinguished by the time interval threshold, and only the EEG signal during the test time after this time interval can be processed as the signal to be detected for fatigue. Reduce the data processed. The subject uses a simulated driving system to drive, also known as car driving simulation, or virtual driving, which refers to the use of modern high-tech means such as: 3D image instant generation technology, vehicle dynamics simulation physical system, and large field of view display technology ( Such as multi-channel stereo projection system), six-degree-of-freedom motion platform (or three-degree-of-freedom motion platform), user input hardware system, stereo sound, central control system, etc., allowing the experiencer to feel close to reality in a virtual driving environment The effect of visual, auditory and somatosensory car driving experience is suitable for the needs of research institutions for automotive engineering, traffic engineering, human factors engineering research and as a simulation experiment platform in the field of automotive research.

Preferably, in step s2, an independent component analysis method is used to perform denoising processing on the EEG signal, so as to remove the interference of the ocular electrical signal and improve the signal-to-noise ratio of the EEG signal.

Preferably, in step s3, wavelet packet transform is used to decompose the EEG signal, and then reconstructed into three sub-band signals.

Preferably, the connection strength threshold in step s5 is 0.2.

Preferably, in step s6, a complex network analysis method is used to quantitatively analyze the differences in the brain function network topology of the three sub-bands during the awake time and the test time sequence from both functional integration and functional differentiation; the functional integration includes features Path length and global efficiency, functional differentiation includes local efficiency and clustering coefficient;

The clustering coefficient represents the degree of clustering of brain function network nodes, which is calculated using formula (2);

Among them, C _i represents the average number of clusters of the brain function network, k _i represents the number of adjacent nodes of node i, adjacent nodes are nodes connected to node i, and E _i is the closed-loop triangle existing in adjacent nodes of node i Number; the other two nodes connected to node i, if there are edges between the three nodes, it is a closed-loop triangle;

The characteristic path length reflects the information transmission ability inside the brain function network, and is calculated using formula (3):

Among them, L is the characteristic path length; L _ij is the shortest path length between node i and node j, that is, the number of shortest edges used from node i to node j; N is the number of nodes;

The global efficiency is the average value of the reciprocal of the shortest path length L _ij , which is used to measure the ability of the brain function network to transmit and process information. It is calculated using formula (4):

Among them, N is the number of nodes, and _Lij is the shortest path length between node i and node j;

The local efficiency is used to measure the local information transmission and processing ability, and it is calculated by formula (5):

Among them, E _global (G _i ) is the global efficiency of node i, namely E _g .

Compared with the prior art, the present invention has the following beneficial effects: by separately collecting EEG signals during awake time and during test time, and performing denoising processing, decomposition and reconstruction on the EEG signals, and then separately analyzing each two The channel calculates the phase lock value PLV during the awake time and the test time series, and forms the functional connection matrix of the channel during the awake time and the test time series according to the PLV, sets the connection strength threshold and connects with each element value in the function connection matrix Make comparisons to obtain the connection relationship between channels in the awake time and the test time series respectively, and form the brain function network of the subject during the waking time and the test time series, and compare and analyze the brain function network during the awake time and the test time series The difference of the topological structure in the three sub-bands is used to determine whether the test time series is in a driving fatigue state, and the reliability and accuracy of the detection are high.

BRIEF DESCRIPTION

FIG. 1 is a diagram of the electrode position of the improved international 10-20 system used in the driving fatigue detection method for constructing a brain function network based on the phase lock value of this embodiment.

Figure 2a is a schematic diagram of the functional connection matrix during awake time, and Figure 2b is a schematic diagram of the functional connection matrix during test time; the row and column lines are channels, and the gradient bar on the right of the grid is the phase lock value PLV.

Figure 3a is a brain function network diagram during awake time, Figure 3b is a brain function network diagram during a test time, and Figure 3c is a comparison diagram of the difference between Figure 3a and Figure 3b.

FIG. 4 is a schematic diagram of the clustering coefficients of the brain function network topology in the three sub-bands during the awake time and the test time sequence in this embodiment.

FIG. 5 is a schematic diagram of the global efficiency of the brain function network topology in the three sub-bands during the awake time and the test time sequence in this embodiment.

detailed description

The drawings of the present invention are only used for exemplary description, and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, some well-known structures in the drawings and their descriptions may be omitted. Understandable.

Examples

This embodiment provides a driving fatigue detection method for constructing a brain function network based on a phase lock value, which is characterized in that it includes the following steps:

S4. In the reconstructed signal, each brain function network node is used as a channel; calculate the phase lock value PLV during the awake time for every two channels to obtain the coupling relationship between every two channels during the awake time; Divide into multiple test time series, calculate the phase lock value PLV in the test time series for each two channels to obtain the coupling relationship between every two channels in the test time series; the phase lock value PLV represents the connection between the two channels Intensity, the phase lock value PLV in the awake time and the test time series is used to form the functional connection matrix of the channel during the awake time and the test time series. The schematic diagram of the functional connection matrix during the awake time is shown in Figure 2a, in the test time series The schematic diagram of the functional connection matrix is shown in Figure 2b; the calculation of PLV uses formula (1):

among them,

S6. According to the connection relationship between channels, the brain function network during the waking time and the test time series is formed. The brain function network during the waking time is shown in Figure 3a, and the brain function network in the test time series is shown in Figure 3b. As shown, Fig. 3c is the difference comparison diagram of Fig. 3a and Fig. 3b; the difference of the brain function network topology in the three sub-bands between the awake time and the test time series is compared and analyzed to determine whether the driving fatigue state is in the test time series.

By collecting the EEG signals of driving during the awake time and the test time, and perform denoising processing, decomposition and reconstruction on it, and then calculate the phase lock in the awake time and the test time series for each two channels respectively Value PLV, and form the functional connection matrix of the channel during the awake time and the test time series according to the PLV, set the connection strength threshold and compare with each element value in the functional connection matrix to obtain the awake time and the test time series respectively The connection relationship between the channels and the formation of the brain function network during the waking time of the subject and the test time series. The difference in the brain function network topology between the waking time and the test time series in the three sub-bands is compared and analyzed to determine the test time series Whether it is in a state of driving fatigue, the reliability and accuracy of detection are high.

Among them, as shown in Figure 1, the EEG acquisition device in step s1 is a wireless stem electrode EEG acquisition device, including 24 electrodes, and the frequency of collecting signals is 250 Hz; the functional connection matrix is 24*24; and the test time is 90 minutes. The improved international 10-20 placement standard is used to place the electrodes. The electrode names are: AFp3h, AFpz, AFp4h, AFF3, AFFz, AFF4, FFC5h, FFC3h, FFCz, FFC4h, FFC6h, CCP5h, CCP1, CCPz, CCP2, CCP6h, PO3, POz, PO4, PO7, O1h, Oz, O2h, PO8, the wireless stem electrode EEG acquisition equipment is published in the literature (Klem G H, Lüders H O, Jasper H H, et al. The ten-twenty electrode system of the International Federation[J]. Electroencephalogr Clin Neurophysiol,1999,52(3):3-6.); the test time of each driving fatigue detection experiment is set at 90 minutes to ensure that the subjects are from the awake state to the fatigue state EEG signals throughout the process.

In addition, in step s1, the subject uses the simulated driving system to drive, and the simulated driving system randomly sends out braking commands to record the time interval between seeing the braking command and responding to the subject; set the time interval threshold, if the reaction time If the interval is greater than or equal to the time interval threshold, the EEG signal during the test time before this time interval will be used as the EEG signal during the awake time, and the EEG signal during the test time after this time interval will be used as the waiting time. Check whether the fatigue signal. The time interval threshold is derived from the training experiment. Due to the individual differences of the subjects, the time interval threshold is not uniform. Therefore, it is necessary to obtain the time interval threshold for individual subjects through the training experiment before the test experiment, that is, before step S1. The calculation method It is the average value of the response time interval during the training experiment when the subject is externally fatigued (such as yawning) or the vehicle's driving path deviates from the normal running track; when driving fatigue, the subject is seeing The time interval between the brake command and the response will be longer. The test time is distinguished by the time interval threshold, and only the EEG signal during the test time after this time interval can be processed as the signal to be detected for fatigue. Reduce the data processed. The subject uses a simulated driving system to drive, also known as car driving simulation, or virtual driving, which refers to the use of modern high-tech means such as: 3D image instantaneous generation technology, vehicle dynamics simulation physical system, large field of view display technology ( Such as multi-channel stereo projection system), six-degree-of-freedom motion platform (or three-degree-of-freedom motion platform), user input hardware system, stereo sound, central control system, etc., allowing the experiencer to feel close to reality in a virtual driving environment The effect of visual, auditory and somatosensory car driving experience is suitable for the needs of research institutions for automotive engineering, traffic engineering, human factors engineering research and as a simulation experiment platform in the field of automotive research.

Among them, the independent component analysis method is used to denoise the EEG signal to remove the interference of the ocular signal and improve the signal-to-noise ratio of the EEG signal.

In addition, in step s3, the wavelet packet transform is used to decompose the EEG signal, and then reconstruct it into three sub-band signals.

Wherein, the connection strength threshold in step s5 is 0.2.

In addition, in step s6, the complex network analysis method is used to quantitatively analyze the differences in the brain function network topology of the three sub-bands during the awake time and the test time series from the aspects of functional integration and functional differentiation. The functional integration includes characteristic paths. Length and global efficiency, functional differentiation includes local efficiency and clustering coefficient;

The clustering coefficient represents the degree of clustering of brain function network nodes, as shown in Figure 4, calculated using formula (2);

Among them, C _i represents the average clustering number of the brain function network, k _i represents the number of adjacent nodes of node i, adjacent nodes are nodes connected to node i, and E _i is the closed-loop triangle existing in adjacent nodes of node i Number; the other two nodes connected to node i, if there are edges between the three nodes, it is a closed-loop triangle;

The global efficiency is the average value of the reciprocal of the shortest path length L _ij , which is used to measure the ability of the brain function network to transmit and process information. As shown in Figure 5, it is calculated by formula (4):

Among them, E _global (G _i ) is the global efficiency of node i, namely E _g .

Obviously, the above-mentioned embodiments of the present invention are merely examples to clearly illustrate the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modification, equivalent replacement and improvement made within the spirit and principle of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

A driving fatigue detection method for constructing a brain function network based on a phase lock value is characterized in that it includes the following steps:

S1. Use the EEG signal acquisition equipment to collect the EEG signals of the subjects when they are awake and driving during the test time;

The position of each electrode in the EEG signal acquisition device is used as a brain function network node, and the number of electrodes is the number of nodes N;

S2. Denoising the EEG signal to improve the signal-to-noise ratio of the EEG signal;

S3. Decompose and reconstruct the denoising EEG signal, and reconstruct three sub-band waveforms according to the frequency range. The frequency of theta wave is 4-8Hz, the frequency of the alpha wave is 8-13Hz, and the frequency of the beta wave is 13- 30Hz;

S4. In the reconstructed signal, each brain function network node is used as a channel; calculate the phase lock value PLV during the awake time for every two channels to obtain the coupling relationship between every two channels during the awake time; Divide into multiple test time series, calculate the phase lock value PLV in the test time series for each two channels to obtain the coupling relationship between every two channels in the test time series; the phase lock value PLV represents the connection between the two channels Intensity, the phase lock value PLV in the awake time and the test time series is used to form the functional connection matrix of the channel during the awake time and the test time series; the calculation of PLV uses the formula (1):

among them,
It is the phase difference between the two channels in the awake time and the test time series. The phase of each channel in the awake time and the test time series is obtained using Hilbert transform, i∈N is the brain function network node, and the PLV value Between [0, 1], 0 means no connection between channels, 1 means complete connection between channels;

S5. Set the connection strength threshold and compare it with the value of each element in the functional connection matrix to obtain the connection relationship between the channels in the awake time and test time series; the element value greater than or equal to the connection strength threshold is the connection between the two channels , Otherwise there is no connection between the two channels;

S6. According to the connection relationship between channels, form the brain function network during the waking time and the test time series; compare and analyze the difference in the three sub-bands of the brain function network topology between the waking time and the test time series to judge Test whether you are in a state of driving fatigue in the time series.
The driving fatigue detection method based on the phase lock value to construct a brain function network according to claim 1, wherein the EEG acquisition device in step s1 is a wireless stem electrode EEG acquisition device, including 24 electrodes, which collect signals The frequency is 250Hz; the function connection matrix is 24*24; the test time is 90 minutes.
The driving fatigue detection method based on the phase lock value to construct a brain function network according to claim 1, characterized in that, in step s1, the subject uses a simulated driving system to drive, and the simulated driving system randomly issues braking commands and records The time interval between the subject seeing the brake command and responding; set the time interval threshold. If the response time interval is greater than or equal to the time interval threshold, the EEG signals from the test time before this time interval will be used as The EEG signal during the awake period will be the EEG signal during the test period after this time interval as the signal to be tested for fatigue.
The driving fatigue detection method for constructing a brain function network based on the phase lock value according to claim 1, wherein in step s2, an independent component analysis method is used to denoise the brain electrical signal to remove the eye electrical signal The interference to improve the signal-to-noise ratio of EEG signals.
The driving fatigue detection method based on the phase-locked value to construct a brain function network according to claim 1, characterized in that, in step s3, wavelet packet transform is used to decompose the EEG signal, and then reconstruct into three sub-band signals .
The driving fatigue detection method for constructing a brain function network based on the phase lock value according to claim 1, wherein the connection strength threshold in step s5 is 0.2.
The driving fatigue detection method for constructing a brain function network based on the phase lock value according to claim 1, characterized in that, in step s6, a complex network analysis method is used to analyze the three sub-bands in terms of functional integration and functional differentiation. Quantitative analysis of the difference in brain function network topology within time and test time series; functional integration includes characteristic path length and global efficiency, and functional differentiation includes local efficiency and clustering coefficient;

The clustering coefficient represents the degree of clustering of brain function network nodes, which is calculated using formula (2);

Among them, C i represents the average clustering number of the brain function network, k i represents the number of adjacent nodes of node i, adjacent nodes are nodes connected to node i, and E i is the closed-loop triangle existing in adjacent nodes of node i Number; the other two nodes connected to node i, if there are edges between the three nodes, it is a closed-loop triangle;

The characteristic path length reflects the information transmission ability inside the brain function network, and is calculated using formula (3):

Among them, L is the characteristic path length; L ij is the shortest path length between node i and node j, that is, the number of shortest edges used from node i to node j; N is the number of nodes;

The global efficiency is the average value of the reciprocal of the shortest path length L ij , which is used to measure the ability of the brain function network to transmit and process information. It is calculated using formula (4):

Among them, N is the number of nodes, and Lij is the shortest path length between node i and node j;

The local efficiency is used to measure the local information transmission and processing ability, and it is calculated by formula (5):

Among them, E global (G i ) is the global efficiency of node i, namely E g .