CN114343644A

CN114343644A - Fatigue driving detection method and equipment

Info

Publication number: CN114343644A
Application number: CN202111675453.0A
Authority: CN
Inventors: 罗煜; 刘彬
Original assignee: Beijing Fenghuo Wanjia Technology Co ltd
Current assignee: Beijing Fenghuo Wanjia Technology Co ltd
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2022-04-15

Abstract

The application discloses a fatigue driving detection method, which comprises the following steps: taking reaction time and electroencephalogram magnetic signal data of the same sample in a set time range as a group of characteristic data, collecting a plurality of groups of characteristic data through a driving environment to form a data set, and training a classifier through a deep learning algorithm; and (4) processing the electroencephalogram magnetic signal data acquired in real time, and identifying the fatigue degree. The application also includes an apparatus for implementing the method. The problem that fatigue driving cannot be accurately and objectively detected and quantified in the prior art is solved.

Description

Fatigue driving detection method and equipment

Technical Field

The application relates to the technical field of computers, in particular to a method and equipment for detecting fatigue driving based on electroencephalogram magnetic signals.

Background

Driving safety is an important public issue. According to the world health organization statistics, over 120 million people die of car accidents every year worldwide, wherein fatigue driving results in a significant number of road traffic deaths and non-fatal injuries. It is necessary to monitor the fatigue level of the driver to control and reduce road traffic injuries. There is a good technology for detecting the problems of drunk driving, overspeed and overload, but there is no reliable fatigue driving detection technology at present. The difficulty in fatigue driving detection is that fatigue driving is a process, unlike drunk driving being a point in time, there is no reliable objective index for fatigue level determination.

Previous studies have demonstrated that drowsy driving is highly correlated with pre-accident behavioral errors. There are three main approaches to monitoring fatigue driving. One approach is to use fatigue-related gauges to detect fatigue. Another approach is to target the driver's behavior by detecting the movement of the steering wheel or the deviation of the vehicle. Yet another approach is to monitor physiological parameters such as the heart rate, respiration and respiration rate of the driver. However, these methods have limitations. For example, the method for detecting fatigue by using the scale is easily influenced by the subjective psychological state of a driver and is not objective; for example, the method for detecting the deviation may be interfered by other factors such as rainy and snowy weather, emergency obstacle avoidance and the like.

Previous studies have also shown that the frequency band of the EEG signal reflects psychological conditions, such as increased energy in the alpha band when the participant is in a drowsy state. Previous studies have demonstrated various methods of processing multidimensional data for cognitive workload assessment: temporal information is essential in EEG analysis, the concatenation of adjacent time frames representing trends in brain dynamic state. Spatial information is a hot spot of recent research, and for example, cognitive load is closely related to frontal lobe, parietal lobe and the like. In some studies, electrode locations are combined into EEG readings and EEG signals are converted into a topographic map. Thus, the use of electroencephalography may be a promising and viable method of fatigue monitoring. However, how to monitor the fatigue degree by using the brain electromagnetic signal is under further study, and a technical solution for accurately monitoring and quantifying the driving fatigue degree needs to be researched.

Disclosure of Invention

The application provides a fatigue driving detection method and equipment, which overcome the problem that fatigue cannot be accurately and objectively detected and quantified in the prior art, and provide a technical scheme for quantifying and objectively detecting driving fatigue.

On one hand, the embodiment of the application provides a fatigue driving detection method, which comprises the following steps:

taking reaction time and electroencephalogram signal data of the same sample in a set time range as a group of feature data, collecting a plurality of groups of feature data under a driving environment to form a data set, determining fatigue degree according to the reaction time, and training a classifier by using the feature data through a deep learning algorithm;

and processing the electroencephalogram and magnetic signal data acquired in real time by using the classifier, and identifying the fatigue degree.

Further, the generation process of the brain electromagnetic signal data comprises the following steps:

collecting brain electromagnetic signals by brain electrical collecting equipment;

and (3) carrying out interference removal, tracing and positioning and neural oscillation analysis on the brain electromagnetic signals to obtain the brain electromagnetic signal data.

Further, the interference removal includes any combination of the following steps:

filtering is performed using a finite-length unit impulse response filter. The pass band is (1-80 Hz);

removing power frequency interference by using a notch filter;

performing baseline correction on the electroencephalogram magnetic signals to reduce data drift;

and removing the motion noise.

In one embodiment, the source tracing location is to use an inverse problem solving algorithm to solve the signal source located in cortex after assuming known brain internal source signals, calculating the epidermal voltage according to the electrical conductivity of each tissue of the head of the brain, and establishing a head volume conduction model.

In one embodiment, the neural oscillation analysis further comprises the steps of:

wavelet transformation is carried out on the traced electroencephalogram magnetic signals, and the electroencephalogram magnetic signals are decomposed into 6 basic rhythms: delta (1-4Hz), theta (5-7Hz), alpha (8-12Hz), beta (15-29Hz), gamma (30-59Hz), and gamma' (60-80 Hz); and calculating the normalized power spectral density of the brain electromagnetic signals under each basic rhythm to serve as brain electromagnetic signal data.

Preferably, the electroencephalographic signal is an EEG signal and/or a MEG signal.

Preferably, the deep learning algorithm is a long-time memory LSTM algorithm or a Support Vector Machine (SVM) algorithm.

On the other hand, the embodiment of the application further provides a fatigue driving detection device, which is used for realizing the method in any one embodiment of the application.

The test module is used for testing reaction time to obtain a reaction time index.

The acquisition module is used for acquiring electroencephalogram signals.

The processing module is used for removing interference, tracing and positioning, and analyzing neural oscillation on the electroencephalogram magnetic signals to obtain electroencephalogram magnetic signal data.

And the classification module is used for classifying the electroencephalogram magnetic signal data according to the fatigue degree by using a support vector machine or a long-time memory deep learning algorithm.

The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:

compared with the prior art, the fatigue detection system has the advantages that reliable and objective fatigue detection progress is achieved, the accuracy rate of the fatigue detection can reach 89.6%, the precision of the fatigue detection is improved, meanwhile, closed-loop feedback is provided, and the driving safety is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a flowchart of an embodiment of a method for detecting fatigue driving by electroencephalogram and magnetic signals according to the present application;

FIG. 2 is a diagram illustrating the processing of electroencephalogram and magnetic signals in the preferred embodiment of the present application;

FIG. 3 is a schematic diagram of the test at reaction time;

fig. 4 is an embodiment of the electroencephalogram magnetic signal fatigue driving detection apparatus of the present application;

fig. 5 is an embodiment of a brain electromagnetic signal processing module.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a flowchart of an embodiment of a fatigue driving detection method according to the present application.

The embodiment of the application provides a fatigue driving detection method, which comprises the following steps:

and 11, taking reaction time and electroencephalogram magnetic signal data from the same sample in a set time range as a group of characteristic data, and collecting a plurality of groups of characteristic data in a driving environment to form a data set.

For example, in real driving scenarios or simulated driving test equipment, let the subject perform driving tasks and record reactions.

Further, the generation process of the brain electromagnetic signal data comprises the following steps: collecting electroencephalogram magnetic signals by high-density electroencephalogram collecting equipment; and (3) carrying out interference removal, tracing and positioning and neural oscillation analysis on the brain electromagnetic signals to obtain the brain electromagnetic signal data.

The source tracing positioning means that the signal source located in the cortex is solved according to the acquired electroencephalogram magnetic signals after a head volume conduction model is established by assuming known brain internal source signals and calculating the epidermal voltage according to the electric conductivity of each tissue of the head of the brain.

The neural oscillation analysis refers to frequency domain analysis of the internal brain source signals obtained after tracing to obtain brain region activation energy values of all rhythms. Preferably, the normalized power spectral density of the brain electromagnetic signals under each fundamental rhythm is calculated as brain electromagnetic signal data.

The term "electromagnetic" in this application is a generic term for both electricity and magnetism. The electroencephalogram signal herein refers to an electroencephalogram signal (EEG) or an electroencephalogram signal (MEG). Therefore, preferably, the electroencephalogram magnetic signal includes an EEG signal and/or an MEG signal, and accordingly, the electroencephalogram magnetic signal data may include at least one of electroencephalogram signal data (or electroencephalogram data for short) and electroencephalogram magnetic signal data (or brain magnetic data for short).

It should be noted that the plurality of sets of feature data may be from different samples or the same sample, where the sample refers to the tested driver.

For example, the entire data set includes 71 sets of test data from 37 healthy participants, each set containing 60 electroencephalographic set-up periods. Participants had no history of psychological or sleep disorders. For another example, the entire data set includes multiple sets of test data from 1 driver at different times, with each set of test data including multiple electroencephalogram set time periods.

In one embodiment, all electroencephalographic data is acquired through a 128-channel electroencephalographic cap and a neural scanning system. The electrode impedance is less than 5k omega, and the sampling rate of electroencephalogram signals is 1000 Hz.

And step 12, classifying the fatigue degree through a deep learning algorithm.

Determining the fatigue degree according to the reaction time; and training a classifier through a deep learning algorithm by using the characteristic data. Preferably, the deep learning algorithm is a long-time memory LSTM algorithm or a Support Vector Machine (SVM) algorithm. For example, classification takes 70% of the data as a training set and 30% of the data as a test set, and five-fold cross validation is performed.

During training, the fatigue degree is labeled through a reaction range value range of the reaction, and corresponding electroencephalogram and magnetic signal data are classified.

The fatigue level is determined from the reaction time, and for example, when the reaction time is greater than a set threshold, the fatigue state is determined, and when the reaction time is less than the set threshold, the non-fatigue state is determined. Or, the reaction time segment corresponds to the fatigue degree grade, for example, the reaction time is greater than a first set threshold value and is a first fatigue degree; and when the reaction time is less than the first set threshold and greater than the second set threshold, the second fatigue degree is obtained.

It should be noted that, when the multiple sets of feature data used for training are from different samples, the trained classifier has a large applicable crowd range; when the sets of feature data used for training are from the same sample only, the classifier in question is most suitable for the sample providing the feature data.

And step 13, processing the characteristic data acquired in real time based on the classification result, and identifying the fatigue degree.

Fig. 2 shows the electroencephalogram and magnetic signal processing process according to the preferred embodiment of the present application.

and step 21, acquiring the electroencephalogram magnetic signals by using electroencephalogram acquisition equipment.

Preferably, a 128-lead high density brain electrical acquisition device is used to acquire and store continuous raw brain electrical or brain magnetic signals (EEG or MEG). For example, EEG has a higher temporal resolution (in milliseconds), and high-density EEG systems are full of brains, increasing spatial resolution relative to low-density EEG systems to improve fatigue detection accuracy.

It should be noted that the high-density electroencephalogram acquisition device can realize the acquisition of full-brain high-density brain electromagnetic signals and provide a basis for the analysis of electroencephalogram electromagnetic signals, for example, has 128 channels. Because partial electrodes are irrelevant to electroencephalogram fatigue characteristics, redundant electrodes such as electro-oculogram electrodes can be deleted after sampling, the electrodes do not contribute to fatigue characteristic extraction and are deleted so as not to cause interference, and then an electrode channel position file is led in to obtain accurate spatial position registration; in addition, when the data collected by some channels are bad, the bad leads can be deleted or repaired by using an interpolation algorithm.

And step 22, removing interference on the electroencephalogram magnetic signals through a preprocessing process.

The interference removal comprises the following steps of:

filtering (1-80Hz) by using a finite-length unit impulse response filter, for example, realizing high-pass filtering (>1Hz) and low-pass filtering (<80 Hz);

the power frequency interference is removed by using a filter, for example, a notch filter (50Hz or 60Hz, depending on the AC power frequency of each country).

Performing baseline correction on the electroencephalogram magnetic signals to reduce data drift; for example, successive brain electrical signals are segmented into a 6 second baseline signal, starting 6 seconds before the event occurs, and then baseline correction is performed to reduce the bias caused by data drift.

Removing action noise specifically comprises the following steps: removing noise such as blink, eye movement, muscle movement and the like by using a blind source separation algorithm; noise associated with eye movement and muscle activity is artificially removed.

Note that to eliminate other differences between samples, the data for each subject participant were normalized for both the measured response time and the electroencephalography kinetics. The data sets collected from each subject were normalized by subtracting the mean and dividing by the standard deviation.

And finally, storing the preprocessed electroencephalogram data without obvious artifacts.

And 23, tracing and positioning the electroencephalogram magnetic signals after the interference is removed.

The source tracing location technology comprises solving a positive problem and solving an inverse problem.

A three-dimensional head model for each subject is built based on each subject head magnetic resonance data and the boundary element method, thereby solving a positive problem. The positive problem is that under the condition of the known source signals in the brain, the electric conductivity of each tissue of the head is obtained through the methods of anatomy and the like, then the voltage of the epidermis is calculated, and a head volume conduction model is established through continuously fitting real data.

Solving the inverse problem, preferably, estimating the source by using a Beamformer algorithm, namely, estimating and solving the source signal distribution of the cortex according to the electroencephalogram cap sensor array signal.

The tracing and positioning technology based on the high-density electroencephalogram magnetic signals can represent the dynamic activity of the brain with higher time resolution and spatial resolution, and provide important characteristic data for quantitatively monitoring the fatigue degree.

And 24, carrying out nerve oscillation analysis on the traceable and positioned brain internal source signal to obtain the brain electromagnetic signal data.

performing Morie wavelet transform on the traced electroencephalogram magnetic signals, converting the electroencephalogram magnetic signals from a time domain to a frequency domain, and decomposing the electroencephalogram magnetic signals into 6 basic rhythms: delta (1-4Hz), theta (5-7Hz), alpha (8-12Hz), beta (15-29Hz), gamma (30-59Hz), and gamma' (60-80 Hz);

respectively calculating the activation energy value of the brain area of each rhythm, further determining the fatigue degree according to the relative energy value, specifically, calculating the normalized power spectral density of the brain electromagnetic signals under each basic rhythm, and normalizing the power spectral density by subtracting the average value and dividing the power spectral density by the standard deviation to be taken as brain electromagnetic signal data.

When the alpha and beta rhythms are dominant, it is indicated that the human consciousness is awake; when the delta and theta rhythms are dominant, the fuzzy consciousness and even slight sleep of the human are indicated; when the gamma and gamma' rhythms become dominant, it indicates that the person is in an attentive state.

In the processing process of steps 21 to 24, the acquisition of electroencephalogram signals of 128 channels, 64 channels, 32 channels, 16 channels and the like can be realized through commercial electroencephalogram detection equipment. It can be understood that the higher the channel density of electroencephalogram detection is, the more complete the detected signal is and the more accurate the result of source tracing analysis is. Meanwhile, the larger the channel density of electroencephalogram detection is, the more physical resources and time are required for data processing.

In order to improve the cost performance of the device, when the classifier is trained and/or detected in real time, the result of training the classifier by testing the characteristic data under different channel densities and the result of classifying the electroencephalogram and magnetic signals detected in real time under different channel densities by using the classifier can be tested, and the minimum channel density with the accuracy exceeding a set threshold value is obtained.

It can be understood that the brain electromagnetic signal data described in the present application may be data generated by an electroencephalogram or magnetoencephalography detection device, such as the electromagnetic signal tested on the surface of the human body in step 21; or may be data obtained as in step 22 after the interference cancellation process; excitation signal data of each sampling point in the cerebral cortex layer obtained through tracing positioning calculation can be obtained; and the spectrum characteristic data can be further obtained through nerve oscillation analysis. When any kind of electroencephalogram magnetic signal data is used for training the classifier, the classifier should use the electroencephalogram magnetic signal data of the same kind collected in real time for fatigue detection.

FIG. 3 is a schematic diagram of the test at the time of reaction.

The reaction time is the delay time between the occurrence time when the vehicle randomly deviates from the center position of the lane and the time when the participant responds to return to the center position, high reaction time indicates that the participant is relatively tired, and low reaction time indicates that the participant is relatively alert.

In testing the response, the experimental task of the person being tested was to keep driving in a lane. For example, a responsive sensing steering wheel is used on a moving platform to simulate a real driving environment. The vehicle in the experiment was driven at a speed of 80 km/h, and the vehicle was randomly deviated from the cruising lane at the center position. The tester is required to control the vehicle to return to the center of the lane as soon as possible when a deviation is recognized.

Fig. 4 is a flowchart of an embodiment of the electroencephalogram magnetic signal fatigue driving detection apparatus according to the present application.

The embodiment of the application also provides an electroencephalogram magnetic signal fatigue driving detection device, which is used for realizing the method in any one of the embodiments of the application, and the device comprises a test module 41, an acquisition module 42, a processing module 43, a classification module 44, an alarm module 46 and a data module 47.

The testing module is used for testing reaction time and obtaining reaction time data, as described in step 11. For example, in a simulated driving device, a subject is asked to perform a driving task and to record a reaction time.

The acquisition module is used for acquiring the electroencephalogram magnetic signals, as in step 21. For example, the signal acquisition module is used in a simulated driving device to acquire electroencephalogram and magnetic signals when a participant performs a driving task in a simulated driving scene by using a 128-lead high-density electroencephalogram technology, or a 128-channel high-density electroencephalogram acquisition device is used to acquire EEG signals and/or MEG signals when a driver drives a vehicle in real time during real driving.

And the processing module is used for removing interference, tracing and positioning and analyzing neural oscillation on the electroencephalogram magnetic signals to obtain electroencephalogram magnetic signal data, and the steps are as follows 22-24. For example, the interference is removed by performing preprocessing analysis such as filtering and denoising on the acquired electroencephalogram magnetic signals.

The data module is used for storing a data set formed by collecting a plurality of groups of characteristic data through a real or simulated driving environment, storing EEG signals and/or MEG signals collected in real time when a driver drives a vehicle, and generating EEG signal data through the processing module.

And the classification module is used for classifying the brain electromagnetic signal data or the characteristic data according to the fatigue degree by using a support vector machine or a long-time memory deep learning algorithm, as shown in the step 12-13. Reaction time and electroencephalogram characteristics are respectively extracted based on analysis results of the test module and the signal processing module, and electroencephalogram magnetic signal data are classified based on the characteristics by using a Support Vector Machine (SVM) and a long-short-term memory (LSTM) deep learning algorithm.

It should be noted that a Long Short Term Memory network (LSTM) is an improved form of a Recurrent Neural Network (RNN), and a basic unit of the LSTM is called a Memory block and is composed of a central node and 3 gate control units. The central node is usually called a memory cell for storing the current network state, and the 3 gate control units are usually called an input gate, an output gate and a forgetting gate respectively for controlling the information flow in the memory block. In the forward propagation process, the input gate is used for controlling the information flow input to the memory cell, and the output gate is used for controlling the information flow from the memory cell to other structural units of the network; during the back propagation, the input gate is used to control the iterative error to flow out of the memory cell, and the output gate is used to control the iterative error to flow into the memory cell. The forgetting gate is used to control the circulation state inside the memory cell and determine the accepting or forgetting of information. Through the gating mechanism, the LSTM network can control the information flow in the unit, so that the LSTM network has the capability of storing long-time information, namely the memory capability, can prevent the internal gradient from being interfered by the outside in the training process, and avoids the problems of gradient dispersion and gradient explosion.

And the alarm module responds to the fact that the fatigue degree exceeds a set threshold value and sends out alarm information. For example, when the classification module detects that the participant is in the fatigue degree, the classification module gives a sound stimulus to the participant to remind the participant to pay attention so as to improve the driving safety.

Further, the processing module 43 further includes an interference elimination module 51, a source tracing module 52, and an analysis module 53.

The interference removing module is used for removing interference on the electroencephalogram magnetic signals in the preprocessing process, and the specific function is realized as the step 22.

The tracing module is used for tracing and positioning the brain electromagnetic signals, and the specific functions are realized as in step 23.

And the analysis module is used for carrying out nerve oscillation analysis on the traceable and positioned brain internal source signals to obtain the brain electromagnetic signal data. The specific function realized is as described in step 23.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application therefore also proposes a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of the embodiments of the present application.

Further, the present application also proposes an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any of the embodiments of the present application.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

In order to further explain the beneficial effects of the scheme of the application, the effectiveness of the fatigue monitoring method is verified by designing a simulation driving experiment through a simulation driving system. The classification accuracy based on the "@ NASA _ TLX scale" and "reaction + EEG features" was compared. The meaning of the @ NASA _ TLX scale is that only the result of the NASA _ TLX scale is included, and the meaning of the 'reaction time + electroencephalogram characteristic' is that the input characteristic comprises the reaction time and the electroencephalogram characteristic, wherein the electroencephalogram characteristic comprises the result of neural oscillation analysis.

It should be noted that NASA-TLX (NASA task load index) is a widely used subjective, multidimensional assessment tool that rates perceived workload to assess the efficiency or other performance of a task, system or team. It was developed by the human performance group of the U.S. space agency eims research center over a three year development cycle, including over 40 laboratory simulations. It is cited in 4400 for several studies.

37 subjects were scheduled for simulated driving trials between 20 and 28 years of age, including 19 males and 18 females. To ensure that the brain state of the participants was not affected by external factors, drinking of alcohol or caffeine-containing beverages and vigorous exercise were prohibited one week prior to the experiment. All participants had a clear understanding of the operation of the relevant equipment and the experimental procedures prior to participation in the experiment and had fully practiced the experimental procedures. The participants did not have any history of psychological or sleep disorders. Participants were asked to conduct simulated driving experiments for more than 2 consecutive hours and EEG acquisition and analysis and fatigue detection using the fatigue detection method of the invention. The vehicle speed during the simulated driving is 80 km/h.

The present study assesses driver driving performance by measuring reaction time. The simulated driving experiments were performed early in the afternoon (13:00-15:00) after lunch, when the circadian rhythm of drowsiness reached a peak. In addition, the virtual road scene is monotonous, the task requirement is low, and sleepiness is induced. Under such conditions, it is difficult for participants to regulate attention and performance, which results in a long reaction time.

Reaction time was tested using EEG data (baseline time) 6 seconds prior to the occurrence of the event as input data for the SVM and LSTM, and the time period between the occurrence of the event and the start of the reaction was defined as the reaction. The time between two consecutive trials is about 7 to 12 seconds. The reaction time is shorter than 10 seconds, and better task performance is reflected; the reaction time is longer than 10 seconds, reflecting poor task performance.

Since the EEG itself has a high temporal resolution (in milliseconds), a 128-channel high-density EEG system spreads over the whole brain, increasing the spatial resolution relative to a low-density EEG system to improve the fatigue detection accuracy. All electroencephalographic data was collected via a 128-channel electroencephalographic cap and a neuroscanning system. The electrode impedance is less than 5k omega, and the sampling rate of electroencephalogram signals is 1000 Hz.

NASA _ TLX scale tests were performed on all participants before and after the experiment.

After obtaining the NASA _ TLX scale and the reaction time and the EEG data, analysis is carried out, such as preprocessing, tracing positioning, neural oscillation analysis and the like on the EEG data, and then classification accuracy, sensitivity and specificity under different conditions based on the reaction time and EEG characteristics are calculated by using an LSTM algorithm and an SVM algorithm.

The sensitivity here is the proportion of samples judged to be fatigue among samples actually judged to be fatigue; the specificity here is a ratio of samples that are actually not fatigued, and are judged to be not fatigued. The classification accuracy here refers to a ratio of samples in which an actual fatigue degree and a judged fatigue degree are consistent in all the classes of the fatigue degree, wherein the actual fatigue degree is measured by a value at the time of reaction, and the judged fatigue degree is determined by classifying, identifying and determining the electroencephalogram signals.

The method is judged from the numerical value, and the classification accuracy, the sensitivity and the specificity classification accuracy are greatly improved after the electroencephalogram characteristics are used in any algorithm. The LSTM classification algorithm performs better on the specific classification method.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A fatigue driving detection method is characterized by comprising the following steps:

2. The fatigue driving detection method according to claim 1, wherein the generation process of the brain electromagnetic signal data comprises the steps of:

the electroencephalogram acquisition equipment acquires an electroencephalogram magnetic signal;

3. The fatigue driving detection method according to claim 2, wherein the removing of the disturbance comprises a combination of any of the following steps:

filtering by using a finite-length unit impulse response filter, wherein the pass band range is 1-80 Hz;

removing power frequency interference by using a notch filter;

and removing the motion noise.

4. The fatigue driving detection method according to claim 2, wherein the tracing location is used for solving the signal source located in cortex by using an inverse problem solving algorithm after assuming known brain internal source signals, calculating skin voltage according to the electrical conductivity of each tissue of the head of the brain, and establishing a head volume conduction model.

5. The fatigue driving detection method of claim 2, wherein the neuro-oscillation analysis comprises the steps of:

6. The fatigue driving detecting method according to claim 1,

the brain electromagnetic signals include EEG signals and/or MEG signals.

7. The fatigue driving detecting method according to claim 1,

the deep learning algorithm is an LSTM (least squares) algorithm or a SVM (support vector machine) algorithm.

8. A fatigue driving detection device is used for realizing the method of any one of claims 1 to 7, and is characterized by comprising a testing module, an acquisition module, a processing module and a classification module;

the test module is used for testing reaction time and obtaining reaction time data;

the acquisition module is used for acquiring electroencephalogram magnetic signals;

the processing module is used for removing interference, tracing and positioning and analyzing neural oscillation on the electroencephalogram electromagnetic signals to obtain electroencephalogram electromagnetic signal data;

9. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method of any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to any of claims 1 to 7 when executing the computer program.