CN117017298A - Method and system for judging attention of driver - Google Patents
Method and system for judging attention of driver Download PDFInfo
- Publication number
- CN117017298A CN117017298A CN202311025026.7A CN202311025026A CN117017298A CN 117017298 A CN117017298 A CN 117017298A CN 202311025026 A CN202311025026 A CN 202311025026A CN 117017298 A CN117017298 A CN 117017298A
- Authority
- CN
- China
- Prior art keywords
- driver
- data
- attention
- vehicle
- physiological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000008569 process Effects 0.000 claims abstract description 29
- 238000013528 artificial neural network Methods 0.000 claims abstract description 23
- 210000004369 blood Anatomy 0.000 claims abstract description 17
- 239000008280 blood Substances 0.000 claims abstract description 17
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims abstract description 14
- 229910052760 oxygen Inorganic materials 0.000 claims abstract description 14
- 239000001301 oxygen Substances 0.000 claims abstract description 14
- 230000000694 effects Effects 0.000 claims abstract description 12
- 230000001133 acceleration Effects 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 56
- 239000013598 vector Substances 0.000 claims description 36
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 230000036760 body temperature Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 5
- 230000008859 change Effects 0.000 description 6
- 210000003403 autonomic nervous system Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 108010064719 Oxyhemoglobins Proteins 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008451 emotion Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 210000002820 sympathetic nervous system Anatomy 0.000 description 3
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 description 2
- 108010054147 Hemoglobins Proteins 0.000 description 2
- 102000001554 Hemoglobins Human genes 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 230000036772 blood pressure Effects 0.000 description 2
- 210000004204 blood vessel Anatomy 0.000 description 2
- 108010002255 deoxyhemoglobin Proteins 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000002996 emotional effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 210000000106 sweat gland Anatomy 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- 238000002834 transmittance Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000037007 arousal Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000036770 blood supply Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000006996 mental state Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000008035 nerve activity Effects 0.000 description 1
- 230000001734 parasympathetic effect Effects 0.000 description 1
- 210000001002 parasympathetic nervous system Anatomy 0.000 description 1
- 210000005259 peripheral blood Anatomy 0.000 description 1
- 239000011886 peripheral blood Substances 0.000 description 1
- 230000004962 physiological condition Effects 0.000 description 1
- 230000006461 physiological response Effects 0.000 description 1
- 230000010349 pulsation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 231100000430 skin reaction Toxicity 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 230000002889 sympathetic effect Effects 0.000 description 1
- 230000001256 tonic effect Effects 0.000 description 1
- 230000024883 vasodilation Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/168—Evaluating attention deficit, hyperactivity
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/20—Workers
- A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
- A61B5/02405—Determining heart rate variability
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/053—Measuring electrical impedance or conductance of a portion of the body
- A61B5/0531—Measuring skin impedance
- A61B5/0533—Measuring galvanic skin response
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/1455—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters
- A61B5/14551—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters for measuring blood gases
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Cardiology (AREA)
- Developmental Disabilities (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Social Psychology (AREA)
- Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Educational Technology (AREA)
- Child & Adolescent Psychology (AREA)
- Pulmonology (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to the field of biomedical information acquisition and processing, in particular to a method for judging the attention of a driver, which comprises the steps of arranging electrodes and sensors on a vehicle direction control device, and acquiring physiological data of the driver through the electrodes and the sensors in the driving process of the driver, wherein the physiological data at least comprise heart rate variability, skin electric activity, blood oxygen saturation and temperature; collecting vehicle state data of a driver in the driving process, wherein the vehicle state data at least comprises vehicle speed, steering wheel rotation angle and acceleration; inputting the driver sign data and the vehicle state data into a pre-trained neural network, the neural network deriving a score for the driver's attention; the invention effectively analyzes the attention level of the driver by utilizing the physiological parameters and the vehicle data, and has good practicability and innovation.
Description
Technical Field
The present invention relates to the field of biomedical information acquisition and processing, and in particular to a method and system for determining driver attention.
Background
With the high-speed development of economy, automobiles gradually become necessities in life of people, irregular driving is a main cause of traffic accidents, especially buses and trucks, and once the traffic accidents occur, huge casualties can be caused. The driver attention analysis can remind the driver to drive normally or help the driver to take emergency measures when the driver is in sudden illness, so that traffic accidents are avoided. Driver attention analysis is an important research direction in the field of automobile safety, aiming at improving the safety level and driving efficiency of drivers. With the continued development of sensor technology and neural network algorithms, more and more studies have employed a variety of physiological signals to assess the driver's attention level, including heart rate variability, galvanic skin activity, blood oxygen saturation, body temperature, and driver control parameters for vehicles, among others.
The human Autonomic Nervous System (ANS) is divided into two parts: the sympathetic nervous system and the parasympathetic nervous system. When a person is under stress, some changes in the human autonomic nervous system occur. In the stress state, the activity of the Sympathetic Nervous System (SNS) increases, while the Parasympathetic Nervous (PNS) activity decreases. Compared with the secrecy of subjective report, the physiological parameter monitoring can break through the defects existing in the subjective report. Physiological signals and physiological indicators that may be used to measure attention include, but are not limited to: electroencephalogram (EEG), myoelectricity (EMG), dermatography (GSR), heart Rate (HR), skin Temperature (ST), heart Rate Variability (HRV), blood Pressure (BPR), blood oxygen saturation (Spo 2), and the like.
In the aspect of driver attention analysis, patent number EP15737409.1 proposes a method for monitoring a safe driving state of a driver based on vision, which comprises detecting a current line-of-sight direction of the driver in real time, acquiring a scene image signal in a front view field of the driver when the vehicle is running, processing the acquired current road scene image signal according to a visual attention calculation model to obtain an expected attention distribution of the driver in a current road scene, performing fusion analysis on the current line-of-sight direction of the driver and the expected attention distribution of the driver, and judging whether the current driver is in a normal driving state. Patent No. US17034307 proposes capturing a video of a driving area of a vehicle by a camera provided on the vehicle, determining a type of gaze area of the driver in a face image frame from each of a plurality of frames of face images of the driver in the driving area contained in the video, and determining a concentration monitoring result of the driver from a type distribution of gaze areas of the face image frames included in at least one sliding time window in the video. The above patents all evaluate the attention of the driver on the basis of visual information, which cannot comprehensively reflect the attention level of the driver due to the individual differences of the driver.
Since the attention of a person is affected not only by the environment but also by psychological factors and health conditions, judging the attention level of a driver from the line of sight and external factors of the driver has a certain limitation. The change in human attention may cause a change in a physiological parameter, so it is more convincing to evaluate the driver's attention level from the point of view of the physiological signal.
Disclosure of Invention
In order to solve the above problems, the present invention proposes a method for determining the attention of a driver, comprising the steps of:
electrodes and sensors are arranged on the vehicle direction control device, physiological data of a driver are collected through the electrodes and the sensors in the driving process of the driver, and the physiological data at least comprise heart rate variability, skin electric activity, blood oxygen saturation and temperature;
collecting vehicle state data of a driver in the driving process, wherein the vehicle state data at least comprises vehicle speed, steering wheel rotation angle and acceleration;
the driver sign data and the vehicle state data are input into a pre-trained neural network, which derives a score for the driver's attention.
Further, a sample entropy value is calculated according to the physiological data of the driver, a feature vector set is constructed based on the sample entropy value, and the feature vector set is used as an input of the pre-trained neural network.
Further, the process of obtaining the feature vector set includes:
denoising the physiological signal, performing empirical mode decomposition after denoising, and calculating a contribution value of each inherent mode function;
and screening k inherent mode functions with highest contribution values to construct a feature vector set.
Further, filtering the interference signal by using a band-pass filter on the physiological data specifically includes:
wherein Z (t) represents a physiological signal after filtering the interference signal by using a band-pass filter; z is a physiological signal acquired by a sensor or an electrode; f (f) s Is the sampling frequency; f (f) c1 And f c2 Are the two cut-off frequencies of the band-pass filter.
Further, the empirical mode decomposition of the physiological signal after noise removal specifically includes:
finding out all maximum value points and minimum value points of the physiological signal Z (t) after filtering the interference signal by using a band-pass filter, and fitting by using a cubic spline interpolation function to obtain an upper envelope e max (t) and lower envelope e min (t)；
Calculating the mean value e of the upper envelope curve and the lower envelope curve avg (t) extracting new data based on the obtained mean, expressed as h (t) =z (t) -e avg (t)；
Judging whether h (t) is an inherent mode function, if so, marking the h (t) as an inherent mode function, replacing the original Z (t) with Z (t) -h (t), otherwise, replacing the original Z (t) with h (t);
repeating the above process until all the inherent mode functions are found, and finally, representing the physiological signals after noise removal as:
therein, imf m (t) represents an mth natural mode function; r is (r) M And (t) is a residual component.
Further, the process of constructing the feature vector set by the intrinsic mode function includes:
dividing each natural mode function into a plurality of segments according to different time windows, wherein each segment comprises a plurality of data;
if an inherent mode function is divided into N segments, each segment has N data, constructing a group of vectors in m-dimensional space according to the N data;
calculating the distance between m-dimensional space vectors of each data, and screening out the data smaller than the set similarity tolerance;
calculating a ratio between the number of data less than the set similarity margin and N-m+1
According to the ratioCalculating a sample entropy value, and forming a feature vector set by the flooded entropy value of each piece of data of each inherent mode function;
where N-m+1 is the total number of vectors for a set of m-dimensional space vectors constructed from N data.
Further, according to the ratioCalculating the sample entropy value includes:
where SampEn (m, r, N) represents a sample entropy value.
Further, the calculation of the contribution value of the intrinsic mode function includes:
wherein λ is the variance contribution of the natural mode function;variance, sigma, of the ith natural mode function 2 Sum of variances of all natural mode functions; x is x i (t) is the value of the ith natural mode function at time t,/>The average value of the ith natural mode function in the whole time range is taken, and N is the sampling point number of the signal.
The invention also provides a system for judging the attention of a driver, which is used for realizing a method for judging the attention of the driver, and comprises a signal acquisition circuit, a vehicle data acquisition module, an attention analysis unit, a pre-training neural network module and an attention level decision module, wherein:
the signal acquisition circuit is used for acquiring physiological parameters of a driver in the driving process, and the physiological signals at least comprise: heart rate, blood oxygen saturation, galvanic skin activity, body temperature;
the vehicle data acquisition module is used for collecting vehicle control of a driver in the running process of the vehicle, namely vehicle state data, wherein the vehicle state data at least comprise: vehicle speed, vehicle acceleration, steering wheel yaw angle;
the attention analysis unit comprises a data storage module and a signal preprocessing module, wherein the data storage module is used for storing physiological parameters of a driver and data of the driver on vehicle control acquired in the running process of the vehicle; the signal preprocessing module is used for constructing a characteristic vector set according to physiological signals;
a pre-training neural network module for assessing a driver's level of attention from the driver's physiological data and vehicle data;
and the attention level decision module is used for evaluating the attention level of the driver according to the model and making different countermeasures.
Compared with the prior art, the invention adopts a non-invasive technology to utilize the physiological signals of the driver and the operation behaviors of the vehicle under the current physiological condition as the evaluation characteristics of the attention level; the deep learning neural network has the capability of analyzing complex data, and can effectively model the relationship between physiological parameters, vehicle data and driver attention, so as to obtain more accurate analysis results. Meanwhile, the collected data of the driver can be retrained, personalized preference of the driver is realized, and accuracy of the evaluation level is further improved. The invention effectively analyzes the attention level of the driver by utilizing the physiological parameters and the vehicle data, and has good practicability and innovation.
Drawings
FIG. 1 is a block diagram of a driver attention analysis method and system in accordance with the present invention;
FIG. 2 is a block diagram of a physiological signal acquisition circuit according to the present invention;
FIG. 3 is a flow chart of the driver attention analysis and system of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention provides a method for judging the attention of a driver, which specifically comprises the following steps:
electrodes and sensors are arranged on the vehicle direction control device, physiological data of a driver are collected through the electrodes and the sensors in the driving process of the driver, and the physiological data at least comprise heart rate variability, skin electric activity, blood oxygen saturation and temperature;
collecting vehicle state data of a driver in the driving process, wherein the vehicle state data at least comprises vehicle speed, steering wheel rotation angle and acceleration;
the driver sign data and the vehicle state data are input into a pre-trained neural network, which derives a score for the driver's attention.
In this embodiment, a system for determining attention of a driver is provided, and as shown in a block diagram of fig. 1, the system mainly includes:
the signal acquisition circuit is used for acquiring physiological parameters of a driver in the driving process, and the physiological signals at least comprise: heart rate, blood oxygen saturation, galvanic skin activity, body temperature;
the vehicle data acquisition module is used for collecting vehicle control of a driver in the running process of the vehicle, namely vehicle state data, wherein the vehicle state data at least comprise: vehicle speed, vehicle acceleration, steering wheel yaw angle;
the attention analysis unit comprises a data storage module and a signal preprocessing module, wherein the data storage module is used for storing physiological parameters of a driver and data of the driver on vehicle control acquired in the running process of the vehicle; the signal preprocessing module is used for converting data into a form that the neural network can learn better, removing noise and abnormal values, reducing errors and interference in the data, scaling the data into the same range, avoiding overlarge influence of certain characteristics on a model, improving the performance and generalization capability of the neural network, and in the embodiment, the signal preprocessing module is mainly used for constructing a characteristic vector set through physiological signals, and predicting whether the driver is concentrated or not by taking the operation data of the driver on the vehicle and the characteristic vector set as the input of the neural network;
a pre-training neural network module for assessing a driver's level of attention from the driver's physiological data and vehicle data;
and the attention level decision module is used for evaluating the attention level of the driver according to the model and making different countermeasures.
As an embodiment, the physiological signal collection of the driver is from the hands of the driver, and the signal collection electrodes are arranged on the outer side of the steering wheel.
As an alternative embodiment, a block diagram of an acquisition circuit of the physiological signal is shown in fig. 2, and the acquired signal is sequentially processed by a filter circuit, a signal amplifying circuit, a sampling circuit and an A/D conversion circuit to obtain a target signal.
Preferably, the collection of blood oxygen saturation measures oxygen saturation in blood by using a non-invasive reflective collection method, and an infrared LED and a photodiode are arranged on a steering wheel and used for receiving reflected light, wherein the infrared light is used for measuring deoxyhemoglobin in blood, and the deoxyhemoglobin absorbs light and reflects back under the irradiation of the infrared light; similarly, the infrared LED is used to measure the oxyhemoglobin in the blood, and the oxyhemoglobin absorbs light and reflects back under the irradiation of the infrared LED, and the ratio of the oxyhemoglobin to the total hemoglobin in the blood is calculated according to the infrared LED signal of the reflected infrared light, so as to obtain the value of the oxygen saturation of the blood. The middle and ring fingers have a richer blood supply than the other fingers, making the measurement of blood oxygen saturation more accurate on these fingers. In order to avoid unnecessary collection, a fingerprint identification device is arranged outside the sensor, a fingerprint or a palm print of a driver is recorded, and after the fingerprint or the palm print is identified, the sensor enters a working state, and the collection time can be set manually.
Preferably, the heart rate is acquired by using electrodes which are arranged on the steering wheel and are bilaterally symmetrical, and a finger electrocardio acquisition principle realized by the technical company of the science and technology based on a WLS12X chip can be adopted.
Another preferable heart rate acquisition scheme is to measure pulse by utilizing the difference of light transmittance caused by human tissue when blood vessels beat, and the sensor used consists of a light source and a photoelectric converter. The light source generally adopts a light emitting diode with a certain wavelength (500 nm-700 nm) which is selective to oxygen and hemoglobin in arterial blood, when a light beam passes through peripheral blood vessels of a human body, the light transmittance of the light beam is changed due to the change of arterial pulse congestion volume, at the moment, the light reflected by human tissues is received by the photoelectric converter and converted into an electric signal, and the electric signal is amplified and output, and because pulse is a signal periodically changed along with the pulsation of a heart, the arterial blood vessel volume also periodically changes, and therefore the electric signal change period of the photoelectric converter is the pulse rate.
Preferably, galvanic skin response is one of the most sensitive emotional feedback, and is derived from autonomous activation of sweat glands of skin, and is closely related to emotion, arousal degree, attention and the like, and is the most widely used type of measurement index in physiological response systems. The skin electric signal is mainly divided into two parts, including a gradation signal toni Data and a mutation signal Phacic Data. Tonic Data refers to a slowly varying signal in general, with a large baseline level difference between individuals ranging from 2-20. Mu.S. Phacic Data refers to the rapidly changing component of sympathetic nerve activity. When the organism is stimulated by the outside or the emotional state is changed, the activity of the autonomic nervous system causes the changes of vasodilation and contraction in the skin, sweat gland secretion and the like, thereby causing the change of skin resistance. After two electrodes arranged on the steering wheel are respectively contacted with the index finger and the middle finger of a driver, a tiny constant voltage is applied to the skin, the current flowing through the skin is in direct proportion to the electric conduction, and after the current and the voltage are converted, a skin electric conduction change signal is recorded. Besides the index finger and the middle finger, a pair of electrodes can be placed on the palm of the hand for collecting skin electric activity.
As an implementation manner, a fingerprint or palmprint activation mode is adopted for the electrodes and the sensors arranged on the steering wheel, so that unnecessary acquisition and inaccurate data generated by acquisition position deviation are avoided.
In this embodiment, the collected physiological signals of the driver are classified into 5 levels, which are indicated by numerals 1-5, the data collected by the vehicle data are also classified into different levels, for example, the vehicle speed and the acceleration are classified into normal and abnormal, and the steering wheel deflection angle is classified into different levels according to the size of the curve. The data is preprocessed, such as normalized and data supplementing, before being input as the input of the neural network, so that the learning of the model is facilitated, and the generalization capability of the model is improved.
In this embodiment, the data used by the pre-training neural network come from different ages, different sexes, different physical conditions and different mental states of the tested person when driving, such as driving after staying up, driving when having low emotion, driving when having high emotion, and the like, and based on individual differences of different people, the pre-training neural network can train the used data after evaluating the attention level of the driver, so as to be used as the personalized preference of the driver, and improve the accuracy of the evaluation level.
After obtaining the driver's attention level, the driver's attention analysis system makes different feedback to the driver according to the attention assessment score, e.g. when the attention level is greater than or equal to 6, no reaction is taken; when the attention level is less than 6, voice warning is made to the driver, and if no corresponding reaction is made within the voice warning time, vehicle control is taken over; when the attention level is less than 4, the vehicle is directly controlled, the existing intelligent driving function is accessed, the vehicle is autonomously moved to a safe position, the safety of a driver is ensured, and alarm information and position information are sent to emergency contacts and police.
In this embodiment, before inputting the physiological signal into the neural network, the processing needs to be performed, that is, a sample entropy value is calculated according to the physiological data of the driver, and a feature vector set is constructed based on the sample entropy value, and the feature vector set is used as an input of the pre-trained neural network, and the process of obtaining the feature vector set includes:
denoising the physiological signal, performing empirical mode decomposition after denoising, and calculating a contribution value of each inherent mode function;
and screening m inherent mode functions with highest contribution values to construct a feature vector set.
The physiological signals Z (t) collected by the collecting electrode and the sensor are filtered by a band-pass filter, interference signals are filtered, and the processing process is expressed as follows:
wherein Z (t) represents a physiological signal after filtering the interference signal by using a band-pass filter; z is a physiological signal acquired by a sensor or an electrode; f (f) s Is the sampling frequency; f (f) c1 And f c2 Are the two cut-off frequencies of the band-pass filter.
For physiological signals preprocessed by band-pass filter, the method is based onEmpirical mode decomposition is performed, where r M (t) is the residual component, imf m (t) is the mth natural mode function, and specifically comprises the following steps:
finding out all maximum value points and minimum value points of the physiological signal Z (t), and fitting by using a cubic spline interpolation function to form an upper envelope curve e of the original data max (t) and lower envelope e min (t)；
Calculating the mean value e of the upper envelope curve and the lower envelope curve avg (t)＝(e max (t)+e min (t))/2；
Extracting new data h (t) =z (t) -e avg (t) judging whether h (t) is an inherent mode function, if so, marking the h (t) as an inherent mode function, replacing the original Z (t) with Z (t) -h (t), otherwise, replacing the original Z (t) with h (t);
the above process is repeated until all the natural mode functions are found.
After finding all the natural mode functions, the natural mode functions need to be screened, and the screening process comprises the following steps:
calculating the variance contribution rate of the obtained intrinsic mode functions, and obtaining the first K intrinsic mode functions of which the accumulated variance contribution rate reaches a preset threshold value;
the formula for calculating the variance contribution rate lambda of the intrinsic mode function is
Wherein,variance, sigma, of the ith natural mode function 2 For the sum of the variances of all the natural mode functions, the variance calculation formula of the natural mode functions is +.>x i (t) is the value of the ith natural mode function at time t,/>The average value of the ith natural mode function in the whole time range is taken, and N is the sampling point number of the signal.
The process for constructing the feature vector set according to the screened natural mode function comprises the following steps:
calculating sample entropy of k natural mode functions, dividing each natural mode function into n segments S according to different time windows 1 ,...,S n ；
Assuming that each segment contains N data { u (i), 1.ltoreq.i.ltoreq.N } a set of vectors x (1), x (2) in m-dimensional space are constructed from the data for each segment, x (N-m+1), where one vector can be expressed as:
x(i)＝[u(i),u(i+1),…,u(i+m-1)],1≤i≤N-m+1；
setting a similarity margin r (r > 0), and calculating the distance d [ x (i), x (j) ] of two m-dimensional space vectors, wherein the distance between the two vectors x (i) and x (j) is defined as:
comparing the calculated distance with a similar tolerance r, and screening d [ x (i), x (j)]Calculating the ratio of the number of vectors less than the similarity margin to the total number N-m+1Expressed as:
wherein { l|d [ x (i), x (j) ] < r } means that when d [ x (i), x (j) ] < r has a value of 1.
According toDefinition of intermediate parameter B m (r) expressed as->
According to B m (r) calculating a sample entropy value samplen (m, r, N) of the sequence, expressed as:
and forming a characteristic vector by using n multiplied by k sample entropy values to form a characteristic vector set, wherein the row n represents the number of segments of each natural mode function divided, and k is the number of selected natural modes.
The process of predicting the attention of the driver according to the invention is shown in fig. 3, and mainly comprises the following steps:
the physiological signals of a driver in the driving process are collected and stored, and the collected physiological signals are preprocessed;
inputting the preprocessed physiological signals and driving data into a pre-trained neural network to obtain an evaluation level of the attention level of the driver;
judging the relation between the predicted attention level and the preset level and implementing control, for example, when the attention of the driver is less than 4 in the embodiment, starting a scheme of the stress auxiliary control vehicle; when the attention of the driver is less than 6 and not less than 4, early warning is carried out on the driver; other situations do not require manipulation of driver or vehicle controls.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. A method for determining driver attention, comprising the steps of:
electrodes and sensors are arranged on the vehicle direction control device, physiological data of a driver are collected through the electrodes and the sensors in the driving process of the driver, and the physiological data at least comprise heart rate variability, skin electric activity, blood oxygen saturation and temperature;
collecting vehicle state data of a driver in the driving process, wherein the vehicle state data at least comprises vehicle speed, steering wheel rotation angle and acceleration;
the driver sign data and the vehicle state data are input into a pre-trained neural network, which derives a score for the driver's attention.
2. A method for determining driver attention according to claim 1, characterized by calculating sample entropy values from physiological data of the driver and constructing a set of feature vectors based on the sample entropy values, the set of feature vectors being used as input to a pre-trained neural network.
3. A method for determining driver attention as in claim 2 wherein the process of obtaining a set of feature vectors comprises:
denoising the physiological signal, performing empirical mode decomposition after denoising, and calculating a contribution value of each inherent mode function;
and screening k inherent mode functions with highest contribution values to construct a feature vector set.
4. A method for determining driver attention as in claim 3 wherein filtering the physiological data using a bandpass filter to remove the interfering signal, and in particular comprising:
wherein Z (t) represents a physiological signal after filtering the interference signal by using a band-pass filter; z is a physiological signal acquired by a sensor or an electrode; f (f) s Is the sampling frequency; f (f) c1 And f c2 Are the two cut-off frequencies of the band-pass filter.
5. A method for determining driver attention according to claim 3, characterized by empirical mode decomposition of the noise-removed physiological signal, in particular comprising:
finding out all maximum value points and minimum value points of the physiological signal Z (t) after filtering the interference signal by using a band-pass filter, and fitting by using a cubic spline interpolation function to obtain an upper envelope e max (t) and lower envelope e min (t)；
Calculating the mean value e of the upper envelope curve and the lower envelope curve avg (t) extracting new data based on the obtained mean, expressed as h (t) =z (t) -e avg (t)；
Judging whether h (t) is an inherent mode function, if so, marking the h (t) as an inherent mode function, replacing the original Z (t) with Z (t) -h (t), otherwise, replacing the original Z (t) with h (t);
repeating the above process until all the inherent mode functions are found, and finally, representing the physiological signals after noise removal as:
therein, imf m (t) represents an mth natural mode function; r is (r) M And (t) is a residual component.
6. A method for determining driver attention as in claim 3 wherein constructing the feature vector set from the natural mode function comprises:
dividing each natural mode function into a plurality of segments according to different time windows, wherein each segment comprises a plurality of data;
if an inherent mode function is divided into N segments, each segment has N data, constructing a group of vectors in m-dimensional space according to the N data;
calculating the distance between m-dimensional space vectors of each data, and screening out the data smaller than the set similarity tolerance;
calculating a ratio between the number of data less than the set similarity margin and N-m+1
According to the ratioCalculating a sample entropy value, and forming a feature vector set by the flooded entropy value of each piece of data of each inherent mode function;
where N-m+1 is the total number of vectors for a set of m-dimensional space vectors constructed from N data.
7. A method for judging driver's attention as recited in claim 6, wherein the ratio is based onCalculating the sample entropy value includes:
where SampEn (m, r, N) represents a sample entropy value.
8. A method for determining driver attention according to claim 3, wherein the calculation of the contribution value of the natural mode function comprises:
wherein λ is the variance contribution of the natural mode function;variance, sigma, of the ith natural mode function 2 Sum of variances of all natural mode functions; x is x i (t) is the value of the ith natural mode function at time t,/>The average value of the ith natural mode function in the whole time range is taken, and N is the sampling point number of the signal.
9. A system for determining driver attention, characterized by implementing a method for determining driver attention as claimed in claim 1, comprising a signal acquisition circuit, a vehicle data acquisition module, an attention analysis unit, a pre-trained neural network module and an attention level decision module, wherein:
the signal acquisition circuit is used for acquiring physiological parameters of a driver in the driving process, and the physiological signals at least comprise: heart rate, blood oxygen saturation, galvanic skin activity, body temperature;
the vehicle data acquisition module is used for collecting vehicle control of a driver in the running process of the vehicle, namely vehicle state data, wherein the vehicle state data at least comprise: vehicle speed, vehicle acceleration, steering wheel yaw angle;
the attention analysis unit comprises a data storage module and a signal preprocessing module, wherein the data storage module is used for storing physiological parameters of a driver and data of the driver on vehicle control acquired in the running process of the vehicle; the signal preprocessing module is used for constructing a characteristic vector set according to physiological signals;
a pre-training neural network module for assessing a driver's level of attention from the driver's physiological data and vehicle data;
and the attention level decision module is used for evaluating the attention level of the driver according to the model and making different countermeasures.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311025026.7A CN117017298A (en) | 2023-08-15 | 2023-08-15 | Method and system for judging attention of driver |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311025026.7A CN117017298A (en) | 2023-08-15 | 2023-08-15 | Method and system for judging attention of driver |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117017298A true CN117017298A (en) | 2023-11-10 |
Family
ID=88627678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311025026.7A Pending CN117017298A (en) | 2023-08-15 | 2023-08-15 | Method and system for judging attention of driver |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117017298A (en) |
-
2023
- 2023-08-15 CN CN202311025026.7A patent/CN117017298A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276273B (en) | Driver fatigue detection method integrating facial features and image pulse heart rate estimation | |
| Murugan et al. | Detection and analysis: Driver state with electrocardiogram (ECG) | |
| US7639146B2 (en) | Blink monitor for detecting blink occurrence in a living subject | |
| Begum | Intelligent driver monitoring systems based on physiological sensor signals: A review | |
| JP7045639B2 (en) | Work suitability judgment system | |
| KR20180001367A (en) | Apparatus and Method for detecting state of driver based on biometric signals of driver | |
| CN106963348A (en) | The biosignal measurement set for seat based on running environment | |
| KR100956406B1 (en) | Driving simulation system and method for controlling tension level, and recording medium used there to | |
| Bittner et al. | Detecting of fatigue states of a car driver | |
| Shalash | Driver fatigue detection with single EEG channel using transfer learning | |
| Arefnezhad et al. | Driver drowsiness classification using data fusion of vehicle-based measures and ECG signals | |
| CN111231970A (en) | Health regulation method and device, computer equipment and storage medium | |
| CN116434959A (en) | Human body fatigue monitoring system based on mobile phone camera | |
| Rundo et al. | Innovative saliency based deep driving scene understanding system for automatic safety assessment in next-generation cars | |
| JP2022082547A (en) | Work appropriateness determination system | |
| KR20220069177A (en) | Smart control system and method of autonomous vehicle | |
| KR20180131045A (en) | Drowsiness preventing apparatus integrally formed by multi-sensored detectionalarm means and central processing means while driving a car and control methed therefor | |
| CN117017298A (en) | Method and system for judging attention of driver | |
| CN115177255A (en) | Fatigue driving monitoring and early warning method and system | |
| Bulygin et al. | Image-Based Fatigue Detection of Vehicle Driver: State-of-the-Art and Reference Model | |
| US20240021307A1 (en) | Computer-Implemented Method for Detecting a Microsleep State of Mind of a Person by Processing at Least One EEG Trace Using an Artificial Intelligence Algorithm and System Configured to Implement Such Method | |
| Montanaro et al. | A hybrid approach based on behavioural and physiological data for driver monitoring systems | |
| JPH09187429A (en) | Health information supplying apparatus | |
| US20230293072A1 (en) | Method and system for detecting drowsiness of an individual | |
| Shahrudin et al. | Development of a driver drowsiness monitoring system using electrocardiogram |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |