CN117717340A - Driver sleepiness detection method, device, equipment and medium - Google Patents
Driver sleepiness detection method, device, equipment and medium Download PDFInfo
- Publication number
- CN117717340A CN117717340A CN202410171984.3A CN202410171984A CN117717340A CN 117717340 A CN117717340 A CN 117717340A CN 202410171984 A CN202410171984 A CN 202410171984A CN 117717340 A CN117717340 A CN 117717340A
- Authority
- CN
- China
- Prior art keywords
- entropy
- eye movement
- sleepiness
- driver
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010041349 Somnolence Diseases 0.000 title claims abstract description 124
- 208000032140 Sleepiness Diseases 0.000 title claims abstract description 88
- 230000037321 sleepiness Effects 0.000 title claims abstract description 88
- 238000001514 detection method Methods 0.000 title claims abstract description 39
- 230000004424 eye movement Effects 0.000 claims abstract description 100
- 238000012546 transfer Methods 0.000 claims abstract description 35
- 238000012360 testing method Methods 0.000 claims abstract description 23
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 238000000034 method Methods 0.000 abstract description 16
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 230000036626 alertness Effects 0.000 description 2
- 238000000540 analysis of variance Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000003930 cognitive ability Effects 0.000 description 2
- 230000006998 cognitive state Effects 0.000 description 2
- 238000010219 correlation analysis Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 1
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 1
- 230000002498 deadly effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- QQWYQAQQADNEIC-RVDMUPIBSA-N tert-butyl [(z)-[cyano(phenyl)methylidene]amino] carbonate Chemical compound CC(C)(C)OC(=O)O\N=C(/C#N)C1=CC=CC=C1 QQWYQAQQADNEIC-RVDMUPIBSA-N 0.000 description 1
Abstract
The invention discloses a driver sleepiness detection method, device, equipment and medium, and relates to the technical field of sleepiness test. The method comprises the following steps: acquiring eye movement data of a driver and a region of interest; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions; searching a sub-region corresponding to each gaze point in the region of interest; acquiring probability distribution of fixation times and fixation duration corresponding to each fixation point in a corresponding subarea; calculating an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy; calling a sleepiness database, and searching a sleepiness grade corresponding to the data consistent with the entropy index in the sleepiness database; the sleepiness database at least comprises: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness grades; and the accuracy of the final detection result is ensured.
Description
Technical Field
The invention relates to the technical field of sleepiness testing, in particular to a driver sleepiness detection method, device, equipment and medium.
Background
Fatigue in driving and inattention by the driver are the main causes of deadly road traffic accidents. In order to prevent the effects of fatigue on driving, driver monitoring systems (Driver Monitoring System, DMS) have begun to spread to discover driver fatigue and drowsiness and alert the driver, potentially reducing the number of accidents. For example, ramzan et al fully analyzed current driving drowsiness detection techniques and classified them according to utilization, including Electrocardiogram (ECG), electroencephalogram (EEG) physiological markers, vehicle behavior-based and behavior parameter-based techniques. Sikander et al have proposed a review study of detecting drowsiness and fatigue of a driver. Otmani et al evaluate the degree of fatigue and sleepiness by applying deep learning techniques.
However, current research mainly detects driver's drowsiness, distraction and cognitive state by using different measurement techniques, the detection results obtained are too ideal, there may be a gap from the actual drowsiness state, and the lack of evidence proves the usefulness and reliability of their measurement and physiological features, especially the study of human driving state is relatively less when quantifying the driver's cognitive ability is poor. Therefore, we propose a driver's sense of well detection method, device, apparatus and medium to solve the above-mentioned problems.
Disclosure of Invention
In view of the foregoing drawbacks or shortcomings in the prior art, it is desirable to provide a driver's drowsiness detection method, apparatus, device and medium that improves detection accuracy and is robust.
In a first aspect, the present invention provides a driver's drowsiness detection method, comprising the steps of:
acquiring eye movement data of a driver and a region of interest; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
searching a sub-region corresponding to each gaze point in the region of interest;
acquiring probability distribution of the gazing times and the gazing duration corresponding to each gazing point in the corresponding subarea;
calculating an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
calling a sleepiness database, and searching a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database comprises at least: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
According to the technical scheme provided by the invention, the sleepiness database is built according to the following steps:
acquiring a test information set; the test information set at least comprises: testing data and corresponding fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
simulating driving according to the test data, and collecting the level of sleepiness input by a driver after each simulated driving;
and establishing a sleepiness database according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade.
According to the technical scheme provided by the invention, after the sleepiness database is built according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade, the method further comprises the following steps:
calculating a correlation index according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade in the sleepiness database to obtain an index set;
when the ratio of the number of related indexes larger than a preset threshold in the index set to the total related indexes is larger than the preset ratio, the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness level are judged to have obvious correlation.
According to the technical scheme provided by the invention, the sub-areas corresponding to the gaze points in the interested area are searched, and the method specifically comprises the following steps:
acquiring a two-dimensional coordinate system of a region of interest;
identifying, in the two-dimensional coordinate system, a region range for each of the sub-regions;
and acquiring the coordinate position of the gaze point, and searching the area range where the coordinate position is located in the two-dimensional coordinate system to obtain the sub-area corresponding to the gaze point.
According to the technical scheme provided by the invention, a two-dimensional coordinate system of the region of interest is established according to the following steps:
and taking any vertex of the region of interest as an origin, and taking two extension lines which pass through the origin and are arranged in parallel with the length directions of two adjacent sides of the region of interest as an X axis and a Y axis respectively to construct a two-dimensional coordinate system.
According to the technical scheme provided by the invention, the fixed cross entropy is calculated according to the following formula:
；
wherein,for a fixed cross entropy->For the ith point of regard +.>For the probability distribution of the number of gazing times of the ith gaze point in the corresponding subregion, +.>A probability distribution of gaze durations for the ith gaze point in the corresponding sub-region;
the eye movement statistical entropy is calculated according to the following formula:
；
wherein,statistical entropy of eye movements for gaze duration>The eye movement statistical entropy corresponding to the gazing times;
the eye movement transfer entropy is calculated according to the following formula:
；
wherein,entropy for eye movement transfer->I-1 th gaze point.
In a second aspect, the present invention provides a driver's drowsiness detection device capable of implementing the above-mentioned driver drowsiness detection method, the device comprising:
the data acquisition module is configured to acquire eye movement data of the region of interest and a driver; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
the data processing module is configured to search the subareas corresponding to the gaze points in the interested area;
the data processing module is further configured to obtain probability distribution of the gazing times and the gazing duration corresponding to each gazing point in the corresponding subarea;
the data processing module is further configured to calculate an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
the data processing module is also configured to call a sleepiness database, and search a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database comprises at least: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
In a third aspect, the present invention provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of a driver drowsiness detection method as described above when executing the computer program.
In a fourth aspect, the present invention provides a computer readable storage medium storing a computer program which when executed by a processor implements the steps of a driver drowsiness detection method as described above.
In summary, the invention discloses a specific flow of a driver's drowsiness detection method. According to the method, the region of interest and the eye movement data of the driver are obtained, and the subareas corresponding to the gaze points in the region of interest are searched; acquiring probability distribution of fixation times and fixation duration corresponding to each fixation point in a corresponding subarea; calculating an entropy index according to the probability distribution; calling a sleepiness database, and searching a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database.
According to the method, the region of interest is divided into a plurality of subareas, probability distribution of fixation times and fixation duration corresponding to each fixation point is obtained in the corresponding subareas, entropy indexes are calculated according to the probability distribution, corresponding sleepiness grades are searched in a sleepiness database, and according to the sleepiness grades, the sleepiness state of a driver is obtained, so that a guiding effect is provided for safe driving of a vehicle; and the sleepiness grade obtained by the entropy index is more accurate.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings.
Fig. 1 is a flow chart of a driver's drowsiness detection method.
FIG. 2 is a flow chart of creating a database of sleepiness.
Fig. 3 is a schematic structural diagram of the driver's drowsiness detecting device.
Fig. 4 is a schematic structural diagram of an electronic device.
Fig. 5 is a schematic diagram of a driver's perspective.
Fig. 6 is a first partitioning diagram of a region of interest.
Fig. 7 is a second partitioning diagram of a region of interest.
Fig. 8 is a third partitioned schematic diagram of a region of interest.
Reference numerals in the drawings: 100. a data acquisition module; 200. a data processing module;
500. an electronic device; 501. a CPU; 502. a ROM; 503. a RAM; 504. a bus; 505. an I/O interface; 506. an input section; 507. an output section; 508. a storage section; 509. a communication section; 510. a driver; 511. removable media.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the invention are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other. The invention will be described in detail below with reference to the drawings in connection with embodiments.
At present, the domestic and foreign automatic driving technologies are classified into 5 grades, namely L1-L5 grades. Where L4 is a high automation level and L5 is a full automation level. In the automatic driving process, high-level automatic driving (above L4 level) has been able to achieve automatic driving within its designed operating range, but it is still necessary for the driver to monitor whether the automatic driving system is operating normally. Research shows that long-time monitoring of an automatic driving system by a driver can increase mental load of the driver, and fatigue driving is easier to generate. The learner considers that the driver alertness is reduced during the use of the autopilot function, and that this reduced alertness is the nature of the human being and does not increase with training. In order to prevent the effects of fatigue on driving, driver monitoring systems (Driver Monitoring System, DMS) have begun to spread to discover driver fatigue and drowsiness and alert the driver, potentially reducing the number of accidents.
However, current research mainly detects driver's drowsiness, distraction and cognitive state by using different measurement techniques, the detection results obtained are too ideal, there may be a gap from the actual drowsiness state, and the lack of evidence proves the usefulness and reliability of their measurement and physiological features, especially the study of human driving state is relatively less when quantifying the driver's cognitive ability is poor. Accordingly, the present invention has been made to solve the above-mentioned problems.
Example 1
Referring to fig. 1, a flow chart of a driver's drowsiness detection method according to the present invention includes the following steps:
s10, acquiring an area of interest and eye movement data of a driver; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
it should be noted that, the time window for selecting the Eye movement data is, for example, five minutes, and the Eye movement data of the driver may be collected by using a driving Eye movement instrument (Smart Eye), where at least three optical cameras are connected to the driving Eye movement instrument, and the optical cameras are mounted on the vehicle console, and the focal length and the exposure of the optical cameras are adjusted, so that the driving Eye movement instrument can clearly identify the driver. In addition, before the test, the driving eye movement instrument can be calibrated by adopting a disk grid calibration plate and a 9-point type calibration mode, so that the accuracy of the acquired eye movement data is ensured.
Further, the region of interest refers to a region of interest at which the driver looks, also referred to as a swept spatial region. The region of interest may be determined by the angle of view of the driver.
The driver's viewing angle includes a pitch angle, which is an angle of rotation about the x-axis, which may be expressed in terms of viewing angle yaw, and a yaw angle, which is an angle of rotation about the y-axis, which may be expressed in terms of viewing angle pitch. Here, as shown in fig. 5, the O-point is the midpoint of the line connecting the eyes of the driver, the x-axis is the extension line passing through the O-point and perpendicular to the line connecting the eyes of the driver, and the y-axis is the extension line passing through the O-point and perpendicular to both the line connecting the eyes of the driver and the x-axis. Accordingly, view angle yaw is +.AOC, which can be represented by AP in FIG. 5; the viewing angle pitch is +.BOC, which can be represented by Ah in FIG. 5.
The division mode of the region of interest at least comprises the following steps: concentric rectangular division, concentric circle division, and equally divided rectangular division.
Fig. 6 is a diagram showing a plurality of subareas divided in a concentric rectangular manner, the subareas are sequentially arranged from the center to the outside, and accordingly, the subareas can be named by numerals in the arrangement manner. Fig. 7 is a diagram of dividing a plurality of sub-areas in a concentric circle manner, and the naming manner is the same as that of the sub-areas divided in a concentric rectangle manner, which is not repeated here. Fig. 8 is a diagram of dividing a plurality of sub-regions in an equally divided rectangular manner, and correspondingly, sub-regions are sequentially named by numerals. Wherein the dots in fig. 6 and 7 represent the centers of concentric rectangles or the centers of concentric circles; the origin in fig. 8 represents the center point of the entire region of interest. D in fig. 7 represents the radius difference between two adjacent sub-areas.
Further, fig. 8 is an area surrounded by the maximum lateral range and the maximum longitudinal range of the eye movement sweeping in a unit time. The region of interest is divided into 9 sub-regions by dividing the region surrounded by the maximum lateral and maximum longitudinal ranges of the sweep by equal division, and the equally divided sub-regions can be determined by +.aoc and +.boc, for example, +.aoc=75°, if the region is to be divided into 10 regions, each sub-region is in the range of 7.5 °, i.e. 0-7.5,7.5-15, & etc. A in FIG. 6 y A is a p Projection of AOC and BOC on plane, instead ofRepresenting the maximum lateral and longitudinal extent of the area swept by the driver, e.g. A y =0.8 (after normalization), then the width of each region is 0.08 if divided into 10 regions.
In the present embodiment, a method of dividing the sub-areas as shown in fig. 6 is employed.
S20, searching a sub-region corresponding to each gaze point in the region of interest;
specifically, searching a sub-region corresponding to each gaze point in the region of interest, specifically including the following steps:
acquiring a two-dimensional coordinate system of a region of interest;
specifically, a two-dimensional coordinate system of the region of interest is established according to the following steps:
and taking any vertex of the region of interest as an origin, and respectively taking two extension lines which pass through the origin and are arranged in parallel with the length directions of two adjacent sides of the region of interest as an X axis and a Y axis to construct a two-dimensional coordinate system.
Identifying, in a two-dimensional coordinate system, a region range for each sub-region;
the method for identifying the region range of each sub-region includes, for example, obtaining four vertex coordinates of the sub-region, and calculating the side length of the sub-region according to two adjacent coordinates, wherein the four side lengths enclose the region range of the sub-region.
And acquiring the coordinate position of the gaze point, and searching the area range where the coordinate position is located in the two-dimensional coordinate system to obtain the sub-area corresponding to the gaze point.
S30, acquiring probability distribution of fixation times and fixation duration corresponding to each fixation point in a corresponding subarea;
the probability distribution refers to the probability that the gaze point falls in a certain subarea in unit time, specifically, the probability distribution of the number of gazing points in the corresponding subarea refers to the ratio of the number of gaze points in one subarea to the total number of gaze points, and the probability distribution of the gaze duration in the corresponding subarea refers to the ratio of the gaze duration in one subarea to the total gaze duration.
S40, calculating an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
specifically, the fixed cross entropy is calculated according to the following formula:
；
wherein,for a fixed cross entropy->For the ith point of regard +.>For the probability distribution of the number of gazing times of the ith gaze point in the corresponding subregion, +.>A probability distribution of gaze durations for the ith gaze point in the corresponding sub-region;
the eye movement statistical entropy is calculated according to the following formula:
；
wherein,statistical entropy of eye movements for gaze duration>The eye movement statistical entropy corresponding to the gazing times;
the eye movement transfer entropy is calculated according to the following formula:
；
wherein,entropy transfer for eye movement，/>I-1 th gaze point.
S50, calling a sleepiness database, and searching a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database at least comprises: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
Wherein, the sleepiness database is shown in table 1;
table 1 sleepiness database
As shown in fig. 2, the sleepiness database is built according to the following steps:
s5001, acquiring a test information set; the test information set at least comprises: testing data and corresponding fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
the test data refer to known gaze points of the driver, corresponding gaze times and gaze duration, and accordingly, fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy are all calculated according to the test data.
S5002, simulating driving according to the test data, and collecting the level of sleepiness input by a driver after each simulated driving;
the simulated driving means that a driver drives the real vehicle to run according to the test data. The test site for carrying out the simulated driving is, for example, a closed high-speed loop with the total length of more than 20km, so that the single driving environment of a driver can be ensured, and the occurrence of the driver's sleepiness can be accelerated. To ensure safety, no other social vehicles are in the test site during the test, and the test is conducted during the day and night, respectively.
Training a driver before the test, wherein the training time is not shorter than two hours; after training, the driver carries out autonomous grade evaluation according to a Chinese and English translation comparison table of a Carlin sleep scale table (Karolinska Sleepiness Scale, KSS).
The English translation comparison table of the Carlin sleep scale table is shown in Table 2;
TABLE 2 Chinese and English translation control Table for Carlin sleep Scale tables
S5003, establishing a sleepiness database according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade.
Further, as shown in fig. 2, after establishing the sleepiness database according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness level, the method further comprises the following steps:
s5004, calculating a correlation index according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade in the sleepiness database to obtain an index set;
it should be noted that the correlation index may be obtained by analysis of variance calculation in the python scipy kit, for example, pearson linear correlation coefficient, a schermann rank order correlation coefficient, and a kendel rank order correlation coefficient.
The index sets are shown in table 3 and table 4, and each index set contains each relevant index corresponding to each subarea;
table 3 entropy index and KSS correlation analysis (all.)
Wherein, in Table 3Represents p<0.5，/>Represents p<0.05，/>Represents p<0.005。
Table 4 analysis of the correlation of entropy indices with KSS (Dry.)
The letters in tables 3 and 4 are explained: all. represents the level of sleepiness of all KSSs; blow. Shows the grade of sleepiness of 6 or more in KSS; l represents a pearson linear correlation coefficient (Pearson Linear Correlation Coefficient, PLCC); s represents a schermann rank order correlation number (Spearman Rank Order Correlation Coefficient, SROCC); k represents a kendel rank order correlation coefficient (Kendall Rank Order Correlation Coefficient, KROCC); p represents a hypothesis value, a hypothesis probability (p-value). AOI represents a first sub-region in the region of interest; AOI II represents a second sub-region in the region of interest; AOI III represents a third sub-region in the region of interest; here, KSS is shown in table 2.
As can be seen from the table 3,、/>、/>and +.>Can have a significant correlation on AOI I,>、、/>there can be significant correlation on AOI II. As can be seen from Table 4, ->、/>、/>And +.>Has a significant correlation on AOI, AOI II and AOI III, and, < >>、/>The most pronounced correlation is evident.
S5005, when the ratio of the number of related indexes larger than a preset threshold in the index set to the total related indexes is larger than the preset ratio, judging that the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness level have obvious correlation.
The preset ratio and the preset threshold can be set according to actual conditions.
In addition, in order to make the selected entropy index, namely the fixed cross entropy, the eye movement statistical entropy and the eye movement transfer entropy, more obvious in usefulness and reliability, the entropy index and the KSS of the invention are compared with the index of the traditional detection scheme.
Specifically, the indexes of the conventional detection scheme at least include: fatigue measurement index (PERCLOS), heart Rate RR variability index (RR Rate), and Blink frequency index (Blink Rate). The index and KSS correlation analysis of the conventional detection scheme is shown in Table 5.
TABLE 5 analysis of index and KSS correlation for traditional detection schemes
The data in Table 5 can also be obtained using analysis of variance calculations in the python scipy kit. As can be seen from table 5, PERCLOS has a significant correlation at all levels of sleepiness identified in KSS, whereas RR Rate can only have a significant correlation at blow with the corresponding levels of sleepiness, while Blink Rate has no significant correlation at all for identifying KSS status.
In conclusion, the entropy index obtained through calculation can have better effect on distinguishing the blow, has robustness compared with the index of the traditional detection scheme, and can strengthen and supplement the problem that PERCLOS is insensitive to the blow and RR Rate is insensitive to ALL; therefore, the entropy index provided by the invention can improve the accuracy of driver sleepiness detection and has good robustness.
In addition, after the accurate sleepiness level is obtained by the entropy index obtained by calculation, the method can provide corresponding guidance for a driver according to the sleepiness level so as to enhance driving safety.
Wherein, the guiding measures at least comprise the following steps:
first, remind the rest: the driver is easy to generate the problems of inattention, reaction speed reduction and the like under the higher level of sleepiness, so the driver needs to be reminded to have a rest regularly, and the driver is given an opportunity to restore energy.
Second, planning a journey: before driving, a proper driving route and time can be recommended according to the level of the driver's mind, so that long-time continuous driving or driving in a complex traffic environment is avoided.
Third, adjust driving mode: if the driver is more sensitive to the level of drowsiness, the sensitivity of other advanced driving assistance system (Advanced Driver Assistance System, ADAS) functions, such as a frontal collision warning system (Forward Collision Warning, FCW) and a lane departure warning system (Lane Departure Warning System, LDW) may be increased for avoiding hazards.
In practical application, if the vehicle detects that the level of the driver's sleepiness is high, the driver can be reminded of rest through voice reminding or vibration, and a nearby rest area or service facility is displayed, so that the driver stops for rest in time when feeling tired. In more intelligent applications, the vehicle may also incorporate navigation functionality to automatically plan a trip to avoid busy road segments or to select safer routes.
Example 2
The present invention provides a driver's drowsiness detection device, which can implement the driver's drowsiness detection method described in embodiment 1, as shown in fig. 3, and includes:
a data acquisition module 100 configured to acquire eye movement data of the driver and a region of interest; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
a data processing module 200, configured to find a sub-region corresponding to each gaze point in the region of interest;
the data processing module 200 is further configured to obtain probability distribution of fixation times and fixation duration corresponding to each fixation point in the corresponding sub-region;
the data processing module 200 is further configured to calculate an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
the data processing module 200 is further configured to call a sleepiness database, and search a sleepiness level corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database at least comprises: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
The type of the data acquisition module 100 is, for example, a driving eye tracker; the type of data processing module 200 is, for example, a programmable logic controller (Programmable Logic Controller, PLC).
Example 3
An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of a driver drowsiness detection method according to the above embodiments when the computer program is executed.
In the present embodiment, as shown in fig. 4, the electronic device 500 includes a CPU501, which can execute various appropriate actions and processes according to a program stored in a ROM502 or a program loaded from a storage section into a RAM 503. In the RAM503, various programs and data required for the system operation are also stored. The CPU501, ROM502, and RAM503 are connected to each other through a bus 504. I/O interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drives are also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, the process described above with reference to flowchart 1 may be implemented as a computer software program according to an embodiment of the invention. For example, embodiment 3 of the present invention includes a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. The above-described functions defined in the system of the present invention are performed when the computer program is executed by the CPU 501.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM (random access memory), a ROM (read-only memory), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases. The described units or modules may also be provided in a processor.
Example 4
The present invention also provides a computer-readable storage medium that may be included in the electronic device described in the above embodiments; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement a driver's drowsiness detection method as described in the above embodiment.
The above description is only illustrative of the preferred embodiments of the present invention and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the invention referred to in the present invention is not limited to the specific combinations of the technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the inventive concept. Such as the above-mentioned features and the technical features disclosed in the present invention (but not limited to) having similar functions are replaced with each other.
Claims (9)
1. A driver's drowsiness detection method, comprising the steps of:
acquiring eye movement data of a driver and a region of interest; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
searching a sub-region corresponding to each gaze point in the region of interest;
acquiring probability distribution of the gazing times and the gazing duration corresponding to each gazing point in the corresponding subarea;
calculating an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
calling a sleepiness database, and searching a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database comprises at least: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
2. The driver's drowsiness detection method according to claim 1, wherein the drowsiness database is built according to the steps of:
acquiring a test information set; the test information set at least comprises: testing data and corresponding fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
simulating driving according to the test data, and collecting the level of sleepiness input by a driver after each simulated driving;
and establishing a sleepiness database according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade.
3. The driver's drowsiness detection method according to claim 2, wherein after creating a drowsiness database according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding drowsiness level, further comprising the steps of:
calculating a correlation index according to the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness grade in the sleepiness database to obtain an index set;
when the ratio of the number of related indexes larger than a preset threshold in the index set to the total related indexes is larger than the preset ratio, the fixed cross entropy, the eye movement statistical entropy, the eye movement transfer entropy and the corresponding sleepiness level are judged to have obvious correlation.
4. The driver's drowsiness detection method according to claim 1, wherein searching for a sub-region corresponding to each gaze point in the region of interest comprises the steps of:
acquiring a two-dimensional coordinate system of a region of interest;
identifying, in the two-dimensional coordinate system, a region range for each of the sub-regions;
and acquiring the coordinate position of the gaze point, and searching the area range where the coordinate position is located in the two-dimensional coordinate system to obtain the sub-area corresponding to the gaze point.
5. The driver's drowsiness detection method according to claim 4, wherein the two-dimensional coordinate system of the region of interest is established according to the steps of:
and taking any vertex of the region of interest as an origin, and taking two extension lines which pass through the origin and are arranged in parallel with the length directions of two adjacent sides of the region of interest as an X axis and a Y axis respectively to construct a two-dimensional coordinate system.
6. The driver's drowsiness detection method according to claim 1, wherein the fixed cross entropy is calculated according to the following formula:
；
wherein,for a fixed cross entropy->For the ith point of regard +.>For the probability distribution of the number of gazing times of the ith gaze point in the corresponding subregion, +.>A probability distribution of gaze durations for the ith gaze point in the corresponding sub-region;
the eye movement statistical entropy is calculated according to the following formula:
；
wherein,statistical entropy of eye movements for gaze duration>The eye movement statistical entropy corresponding to the gazing times;
the eye movement transfer entropy is calculated according to the following formula:
；
wherein,entropy for eye movement transfer->I-1 th gaze point.
7. A driver's drowsiness detection device capable of realizing the driver's drowsiness detection method according to any one of claims 1 to 6, characterized by comprising:
the data acquisition module is configured to acquire eye movement data of the region of interest and a driver; the eye movement data includes at least: a plurality of fixation points, and fixation times and fixation duration corresponding to the fixation points; the region of interest comprises a plurality of sub-regions;
the data processing module is configured to search the subareas corresponding to the gaze points in the interested area;
the data processing module is further configured to obtain probability distribution of the gazing times and the gazing duration corresponding to each gazing point in the corresponding subarea;
the data processing module is further configured to calculate an entropy index according to the probability distribution; the entropy index at least comprises: fixed cross entropy, eye movement statistical entropy and eye movement transfer entropy;
the data processing module is also configured to call a sleepiness database, and search a sleepiness grade corresponding to the data corresponding to the entropy index in the sleepiness database; the sleepiness database comprises at least: fixed cross entropy, eye movement statistical entropy, eye movement transfer entropy and corresponding sleepiness levels.
8. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of a driver drowsiness detection method according to any one of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of a driver drowsiness detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410171984.3A CN117717340A (en) | 2024-02-07 | 2024-02-07 | Driver sleepiness detection method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410171984.3A CN117717340A (en) | 2024-02-07 | 2024-02-07 | Driver sleepiness detection method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117717340A true CN117717340A (en) | 2024-03-19 |
Family
ID=90200095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410171984.3A Pending CN117717340A (en) | 2024-02-07 | 2024-02-07 | Driver sleepiness detection method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117717340A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| USRE39539E1 (en) * | 1996-08-19 | 2007-04-03 | Torch William C | System and method for monitoring eye movement |
| JP2014123883A (en) * | 2012-12-21 | 2014-07-03 | Nikon Corp | Head-mounted type information input output apparatus, and head-mounted type information input output method |
| US20190077409A1 (en) * | 2017-07-31 | 2019-03-14 | Alcohol Countermeasure Systems (International) Inc. | Non-intrusive assessment of fatigue in drivers using eye tracking |
| CN110298303A (en) * | 2019-06-27 | 2019-10-01 | 西北工业大学 | A kind of crowd recognition method based on the long pan of memory network in short-term path learning |
| CN113548056A (en) * | 2020-04-17 | 2021-10-26 | 东北大学秦皇岛分校 | Automobile safety driving assisting system based on computer vision |
| CN113729707A (en) * | 2021-09-06 | 2021-12-03 | 桂林理工大学 | FECNN-LSTM-based emotion recognition method based on multi-mode fusion of eye movement and PPG |
| CN114586077A (en) * | 2019-10-17 | 2022-06-03 | 新唐科技日本株式会社 | Monitoring system and program |
-
2024
- 2024-02-07 CN CN202410171984.3A patent/CN117717340A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| USRE39539E1 (en) * | 1996-08-19 | 2007-04-03 | Torch William C | System and method for monitoring eye movement |
| JP2014123883A (en) * | 2012-12-21 | 2014-07-03 | Nikon Corp | Head-mounted type information input output apparatus, and head-mounted type information input output method |
| US20190077409A1 (en) * | 2017-07-31 | 2019-03-14 | Alcohol Countermeasure Systems (International) Inc. | Non-intrusive assessment of fatigue in drivers using eye tracking |
| CN110298303A (en) * | 2019-06-27 | 2019-10-01 | 西北工业大学 | A kind of crowd recognition method based on the long pan of memory network in short-term path learning |
| CN114586077A (en) * | 2019-10-17 | 2022-06-03 | 新唐科技日本株式会社 | Monitoring system and program |
| CN113548056A (en) * | 2020-04-17 | 2021-10-26 | 东北大学秦皇岛分校 | Automobile safety driving assisting system based on computer vision |
| CN113729707A (en) * | 2021-09-06 | 2021-12-03 | 桂林理工大学 | FECNN-LSTM-based emotion recognition method based on multi-mode fusion of eye movement and PPG |
Non-Patent Citations (1)
| Title |
|---|
| BROOK A. SHIFERAW: ""Stationary gaze entropy predicts lane departure events in sleepdeprived drivers"", 《SCIENTIFIC REPORTS》, 2 February 2018 (2018-02-02), pages 1 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Online prediction of driver distraction based on brain activity patterns | |
| Čegovnik et al. | An analysis of the suitability of a low-cost eye tracker for assessing the cognitive load of drivers | |
| Wang et al. | The sensitivity of different methodologies for characterizing drivers’ gaze concentration under increased cognitive demand | |
| Zeeb et al. | What determines the take-over time? An integrated model approach of driver take-over after automated driving | |
| Yin et al. | Automatic dangerous driving intensity analysis for advanced driver assistance systems from multimodal driving signals | |
| EP3915046A1 (en) | Detection of cognitive state of a driver | |
| Atiquzzaman et al. | Real-time detection of drivers’ texting and eating behavior based on vehicle dynamics | |
| Costa et al. | Detecting driver’s fatigue, distraction and activity using a non-intrusive ai-based monitoring system | |
| Kim et al. | Predicting the EEG level of a driver based on driving information | |
| Qin et al. | Influence of vehicle speed on the characteristics of driver’s eye movement at a highway tunnel entrance during day and night conditions: a pilot study | |
| He et al. | Driver fatigue evaluation model with integration of multi‐indicators based on dynamic Bayesian network | |
| Xu et al. | Left gaze bias between LHT and RHT: a recommendation strategy to mitigate human errors in left-and right-hand driving | |
| Kutila et al. | Driver cognitive distraction detection: Feature estimation and implementation | |
| CN113753059A (en) | Method for predicting takeover capacity of driver under automatic driving system | |
| van Gent et al. | Multi-level driver workload prediction using machine learning and off-the-shelf sensors | |
| Jansen et al. | Does agreement mean accuracy? Evaluating glance annotation in naturalistic driving data | |
| Xie et al. | Analysis of freeway safety influencing factors on driving workload and performance based on the gray correlation method | |
| Wang et al. | Examination of driver visual and cognitive responses to billboard elicited passive distraction using eye-fixation related potential | |
| Zhou et al. | An evaluation method for visual search stability in urban tunnel entrance and exit sections based on markov chain | |
| Hua et al. | Effect of cognitive distraction on physiological measures and driving performance in traditional and mixed traffic environments | |
| Chesney | Numerical distance effect size is a poor metric of approximate number system acuity | |
| Ross et al. | Measuring working memory load effects on electrophysiological markers of attention orienting during a simulated drive | |
| CN117717340A (en) | Driver sleepiness detection method, device, equipment and medium | |
| Tong et al. | Effects of speed, motion type, and stimulus size on dynamic visual search: A study of radar human–machine interface | |
| Wang et al. | Transitioning to multi-dimensional estimation of visual distraction and its safety effects under automated driving: a spatiotemporal and directional estimation approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |