WO2020034541A1

WO2020034541A1 - Driver drowsiness detection method, computer readable storage medium, terminal device, and apparatus

Info

Publication number: WO2020034541A1
Application number: PCT/CN2018/123790
Authority: WO
Inventors: 姜军
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2018-08-14
Filing date: 2018-12-26
Publication date: 2020-02-20
Also published as: CN109190515A

Abstract

The present application belongs to the technical field of computers, and particularly relates to a driver drowsiness detection method, a computer readable storage medium, a terminal device, and an apparatus. In the method, an image of the face of a driver is collected; a first sub-image is extracted from the image of the face of the driver, the first sub-image being an image of a region where the eyes of the driver are located; a feature vector of the first sub-image is calculated; a vector similarity degree between the feature vector of the first sub-image and a preset reference vector is calculated, the reference vector being a feature vector of an image of eyes in a fatigued state; if the vector similarity degree between the feature vector of the first sub-image and the reference vector is larger than a preset similarity degree threshold, the driver is determined to be in a drowsy state. By means of the embodiments of the present application, a reliable detection standard is provided for driver drowsiness detection which can greatly reduce the occurrence of traffic accidents.

Description

Fatigue driving detection method, readable storage medium, terminal equipment and device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on August 14, 2018, with an application number of 201810921792.4, and the invention name is "a fatigue driving detection method, a computer-readable storage medium, and a terminal device." The contents are incorporated herein by reference.

Technical field

The present application belongs to the field of computer technology, and particularly relates to a fatigue driving detection method, a computer-readable storage medium, a terminal device, and a device.

Background technique

At present, with the increasing popularity of cars, there are more and more hidden safety hazards caused by car driving. Drivers often experience traffic accidents due to their own fatigue during driving. How to grasp whether the driver is in a fatigued driving situation has become an urgent problem. But currently, the driver can only judge whether he is in a fatigue driving state by relying on the driver's own feelings or the observation of the passengers around him. The reliability is extremely poor and it is easy to cause traffic accidents.

technical problem

In view of this, embodiments of the present application provide a fatigue driving detection method, a computer-readable storage medium, a terminal device, and a device to solve the current reliability of the fatigue driving detection method, which is extremely unreliable and can easily cause traffic accidents. problem.

Technical solutions

A first aspect of the embodiments of the present application provides a fatigue driving detection method, which may include:

Collect the driver's face image;

Extracting a first sub-image from the driver's face image, where the first sub-image is an image of a region where the driver's eyes are located;

Calculating a feature vector of the first sub-image;

Calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

If the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold, it is determined that the driver is in a fatigue driving state.

A second aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the fatigue driving detection method is implemented. step.

A third aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, where the processor executes the computer When the instructions are readable, the steps of the fatigue driving detection method described above are implemented.

A fourth aspect of the embodiments of the present application provides a fatigue driving detection device, which may include a module for implementing the steps of the foregoing fatigue driving detection method.

Beneficial effect

The embodiment of the present application implements an automatic detection of a driver's fatigue driving state by means of image analysis processing, provides a reliable detection standard for the driver's fatigue driving detection, and can greatly reduce the occurrence of traffic accidents.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of an embodiment of a fatigue driving detection method according to an embodiment of the present application;

FIG. 2 is a schematic flowchart of collecting a driver's face image;

3 is a schematic flowchart of extracting a first sub-image from a driver's face image;

4 is a structural diagram of an embodiment of a fatigue driving detection device in an embodiment of the present application;

FIG. 5 is a schematic block diagram of a terminal device according to an embodiment of the present application.

Embodiments of the invention

Referring to FIG. 1, an embodiment of a fatigue driving detection method in an embodiment of the present application may include:

Step S101: Collect a driver's face image.

In order to ensure that the face image is collected instead of the background image when there is no driver, this embodiment uses a method based on skin color judgment to discriminate the collected image. Skin color is one of the salient features of a person's body. Because of the different races, the skin tone of the skin appears different colors, but after excluding the effects of brightness and visual environment on the skin tone, the skin tone is basically the same, so it can be used as a basis for identification.

Specifically, step S101 may include the steps shown in FIG. 2:

Step S1011, an image of the driving area is collected by a camera disposed in front of the driving area.

Step S1012, converting the image of the driving area from RGB space to YCbCr space to obtain a converted driving area image.

In YCbCr space, Y represents brightness, Cb and Cr represent blue component and red component, respectively, and the two are collectively called color component. YCbCr space has the characteristics of separating chrominance and brightness. In YCbCr space, the clustering characteristics of skin color are better, and it is two-dimensional independent distribution, which can better limit the distribution area of skin color and is not affected by race. . Comparing RGB space and YCbCr space, when the light intensity changes, the three color components R (red component), G (green component), and B (blue component) in RGB space will change at the same time, while the light received in YCbCr space The strong influence is relatively independent, and the color component is not greatly affected by the light intensity, so the YCbCr space is more suitable for skin tone recognition.

Specifically, the conversion from RGB space to YCbCr space can be achieved by the following formula to obtain the converted driving area image:

Y = 0.257 × R + 0.564 × G + 0.098 × B + 16;

Cb = -0.148 * R-0.291 * G + 0.439 * B + 128;

Cr = 0.439 * R-0.368 * G-0.071 * B + 128.

Step S1013: Determine, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and construct a skin color pixel set composed of each skin color pixel.

Since the two color components of the skin color in the YCbCr space are less affected by the brightness information, this solution directly considers the CbCr component of the YCbCr space and maps it into a two-dimensionally independent CbCr space. In the CbCr space, the skin color clustering is good, and the skin color pixels can be determined by using preset skin color determination conditions. In this embodiment, the skin color determination conditions that are preferably used are: 77 <Cb <127 and 133 <Cr <173, the pixels satisfying the skin color determination condition are skin color pixels.

Step S1014: Count the number of skin color pixels in the skin color pixel set, and calculate a dispersion degree of the skin color pixel set.

Specifically, the dispersion degree of the skin color pixel set can be calculated according to the following formula:

Wherein, n is the number of skin color pixels in the skin color pixel set, 1≤n≤N, N is the number of skin color pixels in the skin color pixel set, and SkinPixX _n is _nth in the skin color pixel set. The horizontal coordinate of each skin color pixel, SkinPixY _n is the vertical coordinate of the nth skin color pixel in the skin color pixel set, and DisperDeg is the dispersion of the skin color pixel set. The larger the value, the more these pixels are explained. The more scattered the points, the smaller the value, which means that the more concentrated these pixels are.

Step S1015: Determine whether a preset face determination condition is established.

For a face image, it should be a concentrated area connected by a large number of skin color pixels. Therefore, the face determination condition is that the number of skin color pixels in the skin color pixel set is greater than a preset number threshold , And the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, that is:

Among them, NumThresh is the number threshold, which can be set to 1000, 2000, 5000, or other values according to the actual situation, and DisperThresh is the dispersion threshold, which can be set to 30, 50, 100, or according to the actual situation. Other values.

If the face determination condition is not satisfied, the process returns to step S1011, and if the face determination condition is established, step S1016 and subsequent steps are performed.

Step S1016: Determine a region covered by the skin color pixel set as a face image region.

Step S1017: Extract an image in the face image area, and determine an image in the face image area as a face image of the driver.

Step S102: Extract a first sub-image from a face image of the driver.

The first sub-image is an image of a region where the eyes of the driver are located.

Specifically, step S102 may include the steps shown in FIG. 3:

Step S1021, dividing the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset.

Wherein, the pixels in the left skin color pixel subset are all satisfied:

The pixels in the right skin color pixel subset all satisfy:

ln is the sequence number of the pixels in the left skin color pixel subset, 1≤ln≤LN, LN is the total number of pixels in the left skin color pixel subset, LSkinPixX _ln is the left skin color pixel subset The abscissa of the lnth pixel, rn is the sequence number of the pixels in the right skin color pixel subset, 1≤rn≤RN, RN is the total number of pixels in the right skin color pixel subset, RSkinPixX _rn is the abscissa of the rnth pixel in the right skin color pixel subset.

In step S1022, the abscissa of the center position of the left eye and the abscissa of the center position of the right eye are calculated respectively.

Specifically, the abscissa of the center position of the left eye and the abscissa of the center position of the right eye can be calculated according to the following formulas:

Among them, LeftEyeX is the abscissa of the center position of the left eye, and RightEyeX is the abscissa of the center position of the right eye;

Step S1023: Divide the upper skin color pixel subset from the skin color pixel set.

The pixels in the upper skin color pixel subset all satisfy:

tn is the number of pixels in the upper skin color pixel subset, 1≤tn≤TN, TN is the total number of pixels in the upper skin color pixel subset, TopSkinPixY _tn is the upper skin color pixel subset The ordinate of the tnth pixel of.

Step S1024: Calculate the ordinate of the center position of the left eye and the ordinate of the center position of the right eye, respectively.

Specifically, the ordinate of the center position of the left eye and the ordinate of the center position of the right eye can be calculated according to the following formulas:

Among them, LeftEyeY is the ordinate of the center position of the left eye, and RightEyeY is the ordinate of the center position of the right eye.

Step S1025: Determine the area where the eyes are located according to the center position of the left eye, the center position of the right eye, the preset height of the eye area, and the preset width of the eye area, and use the image extracted from the area where the eyes are located as the first A child image.

In this embodiment, [X1, X2, Y1, Y2] can be used to represent a rectangular region with the abscissa from X1 to X2 and the ordinate from Y1 to Y2, then it can be determined that the area where the eyes are located is:

LeftEyeArea = [LeftEyeX1, LeftEyeX2, LeftEyeY1, LeftEyeY2]

RightEyeArea = [RightEyeX1, RightEyeX2, RightEyeY1, RightEyeY2]

among them:

Width is the preset eye area width, which can be set to 10, 15, 20, or other values according to the actual situation, and Height is the preset eye area height, which can be set to 5, 8, 10, or according to the actual situation Other values.

Step S103: Calculate a feature vector of the first sub-image.

In this embodiment, a local binary pattern (Local Binary Patterns, LBP) algorithm can be used to calculate the feature vector of the first sub-image. Specifically, a relationship between a pixel and its surrounding pixels is constructed. For each pixel in the first sub-image, the gray value of the pixel is converted into an eight-bit binary sequence by calculating the size relationship between each pixel in the neighborhood centered on the pixel and the center pixel. The pixel value of the center point is the threshold. If the pixel value of the neighborhood point is smaller than the center point, the neighborhood point is binarized to 0, otherwise it is 1. The sequence of 0 and 1 obtained by binarization is regarded as an 8-bit Binary number. Convert the binary number to decimal to get the LBP value at the center point. After the LBP value of each first sub-image pixel is calculated, the statistical histogram of the LBP feature spectrum is determined as the feature vector of the first sub-image.

This point is quantified because the relationship between the surrounding point and the point is used. After quantization, the effect of lighting on the image can be eliminated more effectively. As long as the change in illumination is not sufficient to change the size relationship between the pixel values of the two points, the LBP value will not change, which ensures the accuracy of the feature information extraction of the first sub-image.

Step S104: Calculate a vector similarity between a feature vector of the first sub-image and a preset reference vector.

The reference vector is a feature vector of the eye image in a fatigue state, and a specific calculation process thereof is similar to step S103. For details, refer to the detailed description in step S103, and details are not described herein again.

In this embodiment, the vector similarity between the feature vector of the first sub-image and the reference vector may be specifically calculated according to the following formula:

Wherein, the feature vector of the first sub-image is X = (x ₁ , x ₂ , ..., x _d , ..., x _Dim ), and the reference vector is Y = (y ₁ , y ₂ , ..., y _d , ..., y _Dim ), d is the dimension number of the vector, 1≤d≤Dim, Dim is the feature vector of the first sub-image or the number of dimensions of the reference vector, x _d Is the component of the feature vector of the first sub-image in the d-th dimension, y _d is the component of the reference vector in the d-th dimension, and SimDeg is the feature vector of the first sub-image and the reference Vector similarity between vectors.

Step S105: Determine whether the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold.

If the vector similarity between the feature vector of the first sub-image and the reference vector is greater than the similarity threshold, step S106 is performed. If the feature vector of the first sub-image and the reference vector are If the similarity of the vector is less than or equal to the similarity threshold, step S107 is performed.

The similarity threshold may be set according to actual conditions, for example, it may be set to 60%, 70%, 80%, or other values.

Step S106: Determine that the driver is in a fatigue driving state.

Step S107: Determine that the driver is in a normal state.

Since some normal blinking actions of the driver may also be misjudged as fatigued, in order to increase the accuracy, it can be set every 5 minutes, 10 minutes, 20 minutes, or other values within a set period of time. The cycle (1 second, 2 seconds, 10 seconds, or other values) is to acquire an image once and repeat the above process. Then calculate the proportion of the face image in the fatigue driving state in the total face image, if the proportion is greater than a preset proportion threshold (the proportion threshold can be set to 60%, 70%, 80% or other according to the actual situation) Value), the driver can be determined to be fatigue driving, and if the ratio is less than or equal to the ratio threshold, the driver can be determined to be in a normal state.

In summary, the embodiment of the present application realizes the automatic detection of the driver's fatigue driving state by means of image analysis processing. Considering that the driver's eye area will have significant features that can be easily identified during fatigue driving, the embodiment of the present application Using the feature vector of the eye image as the basis for detection, first collect the driver's face image, extract the image of the eye area from it, calculate the feature vector, and then use the feature vector of the eye image in the fatigued state as the reference vector for comparison If the similarity between the two is greater than a certain threshold value, this indicates that the driver's eye area has already exhibited the physiological characteristics of fatigue, and it can be determined that the driver is in a fatigued driving state. Through the embodiments of the present application, a reliable detection standard is provided for the driver to perform fatigue driving detection, which can greatly reduce the occurrence of traffic accidents.

Corresponding to the fatigue driving detection method described in the above embodiment, FIG. 4 shows a structural diagram of an embodiment of a fatigue driving detection device provided by an embodiment of the present application.

In this embodiment, a fatigue driving detection device may include:

A face image acquisition module 401, configured to collect a driver's face image;

A sub-image extraction module 402, configured to extract a first sub-image from a face image of the driver, where the first sub-image is an image of an area where the driver's eyes are located;

A feature vector calculation module 403, configured to calculate a feature vector of the first sub-image;

A vector similarity calculation module 404, configured to calculate a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

The fatigue driving state determination module 405 is configured to determine that the driver is in a fatigue driving state if the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold.

Further, the face image acquisition module includes:

An image acquisition unit, configured to acquire an image of the driving area through a camera disposed in front of the driving area;

An image space conversion unit, configured to convert an image of the driving area from an RGB space to a YCbCr space to obtain a converted driving area image;

A skin color pixel determination unit, configured to determine, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and construct a skin color pixel set composed of each skin color pixel;

Skin color pixel counting unit, configured to count the number of skin color pixels in the skin color pixel set;

A dispersion degree calculating unit, configured to calculate a dispersion degree of the skin color pixel set;

A face image region determining unit is configured to: if the number of skin color pixels in the skin color pixel set is greater than a preset number threshold, and the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, The area covered by the skin color pixel set is determined as a face image area;

A face image determination unit is configured to extract an image in the face image area, and determine an image in the face image area as a face image of the driver.

Further, the dispersion calculation unit may include:

A dispersion degree calculation subunit is configured to calculate a dispersion degree of the skin color pixel set.

Further, the sub-image extraction module may include:

A first dividing unit, configured to divide the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset;

A first calculation unit, configured to calculate the abscissa of the center position of the left eye and the abscissa of the center position of the right eye, respectively;

A second dividing unit, configured to divide an upper skin color pixel subset from the skin color pixel set;

A second calculation unit, configured to calculate the vertical coordinate of the center position of the left eye and the vertical coordinate of the center position of the right eye, respectively;

A sub-image extraction unit, configured to determine an area where an eye is located according to the center position of the left eye, the center position of the right eye, a preset height of the eye area, and a preset width of the eye area, and extract an image from the area where the eye is As the first sub-image.

Further, the vector similarity calculation module may include:

A vector similarity calculation unit is configured to calculate a vector similarity between a feature vector of the first sub-image and the reference vector.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the devices, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.

FIG. 5 shows a schematic block diagram of a terminal device according to an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.

In this embodiment, the terminal device 5 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device 5 may include a processor 50, a memory 51, and computer-readable instructions 52 stored in the memory 51 and executable on the processor 50, such as computer-readable instructions for performing the fatigue driving detection method described above. instruction. When the processor 50 executes the computer-readable instructions 52, the steps in the foregoing embodiments of the fatigue driving detection method are implemented.

When each functional unit in each embodiment of the present application is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium. Including computer-readable instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc. which can store computer-readable instructions The medium.

Claims

A fatigue driving detection method, comprising:

Collect the driver's face image;

Extracting a first sub-image from the driver's face image, where the first sub-image is an image of a region where the driver's eyes are located;

Calculating a feature vector of the first sub-image;

Calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

If the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold, it is determined that the driver is in a fatigue driving state.
The fatigue driving detection method according to claim 1, wherein the collecting a driver's face image comprises:

Collecting an image of the driving area through a camera disposed in front of the driving area;

Converting the image of the driving area from RGB space to YCbCr space to obtain a converted image of the driving area;

Determining, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and constructing a skin color pixel set composed of each skin color pixel;

Counting the number of skin color pixels in the skin color pixel set, and calculating a dispersion degree of the skin color pixel set;

If the number of skin color pixels in the skin color pixel set is greater than a preset number threshold, and the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, the area covered by the skin color pixel set is Determined as a face image area;

An image in the face image area is extracted, and an image in the face image area is determined as a face image of the driver.
The fatigue driving detection method according to claim 2, wherein the calculating the dispersion degree of the skin color pixel set comprises:

Calculate the dispersion of the skin color pixel set according to the following formula:

Wherein, n is the number of skin color pixels in the skin color pixel set, 1≤n≤N, N is the number of skin color pixels in the skin color pixel set, and SkinPixX n is nth in the skin color pixel set. The horizontal coordinate of each skin color pixel, SkinPixY n is the vertical coordinate of the nth skin color pixel in the skin color pixel set, and DisperDeg is the dispersion degree of the skin color pixel set.
The fatigue driving detection method according to claim 3, wherein the extracting a first sub-image from a face image of the driver comprises:

Dividing the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset, wherein the pixels in the left skin color pixel subset all satisfy:

The pixels in the right skin color pixel subset all satisfy:

ln is the sequence number of the pixels in the left skin color pixel subset, 1≤ln≤LN, LN is the total number of pixels in the left skin color pixel subset, LSkinPixX ln is the left skin color pixel subset The abscissa of the lnth pixel, rn is the sequence number of the pixels in the right skin color pixel subset, 1≤rn≤RN, RN is the total number of pixels in the right skin color pixel subset, RSkinPixX rn is the abscissa of the rnth pixel in the right skin color pixel subset;

Calculate the abscissa of the center position of the left eye and the abscissa of the center position of the right eye according to the following formulas:

Among them, LeftEyeX is the abscissa of the center position of the left eye, and RightEyeX is the abscissa of the center position of the right eye;

An upper skin color pixel subset is divided from the skin color pixel set, and the pixels in the upper skin color pixel subset all satisfy:

tn is the number of pixels in the upper skin color pixel subset, 1≤tn≤TN, TN is the total number of pixels in the upper skin color pixel subset, TopSkinPixY tn is the upper skin color pixel subset The ordinate of the tnth pixel point;

Calculate the ordinate of the center position of the left eye and the ordinate of the center position of the right eye according to the following formulas:

Among them, LeftEyeY is the ordinate of the center position of the left eye, and RightEyeY is the ordinate of the center position of the right eye;

Determine the area of the eye according to the center position of the left eye, the center position of the right eye, a preset height of the eye area, and a preset width of the eye area, and use the image extracted from the area where the eye is located as the first sub-image .
The fatigue driving detection method according to any one of claims 1 to 4, wherein calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector includes:

The vector similarity between the feature vector of the first sub-image and the reference vector is calculated according to the following formula:

Wherein, the feature vector of the first sub-image is X = (x 1 , x 2 , ..., x d , ..., x Dim ), and the reference vector is Y = (y 1 , y 2 , ..., y d , ..., y Dim ), d is the dimension number of the vector, 1≤d≤Dim, Dim is the feature vector of the first sub-image or the number of dimensions of the reference vector, x d Is the component of the feature vector of the first sub-image in the d-th dimension, y d is the component of the reference vector in the d-th dimension, and SimDeg is the feature vector of the first sub-image and the reference Vector similarity between vectors.
A computer-readable storage medium storing computer-readable instructions, wherein the computer-readable instructions implement the following steps when executed by a processor:

Collect the driver's face image;

Extracting a first sub-image from the driver's face image, where the first sub-image is an image of a region where the driver's eyes are located;

Calculating a feature vector of the first sub-image;

Calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

If the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold, it is determined that the driver is in a fatigue driving state.
The computer-readable storage medium of claim 6, wherein the collecting a driver's face image comprises:

Collecting an image of the driving area through a camera disposed in front of the driving area;

Converting the image of the driving area from RGB space to YCbCr space to obtain a converted image of the driving area;

Determining, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and constructing a skin color pixel set composed of each skin color pixel;

Counting the number of skin color pixels in the skin color pixel set, and calculating a dispersion degree of the skin color pixel set;

If the number of skin color pixels in the skin color pixel set is greater than a preset number threshold, and the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, the area covered by the skin color pixel set is Determined as a face image area;

An image in the face image area is extracted, and an image in the face image area is determined as a face image of the driver.
The computer-readable storage medium of claim 7, wherein the calculating the dispersion degree of the skin color pixel set comprises:

Calculate the dispersion of the skin color pixel set according to the following formula:

Wherein, n is the number of skin color pixels in the skin color pixel set, 1≤n≤N, N is the number of skin color pixels in the skin color pixel set, and SkinPixX n is nth in the skin color pixel set. The horizontal coordinate of each skin color pixel, SkinPixY n is the vertical coordinate of the nth skin color pixel in the skin color pixel set, and DisperDeg is the dispersion degree of the skin color pixel set.
The computer-readable storage medium according to claim 8, wherein the extracting a first sub-image from a face image of the driver comprises:

Dividing the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset;

Calculate the abscissa of the center position of the left eye and the abscissa of the center position of the right eye, respectively;

Dividing an upper skin color pixel subset from the skin color pixel set;

Calculate the ordinate of the center position of the left eye and the ordinate of the center position of the right eye, respectively;

Determine the area of the eye according to the center position of the left eye, the center position of the right eye, a preset height of the eye area, and a preset width of the eye area, and use the image extracted from the area where the eye is located as the first sub-image .
The computer-readable storage medium according to any one of claims 6 to 9, wherein calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector comprises:

The vector similarity between the feature vector of the first sub-image and the reference vector is calculated according to the following formula:

Wherein, the feature vector of the first sub-image is X = (x 1 , x 2 , ..., x d , ..., x Dim ), and the reference vector is Y = (y 1 , y 2 , ..., y d , ..., y Dim ), d is the dimension number of the vector, 1≤d≤Dim, Dim is the feature vector of the first sub-image or the number of dimensions of the reference vector, x d Is the component of the feature vector of the first sub-image in the d-th dimension, y d is the component of the reference vector in the d-th dimension, and SimDeg is the feature vector of the first sub-image and the reference Vector similarity between vectors.
A terminal device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, and is characterized in that the processor implements the computer-readable instructions as follows: step:

Collect the driver's face image;

Extracting a first sub-image from the driver's face image, where the first sub-image is an image of a region where the driver's eyes are located;

Calculating a feature vector of the first sub-image;

Calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

If the vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold, it is determined that the driver is in a fatigue driving state.
The terminal device according to claim 11, wherein the collecting a driver's face image comprises:

Collecting an image of the driving area through a camera disposed in front of the driving area;

Converting the image of the driving area from RGB space to YCbCr space to obtain a converted image of the driving area;

Determining, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and constructing a skin color pixel set composed of each skin color pixel;

Counting the number of skin color pixels in the skin color pixel set, and calculating a dispersion degree of the skin color pixel set;

If the number of skin color pixels in the skin color pixel set is greater than a preset number threshold, and the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, the area covered by the skin color pixel set is Determined as a face image area;

An image in the face image area is extracted, and an image in the face image area is determined as a face image of the driver.
The terminal device according to claim 12, wherein the calculating the dispersion degree of the skin color pixel set comprises:

Calculate the dispersion of the skin color pixel set according to the following formula:

Wherein, n is the number of skin color pixels in the skin color pixel set, 1≤n≤N, N is the number of skin color pixels in the skin color pixel set, and SkinPixX n is nth in the skin color pixel set. The horizontal coordinate of each skin color pixel, SkinPixY n is the vertical coordinate of the nth skin color pixel in the skin color pixel set, and DisperDeg is the dispersion degree of the skin color pixel set.
The terminal device according to claim 13, wherein the extracting a first sub-image from a face image of the driver comprises:

Dividing the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset;

Calculate the abscissa of the center position of the left eye and the abscissa of the center position of the right eye, respectively;

Dividing an upper skin color pixel subset from the skin color pixel set;

Calculate the ordinate of the center position of the left eye and the ordinate of the center position of the right eye, respectively;

Determine the area of the eye according to the center position of the left eye, the center position of the right eye, a preset height of the eye area, and a preset width of the eye area, and use the image extracted from the area where the eye is located as the first sub-image .
The terminal device according to any one of claims 11 to 14, wherein calculating a vector similarity between a feature vector of the first sub-image and a preset reference vector includes:

The vector similarity between the feature vector of the first sub-image and the reference vector is calculated according to the following formula:

Wherein, the feature vector of the first sub-image is X = (x 1 , x 2 , ..., x d , ..., x Dim ), and the reference vector is Y = (y 1 , y 2 , ..., y d , ..., y Dim ), d is the dimension number of the vector, 1≤d≤Dim, Dim is the feature vector of the first sub-image or the number of dimensions of the reference vector, x d Is the component of the feature vector of the first sub-image in the d-th dimension, y d is the component of the reference vector in the d-th dimension, and SimDeg is the feature vector of the first sub-image and the reference Vector similarity between vectors.
A fatigue driving detection device, comprising:

A face image acquisition module for collecting a driver's face image;

A sub-image extraction module, configured to extract a first sub-image from a face image of the driver, where the first sub-image is an image of a region where the driver's eyes are located;

A feature vector calculation module, configured to calculate a feature vector of the first sub-image;

A vector similarity calculation module, configured to calculate a vector similarity between a feature vector of the first sub-image and a preset reference vector, where the reference vector is a feature vector of an eye image in a fatigue state;

The fatigue driving state determination module is configured to determine that the driver is in a fatigue driving state if a vector similarity between the feature vector of the first sub-image and the reference vector is greater than a preset similarity threshold.
The fatigue driving detection device according to claim 16, wherein the face image acquisition module comprises:

An image acquisition unit, configured to acquire an image of the driving area through a camera disposed in front of the driving area;

An image space conversion unit, configured to convert an image of the driving area from an RGB space to a YCbCr space to obtain a converted driving area image;

A skin color pixel determination unit, configured to determine, in the converted driving area image, pixels that meet preset skin color determination conditions as skin color pixels, and construct a skin color pixel set composed of each skin color pixel;

Skin color pixel counting unit, configured to count the number of skin color pixels in the skin color pixel set;

A dispersion degree calculating unit, configured to calculate a dispersion degree of the skin color pixel set;

A face image region determining unit is configured to: if the number of skin color pixels in the skin color pixel set is greater than a preset number threshold, and the dispersion degree of the skin color pixel set is less than a preset dispersion degree threshold, The area covered by the skin color pixel set is determined as a face image area;

A face image determination unit is configured to extract an image in the face image area, and determine an image in the face image area as a face image of the driver.
The fatigue driving detection device according to claim 17, wherein the dispersion calculation unit includes:

The dispersion degree calculation subunit is configured to calculate the dispersion degree of the skin color pixel set according to the following formula:

Wherein, n is the number of skin color pixels in the skin color pixel set, 1≤n≤N, N is the number of skin color pixels in the skin color pixel set, and SkinPixX n is nth in the skin color pixel set. The horizontal coordinate of each skin color pixel, SkinPixY n is the vertical coordinate of the nth skin color pixel in the skin color pixel set, and DisperDeg is the dispersion degree of the skin color pixel set.
The fatigue driving detection device according to claim 18, wherein the sub-image extraction module comprises:

A first dividing unit, configured to divide the skin color pixel set into a left skin color pixel subset and a right skin color pixel subset;

A first calculation unit, configured to calculate the abscissa of the center position of the left eye and the abscissa of the center position of the right eye, respectively;

A second dividing unit, configured to divide an upper skin color pixel subset from the skin color pixel set;

A second calculation unit, configured to calculate the vertical coordinate of the center position of the left eye and the vertical coordinate of the center position of the right eye, respectively;

A sub-image extraction unit, configured to determine an area where an eye is located according to the center position of the left eye, the center position of the right eye, a preset height of the eye area, and a preset width of the eye area, and extract an image from the area where the eye is As the first sub-image.
The fatigue driving detection device according to any one of claims 16 to 19, wherein the vector similarity calculation module includes:

A vector similarity calculation unit is configured to calculate a vector similarity between a feature vector of the first sub-image and the reference vector according to the following formula:

Wherein, the feature vector of the first sub-image is X = (x 1 , x 2 , ..., x d , ..., x Dim ), and the reference vector is Y = (y 1 , y 2 , ..., y d , ..., y Dim ), d is the dimension number of the vector, 1≤d≤Dim, Dim is the feature vector of the first sub-image or the number of dimensions of the reference vector, x d Is the component of the feature vector of the first sub-image in the d-th dimension, y d is the component of the reference vector in the d-th dimension, and SimDeg is the feature vector of the first sub-image and the reference Vector similarity between vectors.