CN110807351A - Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition - Google Patents
Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition Download PDFInfo
- Publication number
- CN110807351A CN110807351A CN201910801123.8A CN201910801123A CN110807351A CN 110807351 A CN110807351 A CN 110807351A CN 201910801123 A CN201910801123 A CN 201910801123A CN 110807351 A CN110807351 A CN 110807351A
- Authority
- CN
- China
- Prior art keywords
- image
- driver
- face
- fatigue
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Ophthalmology & Optometry (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and particularly relates to an intelligent vehicle-mounted fatigue detection system based on face recognition, which comprises: the image acquisition unit is arranged in the automobile and used for acquiring image information of a driver during driving in real time; the face detection unit is used for determining the area of the face in the image according to the image information acquired by the image acquisition unit; the fatigue detection unit is used for carrying out fatigue detection on the face area according to the determined face area in the image and judging whether the driver is in fatigue driving; and the early warning unit is arranged in the automobile, and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, the early warning unit sends out early warning information, and if the driver is not in fatigue driving, the driver does not give a prompt. The method has the advantages of accurate monitoring, high recognition rate and high efficiency.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an intelligent vehicle-mounted fatigue detection system, method and device based on face recognition.
Background
When the vehicle is driven continuously after fatigue, the vehicle can feel sleepy, weak limbs, unconsciousness, reduced judgment capability, even absentmindedness or instant memory loss, delayed or early action, improper operation pause or correction time and other unsafe factors, and road traffic accidents are easy to happen. Thus, driving the vehicle after fatigue is strictly prohibited.
The factors that cause fatigue driving are manifold. Fatigue of drivers is mainly fatigue of nerves and sense organs, and fatigue of limbs caused by poor blood circulation due to long-term fixed posture. When a driver sits on a fixed seat for a long time, the action is limited to a certain degree, the attention is highly concentrated, the driver is busy judging stimulation information outside the vehicle, and the mental state is highly tense, so that the driving fatigue phenomena of blurred eyes, waist soreness and backache, slow response, inflexible driving and the like occur. The order of fatigue development is: it is also a considerable aspect that the eyes, neck, shoulders, waist, mainly the fatigue of the eyes and body and also the brain.
The judgment ability is reduced, the response is slow and the operation error is increased when the driver is tired. When a driver is in slight fatigue, untimely and inaccurate gear shifting can occur; when the driver is in moderate fatigue, the operation action is dull, and sometimes even the driver forgets the operation; when a driver is severely tired, the driver is often conscious of operation or sleeps for a short time, and the driver loses the control capability of the vehicle in severe cases. When a driver is tired, the phenomena of blurred vision, soreness and pain of the waist and back, stiff movements, fullness in hands and feet, or lack of concentration of energy, slow reaction, poor thinking, distraction, anxiety, impatience and the like can occur. If the vehicle is still being driven barely, a traffic accident may occur.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide an intelligent vehicle-mounted fatigue detection system, method and device based on face recognition, which have the advantages of accurate monitoring, high recognition rate and high efficiency.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
an intelligent vehicle-mounted fatigue detection system based on face recognition comprises:
the image acquisition unit is arranged in the automobile and used for acquiring image information of a driver during driving in real time;
the face detection unit is used for determining the area of the face in the image according to the image information acquired by the image acquisition unit;
the fatigue detection unit is used for carrying out fatigue detection on the face area according to the determined face area in the image and judging whether the driver is in fatigue driving;
and the early warning unit is arranged in the automobile, and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, the early warning unit sends out early warning information, and if the driver is not in fatigue driving, the driver does not give a prompt.
Further, the face detection unit includes: the device comprises a color balance subunit, a human eye positioning subunit and a detection unit; the color balance subunit and the human eye positioning subunit are respectively connected with the detection unit through signals.
Further, the color balance subunit is configured to perform color balance processing on the acquired image information, where the color balance processing method includes: and (3) imaging pixels: p is arranged in descending order of brightness to form an image pixel set: { Pl1, Pl2 … … Pln }; wherein: n is the number of pixels, li is the corresponding luminance value of the pixel, where l1>l2>……>n; selecting the first 5% of elements in the image pixel set
As a reference white color; calculating the average value, meanR, menaG and meanB, of each of the reference white R, G, B components, and calculating the adjustment coefficient of R, G, B components such that aR is meanR/meanI, aG is meanG/meanI, and aB is meanB/meanI, where meanI is the average gray value of the image; modulation R, G, B component: r ' ═ R, G ' ═ G, B ' ═ B,; the adjustment for component values of R ', G ', B ' greater than 255 is 255.
Further, the human eye positioning subunit is configured to process the acquired image information, and position a human eye portion in the image, where the method for positioning the human eye portion in the image performs the following steps: firstly, searching the position of an iris in a human face region, and detecting the iris by using Hough transformation; on the basis of obtaining the positions of two pupils, the pupil of two eyes is taken as the center to demarcate the eye area, then the eye area is processed, and the inner and outer eye angular points are positioned, which comprises the following steps: with a circular area mask with a pixel radius of r, the degree of coincidence of the pixel value of all points in the area of each point in the image with the value of the current point is examined using the following formula:
wherein: t isAn inter-pixel difference threshold value of 27; g is a geometric threshold;
on the basis, the accurate positions of the inner and outer eye corner points of the eyes can be obtained by extracting the corner points of the black edge curve in the image.
An intelligent vehicle-mounted fatigue detection method based on face recognition comprises the following steps:
step 1: acquiring image information of a driver during driving in real time;
step 2: determining the area of the face in the image according to the image information acquired by the image acquisition unit;
and step 3: according to the determined face area in the image, carrying out fatigue detection on the face area, and judging whether a driver is in fatigue driving;
and 4, step 4: and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, early warning information is sent out, and if the driver is not in fatigue driving, no prompt is given.
Further, in step 2, according to the determined face region in the image, fatigue detection is performed on the face region, and the method for determining whether the driver is in fatigue driving executes the following steps:
step 2.1: according to the determined face area, determining the width of the left and right canthi of the human eyes in the face area: w ═ LR-LL; determining the height of the upper and lower eyelids: h is CU-CL; the degree of opening and closing of the human eye is determined using the following formula: r is H/W;
step 2.2: counting the blinking times of the driver in unit time so as to obtain the blinking frequency of the driver;
step 2.3: and judging whether the driver is in a fatigue driving state or not according to the opening and closing degree and the blinking frequency of the human eyes.
Further, the method further comprises: and establishing a deep learning model, and automatically detecting the acquired image information according to the deep learning model.
Further, the method for establishing the deep learning model executes the following steps:
step S1: inputting a face image training set;
step S2: training the input face image to form a forward neural network;
step S3: calculating a training error of the forward neural network; because the output variable E of the training is 'the face area in the image information', but a predicted value generated after the model training is O, the obtained error function is as follows:
where m represents the number of samples input into the modeling this time and i represents the ith variable.
Step S3.4.6: backpropagating update weights w
In order to reduce the error and improve the accuracy of model prediction, the neural network reversely transmits data from the output layer to the input layer, and the value of the weight w is readjusted until the model error reaches the minimum, and then the training is stopped, so that the model creation is completed.
An intelligent vehicle-mounted fatigue detection device based on face recognition, wherein the device is a non-transitory computer-readable storage medium which stores computing instructions and comprises: acquiring an image information code segment of a driver in real time; a code segment for determining a region of a face in the image according to the image information acquired by the image acquisition unit; a code segment for performing fatigue detection on the face region according to the determined face region in the image and judging whether the driver is in fatigue driving; and a code segment for sending out early warning information if the driver is in fatigue driving according to the judgment result of the fatigue detection unit, and not giving a prompt if the driver is not in fatigue driving.
The intelligent vehicle-mounted fatigue detection system, method and device based on face recognition have the following beneficial effects:
1. the identification is accurate: the invention preprocesses the image information through color balance, and can effectively avoid the problem of inaccurate detection result caused by light and skin color. Meanwhile, the fatigue monitoring time is controlled within 50ms, a good real-time effect is achieved, and the fatigue driving detection accuracy rate reaches 97.6 percent
2. The intelligent degree is high: according to the invention, the neural network is constructed to intelligently identify the acquired image information directly in a deep learning mode, and the identification accuracy can be improved in the image identification process, so that the monitoring accuracy is improved.
Drawings
Fig. 1 is a schematic system structure diagram of an intelligent vehicle-mounted fatigue detection system based on face recognition according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of an embodiment of a method for intelligently detecting vehicle-mounted fatigue based on face recognition according to the present invention;
fig. 3 is a schematic diagram of an experimental effect of the intelligent vehicle-mounted fatigue detection system, method and device based on face recognition according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of an error rate experiment for identifying iris areas using the detection algorithm of the present invention according to an embodiment of the present invention.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
Example 1
An intelligent vehicle-mounted fatigue detection system based on face recognition comprises:
the image acquisition unit is arranged in the automobile and used for acquiring image information of a driver during driving in real time;
the face detection unit is used for determining the area of the face in the image according to the image information acquired by the image acquisition unit;
the fatigue detection unit is used for carrying out fatigue detection on the face area according to the determined face area in the image and judging whether the driver is in fatigue driving;
and the early warning unit is arranged in the automobile, and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, the early warning unit sends out early warning information, and if the driver is not in fatigue driving, the driver does not give a prompt.
Example 2
On the basis of the above embodiment, the face detection unit includes: the device comprises a color balance subunit, a human eye positioning subunit and a detection unit; the color balance subunit and the human eye positioning subunit are respectively connected with the detection unit through signals.
Example 3
On the basis of the above embodiment, the color balance subunit is configured to perform color balance processing on the acquired image information, and the method of color balance processing includes: and (3) imaging pixels: p is arranged in descending order of brightness to form an image pixel set: { Pl1, Pl2 … … Pln }; wherein: n is the number of pixels, li is the corresponding luminance value of the pixel, where l1>l2>……>n; selecting the first 5% of elements in the image pixel set
As a reference white color; calculating the average value, meanR, menaG and meanB, of each of the reference white R, G, B components, and calculating the adjustment coefficient of R, G, B components such that aR is meanR/meanI, aG is meanG/meanI, and aB is meanB/meanI, where meanI is the average gray value of the image; modulation R, G, B component: r ' ═ R, G ' ═ G, B ' ═ B,; the adjustment for component values of R ', G ', B ' greater than 255 is 255.
Specifically, the face detection is to process and analyze an input image, determine whether a face exists in the input image, find a face position if the face exists in the input image, and separate the face from a background. In recent years, a large number of face detection methods have appeared, and the methods are mainly divided into two methods, namely feature-based methods and image-based methods. The former uses certain characteristics such as skin color, facial form, nose, mouth and the like as a minimum processing unit; the latter takes the pixels in the image as a processing unit, takes face detection as a typical pattern recognition problem, and uses a training algorithm to distinguish the face from the non-face area. The face detection based on the skin color is easily influenced by abnormal illumination, so the color balance is firstly carried out on the image; in the experiment, a plurality of noise points still exist in the image after the skin color is extracted, and the detection accuracy is directly influenced, so that the morphological filter is adopted for denoising the image after the skin color is extracted, and a good effect is obtained. On the accurate positioning of human eyes, several operators for edge extraction are compared, Susan operators are found to be more suitable for extracting human eye region characteristics, and the operators are adopted to position eye corner points.
Through the color equalization processing of the human face, the effect of subsequent image recognition can be obviously improved, and the final recognition accuracy is improved.
Example 4
On the basis of the above embodiment, the human eye positioning subunit is configured to process the acquired image information, and position the human eye portion in the image, where the method for positioning the human eye portion in the image performs the following steps: firstly, searching the position of an iris in a human face region, and detecting the iris by using Hough transformation; on the basis of obtaining the positions of two pupils, the pupil of two eyes is taken as the center to demarcate the eye area, then the eye area is processed, and the inner and outer eye angular points are positioned, which comprises the following steps: with a circular area mask with a pixel radius of r, the degree of coincidence of the pixel value of all points in the area of each point in the image with the value of the current point is examined using the following formula:wherein: t is the inter-pixel difference threshold, which is 27; g is a geometric threshold;on the basis, the accurate positions of the inner and outer eye corner points of the eyes can be obtained by extracting the corner points of the black edge curve in the image.
Specifically, the skin color is one of the most important features of the human face, does not depend on the detail features of the face, adapts to the change conditions of rotation, expression and the like, has relative stability and is distinguished from the color of most background objects. The prior research results prove the consistency of skin color distribution in biology and physics, and point out that although the skin color of a person is different due to different races and presents different colors, the skin tone is basically consistent after the influence of brightness and the like on the skin color is eliminated, and favorable evidence is provided for the possibility of skin detection by using skin color information.
Example 5
An intelligent vehicle-mounted fatigue detection method based on face recognition comprises the following steps:
step 1: acquiring image information of a driver during driving in real time;
step 2: determining the area of the face in the image according to the image information acquired by the image acquisition unit;
and step 3: according to the determined face area in the image, carrying out fatigue detection on the face area, and judging whether a driver is in fatigue driving;
and 4, step 4: and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, early warning information is sent out, and if the driver is not in fatigue driving, no prompt is given.
Example 6
On the basis of the previous embodiment, in step 2, according to the determined face region in the image, fatigue detection is performed on the face region, and the method for judging whether the driver is in fatigue driving performs the following steps:
step 2.1: according to the determined face area, determining the width of the left and right canthi of the human eyes in the face area: w ═ LR-LL; determining the height of the upper and lower eyelids: h is CU-CL; the degree of opening and closing of the human eye is determined using the following formula: r is H/W;
step 2.2: counting the blinking times of the driver in unit time so as to obtain the blinking frequency of the driver;
step 2.3: and judging whether the driver is in a fatigue driving state or not according to the opening and closing degree and the blinking frequency of the human eyes.
Specifically, the selection of the color space directly affects the result of skin color detection, and common color spaces for skin color detection include HIS, YIQ, YUV, YCbCr, and the like. The YCbCr color space is used by the Xujun and the like to calculate the membership degree of each pixel point belonging to the skin color, and a good face detection effect is obtained in a complex background. However, in the YCbCr color space, the skin color cluster is spindle-shaped, the part with larger or smaller Y value is reduced, the skin color cluster area is also reduced, simple elimination of the Y component is not feasible, and three components must be considered, thereby increasing the workload.
Example 7
On the basis of the above embodiment, the method further includes: and establishing a deep learning model, and automatically detecting the acquired image information according to the deep learning model.
The deep learning model is used for constructing a deep neural network by adopting a conventional method in the prior art, training is carried out by using sample data, and the characteristics of an image are automatically extracted to achieve a recognition result. The convolutional neural network is a key network applied to image recognition by deep learning, and features of an image can be extracted layer by constructing the convolutional neural network. The method for constructing and training the network is the key of the deep neural network recognition effect, the excellent network design can achieve a better training result by using fewer parameters, the special network components can accelerate the training process, and the proper training method can fully exert the network capability. Deep learning generally adopts a model splicing method, integrates a plurality of networks, adjusts parameters of network training and obtains better results. On the basis of image target identification, a post-processing method is designed, various position relations and size relations of two targets are calculated, a category histogram is drawn, the method can be used for analyzing the logical relation of the targets, and the rationality of target identification can be calculated for modifying the identification result. A graphical operation interface for deep learning image recognition is designed and realized, the system supports selection of parameters of a training network, real-time feedback of training progress is achieved, and a special page classification image checking test result is provided. And finally, providing an image recognition service for the user by taking the image recognition program as a network service, and performing recognition analysis on the street image by using the service.
Example 8
On the basis of the previous embodiment, the method for establishing the deep learning model executes the following steps:
step S1: inputting a face image training set;
step S2: training the input face image to form a forward neural network;
step S3: calculating a training error of the forward neural network; because the output variable E of the training is 'the face area in the image information', but a predicted value generated after the model training is O, the obtained error function is as follows:
where m represents the number of samples input into the modeling this time and i represents the ith variable.
Step S4: backpropagating update weights w
In order to reduce the error and improve the accuracy of model prediction, the neural network reversely transmits data from the output layer to the input layer, and the value of the weight w is readjusted until the model error reaches the minimum, and then the training is stopped, so that the model creation is completed.
In particular, established using the method
Example 9
An intelligent vehicle-mounted fatigue detection device based on face recognition, wherein the device is a non-transitory computer-readable storage medium which stores computing instructions and comprises: acquiring an image information code segment of a driver in real time; a code segment for determining a region of a face in the image according to the image information acquired by the image acquisition unit; a code segment for performing fatigue detection on the face region according to the determined face region in the image and judging whether the driver is in fatigue driving; and a code segment for sending out early warning information if the driver is in fatigue driving according to the judgment result of the fatigue detection unit, and not giving a prompt if the driver is not in fatigue driving.
The above description is only an embodiment of the present invention, but not intended to limit the scope of the present invention, and any structural changes made according to the present invention should be considered as being limited within the scope of the present invention without departing from the spirit of the present invention. .
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the system provided in the foregoing embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (9)
1. The utility model provides an intelligent on-vehicle fatigue detection system based on face identification which characterized in that, the system includes:
the image acquisition unit is arranged in the automobile and used for acquiring image information of a driver during driving in real time;
the face detection unit is used for determining the area of the face in the image according to the image information acquired by the image acquisition unit;
the fatigue detection unit is used for carrying out fatigue detection on the face area according to the determined face area in the image and judging whether the driver is in fatigue driving;
and the early warning unit is arranged in the automobile, and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, the early warning unit sends out early warning information, and if the driver is not in fatigue driving, the driver does not give a prompt.
2. The system of claim 1, wherein the face detection unit comprises: the device comprises a color balance subunit, a human eye positioning subunit and a detection unit; the color balance subunit and the human eye positioning subunit are respectively connected with the detection unit through signals.
3. The system of claim 2, wherein the color balancing subunit is configured to perform color balancing processing on the captured image information, the color balancing processing method comprising: and (3) imaging pixels: p is arranged in descending order of brightness to form an image pixel set: { Pl1, Pl2 … … Pln }; wherein: n is the number of pixels, li is the corresponding luminance value of the pixel, where l1>l2>……>n; selecting the first 5% of elements in the image pixel set
As a reference white color; calculating the mean value, mean R, menaG and mean B of the reference white R, G, B components, and calculating the adjustment coefficient of R, G, B components such that aR is mean R/mean I, aG is mean G/mean I, and aB is mean B/mean I, where mean I is the average gray value of the image; modulation R, G, B component: r ' ═ R, G ' ═ G, B ' ═ B,; the adjustment for component values of R ', G ', B ' greater than 255 is 255.
4. The system of claim 3, wherein the eye positioning subunit is configured to process the captured image information to position the portion of the eye in the image, and wherein the method for positioning the portion of the eye in the image performs the steps of: firstly, searching the position of an iris in a human face region, and detecting the iris by using Hough transformation; on the basis of obtaining the positions of two pupils, the pupil of two eyes is taken as the center to demarcate the eye area, then the eye area is processed, and the inner and outer eye angular points are positioned, which comprises the following steps: using a circular area mask with a pixel radius of r, the pixel values of all points in the area of each point in the image and the value of the current point are considered by the following formulaDegree of agreement:
wherein: t is the inter-pixel difference threshold, which is 27; g is a geometric threshold;
on the basis, the accurate positions of the inner and outer eye corner points of the eyes can be obtained by extracting the corner points of the black edge curve in the image.
5. An intelligent vehicle-mounted fatigue detection method based on face recognition based on the system of any one of claims 1 to 4, characterized in that the method executes the following steps:
step 1: acquiring image information of a driver during driving in real time;
step 2: determining the area of the face in the image according to the image information acquired by the image acquisition unit;
and step 3: according to the determined face area in the image, carrying out fatigue detection on the face area, and judging whether a driver is in fatigue driving;
and 4, step 4: and according to the judgment result of the fatigue detection unit, if the driver is in fatigue driving, early warning information is sent out, and if the driver is not in fatigue driving, no prompt is given.
6. The method as claimed in claim 5, wherein in step 2, the fatigue detection is performed on the face region according to the determined face region in the image, and the method for judging whether the driver is in fatigue driving performs the following steps:
step 2.1: according to the determined face area, determining the width of the left and right canthi of the human eyes in the face area: w ═ LR-LL; determining the height of the upper and lower eyelids: h is CU-CL; the degree of opening and closing of the human eye is determined using the following formula: r is H/W;
step 2.2: counting the blinking times of the driver in unit time so as to obtain the blinking frequency of the driver;
step 2.3: and judging whether the driver is in a fatigue driving state or not according to the opening and closing degree and the blinking frequency of the human eyes.
7. The method of claim 6, wherein the method further comprises: and establishing a deep learning model, and automatically detecting the acquired image information according to the deep learning model.
8. The method of claim 7, wherein the method of building a deep learning model performs the steps of:
step S1: inputting a face image training set;
step S2: training the input face image to form a forward neural network;
step S3: calculating a training error of the forward neural network; because the output variable E of the training is 'the face area in the image information', but a predicted value generated after the model training is O, the obtained error function is as follows:
where m represents the number of samples input into the modeling this time and i represents the ith variable.
Step S4: backpropagating update weights w
In order to reduce the error and improve the accuracy of model prediction, the neural network reversely transmits data from the output layer to the input layer, and the value of the weight w is readjusted until the model error reaches the minimum, and then the training is stopped, so that the model creation is completed.
9. An intelligent vehicle fatigue detection device based on face recognition based on the system of any one of claims 5 to 8, wherein the device is a non-transitory computer readable storage medium storing computing instructions, and the device comprises: acquiring an image information code segment of a driver in real time; a code segment for determining a region of a face in the image according to the image information acquired by the image acquisition unit; a code segment for performing fatigue detection on the face region according to the determined face region in the image and judging whether the driver is in fatigue driving; and a code segment for sending out early warning information if the driver is in fatigue driving according to the judgment result of the fatigue detection unit, and not giving a prompt if the driver is not in fatigue driving.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910801123.8A CN110807351A (en) | 2019-08-28 | 2019-08-28 | Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition |
| JP2020131441A JP2021034035A (en) | 2019-08-28 | 2020-08-03 | System, method, and device for intelligent vehicle loaded fatigue detection based on facial discrimination |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910801123.8A CN110807351A (en) | 2019-08-28 | 2019-08-28 | Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110807351A true CN110807351A (en) | 2020-02-18 |
Family
ID=69487460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910801123.8A Pending CN110807351A (en) | 2019-08-28 | 2019-08-28 | Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP2021034035A (en) |
| CN (1) | CN110807351A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111634233A (en) * | 2020-05-25 | 2020-09-08 | 杭州鸿泉物联网技术股份有限公司 | Safe driving system and method |
| CN112954848A (en) * | 2021-03-10 | 2021-06-11 | 深圳市大鑫浪电子科技有限公司 | Intelligent light control method and device, computer equipment and storage medium |
| CN115171082A (en) * | 2022-06-29 | 2022-10-11 | 北京百度网讯科技有限公司 | Driving behavior detection method and device, electronic equipment and readable storage medium |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113239729B (en) * | 2021-04-08 | 2024-02-27 | 南京交通职业技术学院 | Fatigue driving identification method based on data fusion |
| CN112959966B (en) * | 2021-04-26 | 2022-11-25 | 联友智连科技有限公司 | Vehicle-mounted multimedia regulation and control method, system, equipment and computer storage medium based on internet trip and user habit deep learning |
| CN113591533A (en) * | 2021-04-27 | 2021-11-02 | 浙江工业大学之江学院 | Anti-fatigue driving method, device, equipment and storage medium based on road monitoring |
| CN113269069B (en) * | 2021-05-18 | 2022-08-23 | 青岛科技大学 | Sleepiness identification method and device based on facial features of truck driver |
| CN113449678A (en) * | 2021-07-14 | 2021-09-28 | 宝能(广州)汽车研究院有限公司 | Driving state detection method and device, computer equipment and storage medium |
| CN113602287B (en) * | 2021-08-13 | 2024-01-26 | 吉林大学 | Man-machine co-driving system for drivers with low driving ages |
| CN114299756A (en) * | 2021-12-29 | 2022-04-08 | 盐城工学院 | Bus intelligent safety management system based on Internet of things and big data analysis |
| CN114373535A (en) * | 2022-01-13 | 2022-04-19 | 刘威 | Novel doctor-patient mechanism system based on Internet |
| CN114821713B (en) * | 2022-04-08 | 2023-04-07 | 湖南大学 | Video transform-based fatigue driving detection method |
| CN115107915A (en) * | 2022-06-10 | 2022-09-27 | 西北工业大学 | Bicycle-mounted system and using method thereof |
| CN114821757B (en) * | 2022-06-24 | 2022-09-16 | 北京鹰之眼智能健康科技有限公司 | Data processing system for acquiring visual fatigue state |
| CN115798019B (en) * | 2023-01-06 | 2023-04-28 | 山东星科智能科技股份有限公司 | Computer vision-based intelligent early warning method for practical training driving platform |
| CN116831581A (en) * | 2023-06-15 | 2023-10-03 | 中南大学 | Remote physiological sign extraction-based driver state monitoring method and system |
| CN117079255B (en) * | 2023-10-17 | 2024-01-05 | 江西开放大学 | Fatigue driving detection method based on face recognition and voice interaction |
| CN117115894A (en) * | 2023-10-24 | 2023-11-24 | 吉林省田车科技有限公司 | Non-contact driver fatigue state analysis method, device and equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN107595307A (en) * | 2017-10-23 | 2018-01-19 | 湖南科技大学 | Fatigue driving detection device and detection method based on machine vision eye recognition |
| CN109190468A (en) * | 2018-07-26 | 2019-01-11 | 深圳市赛亿科技开发有限公司 | A kind of fatigue driving monitoring method and system |
| CN109308445A (en) * | 2018-07-25 | 2019-02-05 | 南京莱斯电子设备有限公司 | A kind of fixation post personnel fatigue detection method based on information fusion |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09301011A (en) * | 1996-05-20 | 1997-11-25 | Honda Motor Co Ltd | Operating condition monitoring device for vehicle |
| KR101078466B1 (en) * | 2007-01-19 | 2011-10-31 | 아사히 가세이 가부시키가이샤 | Awake state judging model making device, awake state judging device, warning device, vehicle, awake state judging model making method, awake state judging method, computer readable medium recording the awake state judging model making method, and computer readable medium recording the awake state judging method |
| JP2009166783A (en) * | 2008-01-18 | 2009-07-30 | Toyota Motor Corp | Symptom inference device |
-
2019
- 2019-08-28 CN CN201910801123.8A patent/CN110807351A/en active Pending
-
2020
- 2020-08-03 JP JP2020131441A patent/JP2021034035A/en not_active Ceased
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN107595307A (en) * | 2017-10-23 | 2018-01-19 | 湖南科技大学 | Fatigue driving detection device and detection method based on machine vision eye recognition |
| CN109308445A (en) * | 2018-07-25 | 2019-02-05 | 南京莱斯电子设备有限公司 | A kind of fixation post personnel fatigue detection method based on information fusion |
| CN109190468A (en) * | 2018-07-26 | 2019-01-11 | 深圳市赛亿科技开发有限公司 | A kind of fatigue driving monitoring method and system |
Non-Patent Citations (1)
| Title |
|---|
| 徐锋 等: ""基于肤色检测和人眼定位的人脸检测方法"", 《计算机系统应用》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111634233A (en) * | 2020-05-25 | 2020-09-08 | 杭州鸿泉物联网技术股份有限公司 | Safe driving system and method |
| CN112954848A (en) * | 2021-03-10 | 2021-06-11 | 深圳市大鑫浪电子科技有限公司 | Intelligent light control method and device, computer equipment and storage medium |
| CN115171082A (en) * | 2022-06-29 | 2022-10-11 | 北京百度网讯科技有限公司 | Driving behavior detection method and device, electronic equipment and readable storage medium |
| CN115171082B (en) * | 2022-06-29 | 2024-01-19 | 北京百度网讯科技有限公司 | Driving behavior detection method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2021034035A (en) | 2021-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110807351A (en) | Intelligent vehicle-mounted fatigue detection system, method and device based on face recognition | |
| CN109389806B (en) | Fatigue driving detection early warning method, system and medium based on multi-information fusion | |
| Ji et al. | Fatigue state detection based on multi-index fusion and state recognition network | |
| CN109460703B (en) | Non-invasive fatigue driving identification method based on heart rate and facial features | |
| CN108664947A (en) | A kind of fatigue driving method for early warning based on Expression Recognition | |
| CN104200192A (en) | Driver gaze detection system | |
| CN110464633A (en) | Acupuncture point recognition methods, device, equipment and storage medium | |
| Batista | A drowsiness and point of attention monitoring system for driver vigilance | |
| CN111062292B (en) | Fatigue driving detection device and method | |
| CN110728241A (en) | Driver fatigue detection method based on deep learning multi-feature fusion | |
| CN107798318A (en) | The method and its device of a kind of happy micro- expression of robot identification face | |
| CN108960185A (en) | Vehicle target detection method and system based on YOLOv2 | |
| CN109977930A (en) | Method for detecting fatigue driving and device | |
| Tabrizi et al. | Open/closed eye analysis for drowsiness detection | |
| CN102567734A (en) | Specific value based retina thin blood vessel segmentation method | |
| CN113989788A (en) | Fatigue detection method based on deep learning and multi-index fusion | |
| Sun et al. | Driver fatigue detection system based on colored and infrared eye features fusion | |
| CN111522437A (en) | Method and system for obtaining product prototype based on eye movement data | |
| CN106960199B (en) | Complete extraction method of white eye region of true color eye picture | |
| Zhao et al. | Deep convolutional neural network for drowsy student state detection | |
| Panicker et al. | Open-eye detection using iris–sclera pattern analysis for driver drowsiness detection | |
| Reddy et al. | Diabetic retinopathy through retinal image analysis: A review | |
| CN115496700A (en) | Disease detection system and method based on eye image | |
| CN108288040A (en) | Multi-parameter face identification system based on face contour | |
| CN116030042B (en) | Diagnostic device, method, equipment and storage medium for doctor's diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200218 |
|
| RJ01 | Rejection of invention patent application after publication |