CN110751051B - Abnormal driving behavior detection method based on machine vision - Google Patents
Abnormal driving behavior detection method based on machine vision Download PDFInfo
- Publication number
- CN110751051B CN110751051B CN201910899783.4A CN201910899783A CN110751051B CN 110751051 B CN110751051 B CN 110751051B CN 201910899783 A CN201910899783 A CN 201910899783A CN 110751051 B CN110751051 B CN 110751051B
- Authority
- CN
- China
- Prior art keywords
- image
- driver
- abnormal
- skin color
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 80
- 238000001514 detection method Methods 0.000 title claims abstract description 76
- 230000006399 behavior Effects 0.000 claims abstract description 75
- 238000013461 design Methods 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 21
- 230000033001 locomotion Effects 0.000 claims abstract description 10
- 206010000117 Abnormal behaviour Diseases 0.000 claims abstract description 4
- 238000000605 extraction Methods 0.000 claims description 35
- 238000007781 pre-processing Methods 0.000 claims description 26
- 238000011161 development Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 9
- 238000003708 edge detection Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000013145 classification model Methods 0.000 claims description 2
- 238000007405 data analysis Methods 0.000 claims description 2
- 208000019622 heart disease Diseases 0.000 claims description 2
- 238000013186 photoplethysmography Methods 0.000 claims description 2
- 238000012549 training Methods 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 6
- 230000001815 facial effect Effects 0.000 description 11
- 238000011160 research Methods 0.000 description 9
- 206010039203 Road traffic accident Diseases 0.000 description 5
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 210000003296 saliva Anatomy 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a machine vision-based abnormal driving behavior detection method, which comprises the following steps: on the hardware selection type 2, the image acquisition module 4 is selected to acquire face information and hand gesture information of a driver, heart rate detection and hand gesture detection of the driver are completed on the face of the driver through the algorithm design 1, and then the abnormal result processing module 15 synthesizes heart rate results of the heart rate detection and results of the hand gesture abnormal detection, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken. According to the invention, the two detection results are synthesized, the driving behavior classification is carried out on the acquired movement characteristics of the driver through the design classifier, the recognition of abnormal driving behaviors is realized, and the abnormal result processing module processes the abnormal result.
Description
Technical Field
The invention relates to the technical field of industrial intellectualization and machine vision, in particular to a machine vision-based abnormal driving behavior detection method which is used for accurately detecting the behavior of a driver in real time.
Background
At present, detection and monitoring of abnormal driving behaviors at home and abroad become key research work in the field of intelligent traffic research, and timely monitoring of driving states of drivers has important significance for improving safety work efficiency of drivers, reducing accident rate and improving traffic environment.
The current research on abnormal driving is mainly limited to the research on drunk driving, and the research on drunk driving comprises an on-site test method, respiratory alcohol detection, blood alcohol detection, saliva alcohol detection and alcohol key detection, and although the technologies can avoid the occurrence of drunk driving accidents, the feasibility is not strong, and the technologies are only detected after drunk driving, so that the anti-drunk driving effect is not obvious. In addition to the above several conventional technologies, detection of the driving state based on the sensor is also the current research direction, but thus the driver needs to wear a complex physiological state detector, and meanwhile needs to install more sensors in the vehicle, which is not feasible and has high cost.
Disclosure of Invention
Based on market demands and current insufficient researches, the invention provides a vision-based abnormal driving behavior detection system, which is based on a machine vision algorithm, analyzes facial features and gesture features of a driver, further judges whether the driver has drunk driving and other abnormal driving behaviors, further takes preventive measures, adopts a camera to collect information, has little influence on the driving behaviors of the driver, and has feasibility.
The system not only effectively guarantees the life and property safety of the public, reduces the economic loss and casualties caused by traffic accidents, but also has great contribution to the promotion of modern traffic loss and the promotion of industrial intellectualization, and has wide application prospect.
The invention provides a design method of an abnormal driving behavior detection system based on a machine vision learning algorithm, which is used for detecting the face of a driver and the abnormal driving behavior of the driver, judging and alarming the face of the driver and further preventing traffic accidents.
In order to achieve the above object, the present invention adopts the following specific detection method:
a machine vision-based abnormal driving behavior detection method, the method comprising the steps of: on the hardware selection type 2, the image acquisition module 4 is selected to acquire face information and hand gesture information of a driver, heart rate detection and hand gesture detection of the driver are completed on the face of the driver through the algorithm design 1, and then the abnormal result processing module 15 synthesizes heart rate results of the heart rate detection and results of the hand gesture abnormal detection, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken.
Furthermore, the algorithm design 1 mainly comprises the steps of an image preprocessing algorithm, a skin color model building algorithm, an interested region extraction algorithm and a classifier design; the algorithm steps are respectively realized by an image preprocessing algorithm module 5, a skin color model building algorithm module 6, an interested region extraction algorithm module 7 and a classifier design module 8;
the image preprocessing algorithm module 5 is mainly used for preprocessing images, and mainly comprises the step of preprocessing the obtained face images and hand gesture images of a driver so as to facilitate the extraction of the characteristics of the images, and specifically comprises the steps of image graying 9, image equalization 10 and image denoising filtering 11; the skin color model building algorithm module 6 is used for building a skin color model by researching skin color clustering characteristics in a specific color space, selecting skin color samples, building a YCrCb skin color model based on no difference of brightness information and obvious clustering distribution of a human face on chromaticity information, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human face by using a LabVIEW tool function to obtain a skin color area of the human face; the interested region extraction algorithm module 7 is used for extracting interested regions of human faces and human hands; extracting and adopting coordinates of a central point of a face to position the region of interest of the face, calculating the radius of the region of interest, and further calculating the area of the region of interest by utilizing the aspect ratio of the face of the public; firstly, carrying out edge detection on an image by using an operator aiming at the extraction of an interested region of the hand gesture of a driver, then carrying out least square ellipse fitting to locate the position of a steering wheel, and determining the interested region; the classifier design module 8 analyzes the extracted features based on a Bayesian classification model, designs a classifier, trains the acquired images by taking the positions, the movement speeds and the movement angles of the hands of the driver as movement feature parameters, further establishes a model, classifies the acquired movement features of the driver in driving behaviors, and judges whether abnormal conditions exist in the driving behaviors;
after preprocessing the facial image and the gesture image of the driver through the image preprocessing algorithm module 5, removing redundant noise, respectively establishing a facial skin color model and a hand skin color model through the interested trend extraction algorithm module 7, judging whether the extraction of the interested region is finished, continuously acquiring the image if the extraction is not finished, extracting and analyzing facial features if the extraction is finished, judging whether the heart rate is normal, judging drunk driving if the heart rate is abnormal, and alarming; if the driving behavior is normal, the characteristics of the hand are extracted, the abnormal driving behavior is classified through the classifier design module 8, whether the driving behavior is illegal or not is judged, if the driving behavior is illegal, an alarm is given, if the driving behavior is not illegal, whether the detection is to be stopped is judged, if the detection is to be stopped, the detection is ended, and otherwise, the characteristic extraction is carried out again to judge.
Furthermore, the image preprocessing algorithm module 5, the skin color model building algorithm module 6, the region of interest extraction algorithm module 7 and the classifier design module 8 in the algorithm design 1 are all realized through the software design 3, and mainly comprise a development environment, a development language and a software design process of the system, and are developed and designed based on LabVIEW and G languages.
Further, the graying 9 of the image is to convert the color image of the driver acquired by the image acquisition module into an 8-bit gray scale image, and the weight values of r, g and b of the image are acquired by adopting a weighted average method by means of an image processing tool of vision of LabVIEW; the equalization 10 of the image is to avoid the influence of factors such as ambient light and the like on the image, so as to influence the stability of image detection, enhance the contrast of the image through a histogram equalization algorithm and avoid the loss of image information; the image denoising filter 11 is used for suppressing and removing other information in the image, filtering and denoising the equalized image by adopting a median filtering method, removing redundant noise points and better retaining details of the image.
Furthermore, the heart rate detection is to carry out non-contact measurement on the face of a driver based on a photoplethysmography, when the face of the driver has abnormal conditions such as drunk driving or heart disease burst, the skin color and heart rate of the face are changed, a face skin color model YCrCb of the driver is established, a characteristic region is extracted, a heart rate signal QRS is further detected based on a wavelet algorithm, the characteristics of the heart rate signal QRS are extracted and analyzed, the Q, R, S points of signals are detected by means of a LabVIEW advanced signal toolbox, the wavelet classification is further carried out on the heart rate signal, the cycle calculation of R-R points is further carried out, and the heart rate is calculated.
Further, the image acquisition module 4 selects a Kinect camera for reading a video image file of a driver, and performs development based on a LabVIEW development environment and a G language.
Further, the hand gesture detection of the driver is to collect images of the hands of the driver through a Kinect camera, extract gesture features of the hands and classify the gesture features.
Furthermore, the abnormal result processing module 15 is mainly used for processing the abnormal detection behaviors output by the abnormal driving behavior detection system, and when detecting that the heart rate of the driver is abnormal and drunk driving is possible, the abnormal result processing module alarms, and the specific alarm mode can upload the driving condition and driving position of the driver to a traffic department through a cloud or other modes so as to facilitate the interference and tracking processing of the behaviors by the traffic department; after the abnormal driving behaviors are classified, when dangerous driving possibility of the driving behaviors is found, safety braking is carried out, and the vehicle is safely stopped.
The beneficial effects of the invention are as follows:
the invention not only can effectively prevent drunk driving events, but also can prevent accidents caused by other abnormal driving, and can provide guarantee for driving safety of people, is more beneficial to social safety and provides effective guarantee for development of automobile industry. At present, china only stays on legal support and propaganda for drunk driving, however, the effect of the drunk driving is very small, few people have a mind of being happy, and therefore traffic accidents are caused, and the phenomenon can be completely eradicated by the products, so that the drunk driving has wide market popularization.
The method for comprehensively detecting the heart rate and the hand gesture abnormality reduces erroneous judgment on abnormal driving behaviors and is higher in accuracy. The heart rate detection is based on vision acquisition, and the non-contact measurement can detect and monitor the heart rate of a human body, has no interference on driving behaviors, and plays a role in assisting the health detection of the human body at any time. The abnormal hand gesture detection based on vision effectively detects the abnormal gesture of the hand in the driving process of the driver, and can timely monitor the behavior in the driving process so as to reduce traffic accidents. The design of the abnormal result processing module can further timely remind the driver of improper driving behaviors, timely remind the driver of correction, meanwhile, the driver can be helped to know the health state of the driver to a certain extent, the traffic department can also conveniently record and process the abnormal driving behaviors of the driver, the abnormal driving behaviors can be further reduced, the traffic accidents are reduced to a certain extent, and the traffic safety and the public security are maintained.
Drawings
FIG. 1 is a general design of the system of the present invention
Fig. 2 is a system workflow diagram of the present invention.
Fig. 3 is a flowchart of the algorithm of the present invention.
Fig. 4 is a software design flow chart of the present invention.
Fig. 5 is a technical flowchart of the present invention.
FIG. 6 is a flow chart of heart rate algorithm detection.
Detailed Description
The following detailed description of the invention, taken in conjunction with the drawings, is provided to more fully illustrate the advantages and features of the invention and to more clearly and unequivocally define the scope of the invention, as will be readily understood by those skilled in the art.
The abnormal driving behavior detection system based on machine vision mainly comprises three major parts, namely algorithm design, hardware model selection and software design.
The key of hardware model selection is that the system is provided with an image acquisition module, and the Kinect sensor is provided with an infrared camera and a depth camera, can obtain color images and depth images and has rich image information.
The software design is a specific software programming flow chart of the system, and is developed based on LabVIEW program development environment and G language.
Fig. 1 shows a general design of the system of the present invention. The invention relates to a machine vision-based abnormal driving behavior detection system, which mainly comprises an algorithm design 1, a hardware selection type 2 and a software design 3. The hardware model selection 2 comprises an image acquisition module 4 of the system, which is used for acquiring facial information of people; the software design 3 mainly comprises a development environment of a system, a development language and a software design process of the system, and is developed and designed based on LabVIEW and G languages; the algorithm design 1 is an algorithm for realizing system functions and mainly comprises an image preprocessing algorithm module 5, a skin color model building algorithm module 6, a region of interest extraction algorithm module 7 and a classifier design module 8.
The image preprocessing algorithm 5 mainly performs a preprocessing algorithm on an image, and mainly comprises the step of performing image preprocessing on an acquired face image and hand gesture image of a driver so as to facilitate the extraction of the characteristics of the image, and specifically comprises the step of graying 9, the step of image equalization 10 and the step of image denoising and filtering 11.
The graying of the image is 9, namely, the color image of the driver acquired by the image acquisition module is converted into an 8-bit gray scale image, the weight of R, G, B of the image is acquired by adopting a weighted average method by means of an image processing tool of vision of LabVIEW, and a gray scale value calculation formula is as follows: k= (ar+bg+cb)/3, a=0.25, b=0.64, and c=0.11 among parameters of examples.
The image equalization 10 is to avoid the influence of factors such as ambient light, etc. on the image, so as to further influence the stability of image detection, and enhance the contrast of the image, i.e. increase the brightness information of the image, by using a histogram equalization algorithm, so as to avoid the loss of image information.
The denoising filter 11 of the image is to suppress and remove other information in the image, filter and denoise the equalized image by selecting a median filtering method, remove redundant noise points and betterThe details of the image are preserved. Selecting one of x ij Rectangular area 5×5 as center
Filtered output
The skin color model building algorithm 6 is used for researching skin color clustering characteristics in a specific color space, selecting skin color samples to build a skin color model, building a YCrCb (Y is a gray scale value, cr is a tone, cb is saturation) skin color model based on no difference of brightness information and obvious clustering distribution of a human face on chromaticity information, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human face by using a LabVIEW tool function, so that the skin color area of the human face can be obtained, wherein a color space conversion formula is as follows:
based on data analysis, the feature region of the skin color model resembles an ellipse (ellipse constructed with respect to x, y coordinates), and the skin color model builds a formula:
the region of interest extraction algorithm 7 is a region of interest for extracting a human face and a human hand. Extracting and adopting coordinates of a central point of a face to position the region of interest of the face, calculating the radius of the region of interest, and further calculating the area of the region of interest by utilizing the aspect ratio of the face of the public; the method comprises the steps of firstly, carrying out edge detection on an image by using a Canny operator aiming at the extraction of a region of interest of the hand gesture of a driver, then, carrying out least square ellipse fitting to locate the position of a steering wheel, and determining the region of interest.
Extraction of region of interest of human face to obtain coordinate points (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Further calculate the center point coordinatesI.e. radius of the region of interest +.>Thereby determining the region of interest.
Figure 2 shows a system workflow diagram of the present invention. Firstly, a Kinect camera of the image acquisition module 1 reads a video file of a driver, and develops based on a LabVIEW development environment and G language, firstly, an abnormal driving behavior detection system detects facial information of the driver, after judging whether heart rate is normal or not, if not, an abnormal behavior result is output, an alarm is given, if normal, abnormal driving behavior is detected, if not, an alarm is given, and if normal, continuous detection is continued.
Fig. 3 shows a flowchart of the algorithm of the present invention. Firstly, an image preprocessing algorithm module 5 is adopted for processing an image of an acquired video file, specifically comprising an image graying 8, an image equalizing 9, a wavelet filtering algorithm 10 and an edge detection 11, after the preprocessing of the image is completed, the characteristics of the image are analyzed, specifically, ellipse detection is adopted, further, an interested region extraction algorithm module 7 is adopted for respectively extracting an interested region and a hand characteristic region of the face of a driver, a skin color model algorithm building module 6 is adopted for respectively building a skin color model YCrCb of the face and a self-adaptive skin color feature model of the hand, and a heart rate detection algorithm is further adopted for calculating the heart rate by utilizing the skin color model of the face to judge whether the heart rate is abnormal or not, if so, drunk driving behaviors possibly exist, and further abnormal results are processed; and classifying the driving behaviors by using the self-adaptive skin color feature model of the hand and adopting a classifier design module 8 to judge whether the driving behaviors are abnormal, and if so, processing the abnormal results.
FIG. 4 is a flow chart of the software design of the present invention. Firstly, the abnormal driving behavior detection system based on machine vision is used for preprocessing the facial image and the gesture image of a driver through an image acquisition module 4, removing redundant noise after the image preprocessing algorithm module 5 is used for preprocessing the facial image and the gesture image of the driver, respectively establishing a skin color model of the face and a skin color model of the hand through an interesting trend extraction algorithm module 7, judging whether extraction of an interesting region is finished, if not finishing, continuing to acquire an image, if finishing, extracting and analyzing facial features, judging whether the heart rate is normal, if not, judging drunk driving, and alarming; if the driving behavior is normal, the characteristics of the hand are extracted, the abnormal driving behavior is classified through the classifier design module 8, whether the driving behavior is illegal or not is judged, if the driving behavior is illegal, an alarm is given, if the driving behavior is not illegal, whether the detection is to be stopped is judged, if the detection is to be stopped, the detection is ended, and otherwise, the characteristic extraction is carried out again to judge.
Fig. 5 shows an abnormal result processing module 15 according to the present invention. The abnormal result processing mainly processes abnormal detection behaviors output by the abnormal driving behavior detection system, when detecting that the heart rate of the driver is abnormal and drunk driving is possible, an alarm is given, and a specific alarm mode can upload the driving condition and driving position of the driver to a traffic department through a cloud or other modes so as to facilitate interference and tracking processing of the behaviors by the traffic department; after the abnormal driving behaviors are classified, when dangerous driving possibility of the driving behaviors is found, safety braking is carried out, and the vehicle is safely stopped.
In the invention, the heart rate detection is based on the acquisition of facial information of a human face by a camera, and the software based on the heart rate detection is LabVIEW software for programming, and a software flow chart is shown in FIG. 6. The hand gesture detection of the driver is to collect images of the hands of the driver through a Kinect camera, extract gesture features of the hands and classify the gesture features. The abnormal result processing module 15 specifically comprises a language prompt warning part and a remote warning part, and the abnormal result is based on the LabVIEW software G language design and adopts a voice broadcasting mode to remind abnormal driving behaviors.
The invention provides a design method of an abnormal driving behavior detection system based on machine vision, which aims at detecting abnormal driving behaviors, can reduce the occurrence of the abnormal driving behaviors to a certain extent and improves traffic conditions.
In summary, the design method of the abnormal driving behavior detection system based on machine vision is used for detecting abnormal driving behaviors of a driver in the driving process so as to take effective measures, and mainly comprises three parts, namely algorithm design, hardware model selection and software design. The algorithm of the main research of algorithm design comprises an image preprocessing algorithm, a region of interest extraction algorithm, a skin color model building algorithm and a classifier design research. The original video image obtained by the camera contains complex background and a plurality of image noise which causes interference to video detection, and the collected image needs to be preprocessed in order to improve the recognition and detection precision of abnormal driving behaviors. The method comprises the steps of researching skin color clustering characteristics in a specific color space, selecting skin color samples to establish a skin color model YCrCb, adopting a locating face center point coordinate and calculating the radius of an interested region, further calculating the area of the interested region of the face by utilizing the aspect ratio of the face of the public, and adopting a direct least square ellipse fitting algorithm to detect and locate the interested region of the hand gesture of a driver in the steering wheel region of the image. And analyzing the facial feature information and the hand gesture feature information of the person to finish heart rate detection and hand gesture detection, integrating the two detection results, classifying the driving behaviors of the acquired motion features of the driver through a design classifier, realizing the recognition of abnormal driving behaviors, and processing the abnormal results by an abnormal result processing module.
Claims (5)
1. The abnormal driving behavior detection method based on machine vision is characterized by comprising the following steps of: the method comprises the following steps: on the hardware selection (2), an image acquisition module (4) is selected to acquire face information and hand gesture information of a driver, heart rate detection and hand gesture detection of the driver are completed on the face of the driver through an algorithm design (1), and then an abnormal result processing module (15) synthesizes heart rate results of heart rate detection and results of hand gesture abnormal detection, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken;
the algorithm design (1) mainly comprises the steps of an image preprocessing algorithm, a skin color model building algorithm, an interested region extraction algorithm and a classifier design; the algorithm steps are respectively realized by an image preprocessing algorithm module (5), a skin color model building algorithm module (6), an interested region extraction algorithm module (7) and a classifier design module (8);
the image preprocessing algorithm module (5) is mainly used for preprocessing images and mainly comprises the step of preprocessing the obtained face images and hand gesture images of a driver so as to facilitate the extraction of the characteristics of the images, and specifically comprises the steps of image graying (9), image equalization (10) and image denoising filtering (11); the skin color model building algorithm module (6) is used for building a skin color model by researching skin color clustering characteristics in a specific color space, selecting skin color samples, building a YCrCb skin color model based on no difference of brightness information and obvious clustering distribution of a human face on chromaticity information, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human face by using a LabVIEW tool function to obtain a skin color area of the human face; the interested region extraction algorithm module (7) is used for extracting interested regions of human faces and human hands; extracting and adopting coordinates of a central point of a face to position the region of interest of the face, calculating the radius of the region of interest, and further calculating the area of the region of interest by utilizing the aspect ratio of the face of the public; firstly, carrying out edge detection on an image by using an operator aiming at the extraction of an interested region of the hand gesture of a driver, then carrying out least square ellipse fitting to locate the position of a steering wheel, and determining the interested region; the classifier design module (8) is used for analyzing the extracted features based on a Bayesian classification model, designing a classifier, taking the positions, the movement speeds and the movement angles of the hands of a driver as movement feature parameters, training the acquired images, further establishing a model, classifying the acquired movement features of the driver in driving behaviors, and judging whether abnormal conditions exist in the driving behaviors;
after preprocessing a face image and a gesture image of a driver through an image preprocessing algorithm module (5), removing redundant noise, respectively establishing a face skin color model and a hand skin color model through an interested region extraction algorithm module (7), judging whether the interested region extraction is finished, continuously acquiring the image if the extraction is not finished, extracting and analyzing the face features if the extraction is finished, judging whether the heart rate is normal, judging drunk driving if the heart rate is abnormal, and giving an alarm; if the driving behavior is normal, extracting the characteristics of the hand, classifying the driving abnormal driving behavior through a classifier design module (8), judging whether the driving behavior is illegal, alarming if the driving behavior is illegal, judging whether to stop detection if the driving behavior is not illegal, and ending if the detection is stopped, otherwise, carrying out characteristic extraction again to judge;
the heart rate detection is to carry out non-contact measurement on the face of a driver based on a photoplethysmography, when the abnormal conditions such as drunk driving or heart disease burst exist in the driver, the skin color and heart rate of the face are changed, a face skin color model YCrCb of the driver is established, a characteristic region is extracted, a heart rate signal QRS is further detected based on a wavelet algorithm, the characteristics of the heart rate signal QRS are extracted and analyzed, the Q, R, S points of a signal are detected by a high-level signal toolbox of LabVIEW, the wavelet classification is further carried out on the heart rate signal, the period calculation of R-R points is further carried out, and the heart rate is calculated;
the abnormal result processing module (15) is mainly used for processing abnormal detection behaviors output by the abnormal driving behavior detection system, and when detecting that the heart rate of the driver is abnormal and drunk driving is possible, the abnormal result processing module alarms, and a specific alarm mode can upload the driving condition and driving position of the driver to a traffic department through a cloud or other modes so as to facilitate interference and tracking processing of the behaviors by the traffic department; after the abnormal driving behaviors are classified, when dangerous driving possibility of the driving behaviors is found, safety braking is carried out, so that the vehicle is safely stopped;
the skin color model building algorithm module (6) is used for building a skin color model by researching skin color clustering characteristics in a specific color space, selecting skin color samples, building a YCrCb skin color model based on no difference of human faces in brightness information and obvious clustering distribution on chromaticity information, wherein Y is a gray scale value, cr is a tone, cb is saturation, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human faces by using a LabVIEW tool function, so that the skin color area of the human faces can be obtained, and the color space conversion formula is as follows:
based on data analysis, the feature area of the skin color model resembles an ellipse constructed about x and y coordinates, and the skin color model builds a formula:
the interested region extraction algorithm module (7) is used for extracting the interested region of the human face and the human hand, positioning the coordinates of the central point of the human face and calculating the radius of the interested region aiming at the interested region of the human face, and further calculating the area of the interested region by utilizing the aspect ratio of the face of the public; firstly, carrying out edge detection on an image by using a Canny operator aiming at the extraction of an interested region of the hand gesture of a driver, then carrying out least square ellipse fitting to locate the position of a steering wheel, and determining the interested region;
extraction of region of interest of human face to obtain coordinate points (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Further calculate the center point coordinatesI.e. radius of the region of interest +.>Thereby determining the region of interest.
2. The machine vision-based abnormal driving behavior detection method according to claim 1, wherein: the image preprocessing algorithm module (5), the skin color model building algorithm module (6), the region of interest extraction algorithm module (7) and the classifier design module (8) in the algorithm design (1) are all realized through the software design (3), and mainly comprise a development environment of a system, a development language and a software design process of the system, and are developed and designed based on LabVIEW and G languages.
3. The machine vision-based abnormal driving behavior detection method according to claim 1, wherein: the graying (9) of the image is to convert the color image of the driver acquired by the image acquisition module into an 8-bit gray scale image, and the weight values of r, g and b of the image are acquired by adopting a weighted average method by means of an image processing tool of vision of LabVIEW; the equalization (10) of the image is to avoid the influence of factors such as ambient light and the like on the image, so as to influence the stability of image detection, enhance the contrast of the image through a histogram equalization algorithm and avoid the loss of image information; the image denoising filter (11) is used for suppressing and removing other information in the image, filtering and denoising the equalized image by adopting a median filtering method, removing redundant noise points and better retaining details of the image.
4. The machine vision-based abnormal driving behavior detection method according to claim 1, wherein: the image acquisition module (4) selects a Kinect camera and is used for reading a video image file of a driver and developing based on a LabVIEW development environment and G language.
5. The machine vision-based abnormal driving behavior detection method according to claim 4, wherein: the hand gesture detection of the driver is to collect images of the hands of the driver through a Kinect camera, extract gesture features of the hands and classify the gesture features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899783.4A CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899783.4A CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110751051A CN110751051A (en) | 2020-02-04 |
| CN110751051B true CN110751051B (en) | 2024-03-19 |
Family
ID=69276902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910899783.4A Active CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751051B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310730A (en) * | 2020-03-17 | 2020-06-19 | 扬州航盛科技有限公司 | Driving behavior early warning system based on facial expressions |
| CN113642358B (en) * | 2020-04-27 | 2023-10-10 | 华为技术有限公司 | Skin color detection method, device, terminal and storage medium |
| CN111666818B (en) * | 2020-05-09 | 2023-06-16 | 大连理工大学 | Driver abnormal posture detection method |
| CN111637610B (en) * | 2020-06-24 | 2022-04-01 | 山东建筑大学 | Indoor environment health degree adjusting method and system based on machine vision |
| JP7327301B2 (en) * | 2020-07-02 | 2023-08-16 | トヨタ自動車株式会社 | Driver monitoring device |
| CN111914707A (en) * | 2020-07-22 | 2020-11-10 | 上海大学 | System and method for detecting drunkenness behavior |
| CN113792663B (en) * | 2021-09-15 | 2024-05-14 | 东北大学 | Method, device and storage medium for detecting drunk driving and fatigue driving of driver |
| CN114469024B (en) * | 2021-12-23 | 2023-12-22 | 广东智云城建科技有限公司 | Intelligent bracelet-based construction worker safety early warning method and system |
| CN114663863A (en) * | 2022-02-24 | 2022-06-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN117495384B (en) * | 2023-11-07 | 2024-04-26 | 广州准捷电子科技有限公司 | KTV face brushing payment method based on AI face recognition technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN106651910A (en) * | 2016-11-17 | 2017-05-10 | 北京蓝天多维科技有限公司 | Intelligent image analysis method and alarm system for abnormal driver behavior state |
| CN109034132A (en) * | 2018-09-03 | 2018-12-18 | 深圳市尼欧科技有限公司 | A kind of detection method driving abnormal behaviour |
-
2019
- 2019-09-23 CN CN201910899783.4A patent/CN110751051B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN106651910A (en) * | 2016-11-17 | 2017-05-10 | 北京蓝天多维科技有限公司 | Intelligent image analysis method and alarm system for abnormal driver behavior state |
| CN109034132A (en) * | 2018-09-03 | 2018-12-18 | 深圳市尼欧科技有限公司 | A kind of detection method driving abnormal behaviour |
Non-Patent Citations (2)
| Title |
|---|
| 基于感兴趣区域的头像视频前处理方法;曾鸿军;沈燕飞;王毅;;计算机工程与应用;20171231(第06期);全文 * |
| 基于红外光源的驾驶员眼睛特征提取;邸巍;王荣本;;交通信息与安全;20090215(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110751051A (en) | 2020-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110751051B (en) | Abnormal driving behavior detection method based on machine vision | |
| Omidyeganeh et al. | Intelligent driver drowsiness detection through fusion of yawning and eye closure | |
| CN101950355B (en) | Method for detecting fatigue state of driver based on digital video | |
| CN106846734B (en) | A kind of fatigue driving detection device and method | |
| CN103714660B (en) | System for achieving fatigue driving judgment on basis of image processing and fusion between heart rate characteristic and expression characteristic | |
| CN107292251B (en) | Driver fatigue detection method and system based on human eye state | |
| CN108693973B (en) | Emergency condition detection system fusing electroencephalogram signals and environmental information | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| CN107133564B (en) | Tooling cap detection method | |
| CN105286802B (en) | Driver Fatigue Detection based on video information | |
| CN110728241A (en) | Driver fatigue detection method based on deep learning multi-feature fusion | |
| WO2013013487A1 (en) | Device and method for monitoring driving behaviors of driver based on video detection | |
| CN104013414A (en) | Driver fatigue detecting system based on smart mobile phone | |
| CN102938058A (en) | Method and system for video driving intelligent perception and facing safe city | |
| CN102616241A (en) | Lane departure alarm system based on lane line model detection method and on-line study method | |
| CN102289660A (en) | Method for detecting illegal driving behavior based on hand gesture tracking | |
| CN111553214B (en) | Method and system for detecting smoking behavior of driver | |
| CN104881956A (en) | Fatigue driving early warning system | |
| CN107563346A (en) | One kind realizes that driver fatigue sentences method for distinguishing based on eye image processing | |
| CN112241647B (en) | Dangerous driving behavior early warning device and method based on depth camera | |
| CN117523537A (en) | Dynamic judging method for dangerous degree of vehicle driving | |
| Rani et al. | Development of an Automated Tool for Driver Drowsiness Detection | |
| CN113232667A (en) | Physiological state identification and driving safety early warning system based on IPPG technology | |
| CN116012822B (en) | Fatigue driving identification method and device and electronic equipment | |
| CN111724565A (en) | Safe driving monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |