CN110751051A - Abnormal driving behavior detection method based on machine vision - Google Patents
Abnormal driving behavior detection method based on machine vision Download PDFInfo
- Publication number
- CN110751051A CN110751051A CN201910899783.4A CN201910899783A CN110751051A CN 110751051 A CN110751051 A CN 110751051A CN 201910899783 A CN201910899783 A CN 201910899783A CN 110751051 A CN110751051 A CN 110751051A
- Authority
- CN
- China
- Prior art keywords
- image
- driver
- abnormal
- detection
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 77
- 238000001514 detection method Methods 0.000 title claims abstract description 74
- 230000006399 behavior Effects 0.000 claims abstract description 78
- 238000013461 design Methods 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 23
- 230000033001 locomotion Effects 0.000 claims abstract description 10
- 206010000117 Abnormal behaviour Diseases 0.000 claims abstract description 4
- 206010036436 Posture abnormal Diseases 0.000 claims abstract description 3
- 238000007781 pre-processing Methods 0.000 claims description 21
- 238000000605 extraction Methods 0.000 claims description 19
- 238000001914 filtration Methods 0.000 claims description 13
- 230000018109 developmental process Effects 0.000 claims description 11
- 238000011161 development Methods 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 5
- 238000003708 edge detection Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 3
- 230000033772 system development Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000013145 classification model Methods 0.000 claims description 2
- 208000010125 myocardial infarction Diseases 0.000 claims description 2
- 230000002401 inhibitory effect Effects 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 4
- 238000011160 research Methods 0.000 description 7
- 206010039203 Road traffic accident Diseases 0.000 description 5
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000008140 language development Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000013186 photoplethysmography Methods 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 210000003296 saliva Anatomy 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a machine vision-based abnormal driving behavior detection method, which comprises the following steps: on hardware model selection 2, an image acquisition module 4 is selected to acquire face information and hand posture information of a driver, heart rate detection and hand posture detection of the driver are completed on the face of the driver through algorithm design 1, and then an abnormal result processing module 15 integrates heart rate results of the heart rate detection and hand posture abnormal detection results, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken. The invention integrates two detection results, classifies the driving behaviors of the acquired driver motion characteristics by designing a classifier, realizes the identification of abnormal driving behaviors, and processes abnormal results by an abnormal result processing module.
Description
Technical Field
The invention relates to the technical field of industrial intellectualization and machine vision, in particular to an abnormal driving behavior detection method based on machine vision, which is used for accurately detecting the behavior of a driver in real time.
Background
At present, the detection and monitoring of abnormal driving behaviors at home and abroad become key research work in the field of intelligent traffic research, and the timely monitoring of the driving state of a driver has important significance for improving the safe working efficiency of the driver, reducing the accident rate and improving the traffic environment.
The study to the abnormal driving is mainly limited to the study that the wine drove at present, and to the study that the wine drove, there is the on-the-spot test method, and respiratory formula alcohol detects, and blood alcohol detects, and saliva alcohol detects, and alcohol key detects, and although these techniques can avoid some wine to drive the emergence of accident, the feasibility is not strong, and these all probably all obtain the detection after the wine drives, and it is not significant to prevent that the wine drives the effect. In addition to the above conventional technologies, the detection of the driving state based on the sensor is also the current research direction, but therefore, the driver needs to wear a complex physiological state detector, and needs to install more sensors in the vehicle, which is not feasible and has higher cost.
Disclosure of Invention
Based on market demands and current research deficiencies, the abnormal driving behavior detection system based on vision is provided by the project, the invention is based on an algorithm of machine vision, facial features and posture features of a driver are analyzed, whether the driver has drunk driving and other abnormal driving behaviors is further judged, preventive measures are further taken, a camera is adopted for information acquisition, the influence on the driving behavior of the driver is small, and the system has feasibility.
The invention of the system not only effectively guarantees the life and property safety of the social public, reduces the economic loss and the casualties caused by traffic accidents, but also greatly contributes to the promotion of modern traffic unemployment and the promotion of industrial intelligence, and has wide application prospect.
The invention provides a design method of an abnormal driving behavior detection system based on a machine vision learning algorithm, which is used for detecting the face of a driver and the abnormal driving behavior of the driver, and further preventing traffic accidents by judging and alarming.
In order to achieve the purpose of the invention, the invention adopts the following design of a specific detection method:
a method for detecting abnormal driving behaviors based on machine vision comprises the following steps: on hardware model selection 2, an image acquisition module 4 is selected to acquire face information and hand posture information of a driver, heart rate detection and hand posture detection of the driver are completed on the face of the driver through algorithm design 1, and then an abnormal result processing module 15 integrates heart rate results of the heart rate detection and hand posture abnormal detection results, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken.
Further, the algorithm design 1 mainly comprises a plurality of algorithm steps of an image preprocessing algorithm, a skin color model establishing algorithm, an interesting region extracting algorithm and a classifier design; the algorithm steps are respectively realized by an image preprocessing algorithm module 5, a skin color model establishing algorithm module 6, an interesting region extracting algorithm module 7 and a classifier design module 8;
the image preprocessing algorithm module 5 is mainly used for preprocessing an image, and mainly comprises the steps of preprocessing the acquired face image and hand posture image of a driver so as to extract the characteristics of the image, specifically comprising the steps of graying 9, image equalization 10 and image denoising and filtering 11; the skin color model establishing algorithm module 6 is used for researching the skin color clustering characteristics in a specific color space, selecting a skin color sample to establish a skin color model, establishing a YCrCb skin color model based on the non-difference of the human face in the luminance information and the obvious clustering distribution of the human face in the chrominance information, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human face by using a LabVIEW tool function to obtain the skin color area of the human face; the interesting region extraction algorithm module 7 is used for extracting the interesting regions of the human face and the human hand; the method comprises the steps of aiming at the interesting region extraction of a human face, positioning the coordinates of the center point of the human face, calculating the radius of the interesting region, and further calculating the area of the interesting region by utilizing the length-width ratio of a popular face; extracting an interested region aiming at the hand gesture of a driver, firstly, carrying out edge detection on an image by using an operator, then, carrying out least square ellipse fitting to position the position of a steering wheel, and determining the interested region; the classifier design module 8 analyzes the extracted features based on a Bayesian classification model, designs a classifier, trains the acquired images by taking the positions, the movement speeds and the movement angles of the two hands of the driver as movement feature parameters, further establishes a model, classifies the acquired movement features of the driver for driving behaviors, and judges whether the driving behaviors are abnormal or not;
after the face image and the gesture image of the driver are preprocessed through the image preprocessing algorithm module 5, redundant noise is removed, a skin color model of the face and a skin color model of the hand are respectively established through the interested trend extraction algorithm module 7, whether the extraction of the interested area is finished or not is judged, if not, the image is continuously obtained, if the extraction and the analysis of the face characteristic are finished, whether the heart rate is normal or not is judged, if not, drunk driving is judged, and an alarm is given; if the driving behavior is normal, the features of the hands are extracted, the driving behavior with abnormal driving is classified through the classifier design module 8, whether the driving behavior violates rules is judged, if the driving behavior violates the rules, the alarm is given, if the driving behavior does not violate the rules, whether the detection needs to be stopped is judged, if the detection needs to be stopped, the detection is finished, otherwise, the feature extraction is carried out again for judgment.
Further, an image preprocessing algorithm module 5, a skin color model establishing algorithm module 6, an interesting region extracting algorithm module 7 and a classifier design module 8 in the algorithm design 1 are all realized through a software design 3, and the method mainly comprises a system development environment, a development language and a system software design process, and is based on LabVIEW and G language development design.
Further, the graying 9 of the image is to convert the color image of the driver collected by the image collection module into an 8-bit grayscale image, and by means of an image processing tool of vision of LabVIEW, the weights of r, g and b of the image are obtained by adopting a weighted average method; the image equalization 10 is to avoid the influence of ambient light and other factors on the image, so as to further influence the stability of image detection, enhance the contrast of the image through a histogram equalization algorithm, and avoid the loss of image information; the image denoising and filtering 11 is to suppress and remove other information in the image, and a median filtering method is selected to perform filtering and denoising on the equalized image, remove redundant noise points and better retain details of the image.
Further, the heart rate detection is to perform non-contact measurement on the face of the driver based on a photoplethysmography, when abnormal conditions such as drunk driving or heart attack exist in the driver, the skin color and the heart rate of the face change, a face skin color model YCrCb of the driver is established, a feature area is extracted, a heart rate signal QRS is further detected based on a wavelet algorithm to extract and analyze the features of the heart rate signal QRS, Q, R, S points of the signal are detected by means of a LabVIEW advanced signal tool kit, wavelet classification is further performed on the heart rate signal, periodic calculation of R-R points is further performed, and the heart rate is calculated.
Further, the image acquisition module 4 selects a Kinect camera for reading a video image file of a driver, and develops based on a LabVIEW development environment and a G language.
Further, the hand gesture detection of the driver is to acquire images of the hand of the driver through a Kinect camera, and extract and classify gesture features of the hand.
Further, the abnormal result processing module 15 is mainly used for processing the abnormal detection behavior output by the abnormal driving behavior detection system, when the heart rate of the driver is detected to be abnormal and the possibility of drunk driving is high, the driver gives an alarm, and a specific alarm mode can upload the driving condition and the driving position of the driver to a traffic department through a cloud end or other modes so as to facilitate the interference and tracking processing of the behavior by the traffic department; after the abnormal driving behaviors are classified, when the driving behaviors are possibly driven in danger, safe braking is carried out, so that the vehicle is safely stopped.
The invention has the beneficial effects that:
the invention can effectively prevent drunk driving accidents and accidents caused by other abnormal driving, provides guarantee for the driving safety of people, is more beneficial to social safety, and provides effective guarantee for the development of the automobile industry. At present, China only stays on legal support and publicity for drunk driving, but the effect of the Chinese is small, a small number of people still have lucky psychology, traffic accidents are caused, the products can prevent the occurrence of the phenomena, and therefore the popularization of the products has wide market.
The method for comprehensively detecting the heart rate and the hand posture abnormity reduces the misjudgment of the abnormal driving behavior, and is higher in accuracy. The heart rate detection is based on the non-contact measurement of visual acquisition, can detect and monitor the heart rate of a human body, does not have any interference on driving behaviors, and plays a certain auxiliary health detection role at any time on the health of the human body. The hand gesture abnormity detection based on vision effectively detects the abnormal gesture of the hand of the driver in the driving process, and can monitor the behavior in the driving process in time so as to reduce the occurrence of traffic accidents. The design of the abnormal result processing module can further prompt the driver of improper driving behaviors in time, prompt the driver of correction in time, and simultaneously can help the driver to know the health state of the driver to a certain extent, so that the abnormal driving behaviors of the driver can be conveniently recorded and processed by a traffic department, the abnormal driving behaviors can be further reduced, traffic accidents are reduced to a certain extent, and traffic safety and security are maintained.
Drawings
FIG. 1 is a general system layout of the present invention
FIG. 2 is a system workflow diagram of the present invention.
Fig. 3 is a flow chart of the algorithm of the present invention.
FIG. 4 is a software design flow diagram of the present invention.
FIG. 5 is a technical flow diagram of the present invention.
Fig. 6 heart rate algorithm detection flow chart.
Detailed Description
The embodiments of the present invention will be described in detail with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and the scope of the present invention can be clearly and clearly defined.
A system for detecting abnormal driving behaviors based on machine vision mainly comprises three parts, namely algorithm design, hardware type selection and software design.
The key point of hardware type selection lies in the selection of an image acquisition module of the system, and the Kinect sensor is provided with an infrared camera and a depth camera, so that a color image and a depth image can be obtained, and the image information is rich.
The software design refers to a specific software program design flow chart of a system, and software design development is carried out based on a LabVIEW program development environment and a G language.
Fig. 1 shows the overall design of the system of the present invention. The invention relates to an abnormal driving behavior detection system based on machine vision, which mainly comprises an algorithm design 1, a hardware model selection 2 and a software design 3. The hardware model selection 2 comprises an image acquisition module 4 of the system, which is used for acquiring the information of the human face; the software design 3 mainly comprises a system development environment, a development language and a system software design process, and is developed and designed based on LabVIEW and G language; the algorithm design 1 is an algorithm for realizing system functions, and mainly comprises an image preprocessing algorithm module 5, a skin color model establishing algorithm module 6, an interesting region extracting algorithm module 7 and a classifier design module 8.
The image preprocessing algorithm 5 is mainly a preprocessing algorithm for an image, and mainly includes image preprocessing for the acquired face image and hand posture image of the driver, so as to extract the features of the image, specifically including graying 9, image equalization 10 and image denoising and filtering 11.
The graying 9 of the image is to convert the color image of the driver collected by the image collection module into an 8-bit grayscale image, obtain the weight of R, G, B of the image by using a weighted average method by means of an image processing tool of vision of LabVIEW, and calculate the formula of the grayscale value: k is (aR + bG + cB)/3, and in the parameters of the examples, a is 0.25, b is 0.64, and c is 0.11.
The equalization 10 of the image is to avoid that the image is affected by factors such as ambient light and further affects the stability of image detection, and enhance the contrast of the image through a histogram equalization algorithm, that is, increase the brightness information of the image, and avoid the loss of the image information.
The image denoising and filtering 11 is to suppress and remove other information in the image, and a median filtering method is selected to perform filtering and denoising on the equalized image, remove redundant noise points and better retain the details of the image. Selecting one with xijA 5 × 5 rectangular region as a center
Filtered output
The skin color model establishing algorithm 6 is to research the skin color clustering characteristics in a specific color space, select skin color samples to establish a skin color model, establish a YCrCb (Y is a gray level value, Cr is a hue, Cb is a saturation) skin color model based on the no difference of the human face in the brightness information and the obvious clustering distribution on the chrominance information, research a model area of a Cr-Cb plane, reconstruct a binary image of the human face by using a LabVIEW tool function, and obtain the skin color area of the human face, wherein the color space conversion formula is as follows:
based on data analysis, the characteristic region of the skin color model resembles an ellipse (ellipse constructed about x, y coordinates), and the skin color model establishes the formula:
the region-of-interest extraction algorithm 7 is a region-of-interest for extracting a human face and a human hand. The method comprises the steps of aiming at the interesting region extraction of a human face, positioning the coordinates of the center point of the human face, calculating the radius of the interesting region, and further calculating the area of the interesting region by utilizing the length-width ratio of a popular face; extracting an interested area aiming at the hand gesture of a driver, firstly, carrying out edge detection on an image by using a Canny operator, then, carrying out least square ellipse fitting to position the position of a steering wheel, and determining the interested area.
Extracting coordinate points (x) for acquiring planar projection and vertical projection of human face image in interested region of human face1,y1) And (x)2,y2) Further calculating the coordinates of the center point
I.e. the radius of the region of interestThereby determining the region of interest.
FIG. 2 is a flow chart of the system operation of the present invention. Firstly, a Kinect camera of an image acquisition module 1 reads a video file of a driver, development is carried out based on a LabVIEW development environment and a G language, firstly, an abnormal driving behavior detection system detects face information of the driver, judges whether the heart rate is normal or not, outputs an abnormal behavior result if the heart rate is abnormal, gives an alarm, carries out abnormal detection on the driving behavior if the heart rate is normal, gives an alarm if the heart rate is abnormal, and continues continuous detection if the heart rate is normal.
FIG. 3 is a flow chart of the algorithm of the present invention. Firstly, processing an image by using an image preprocessing algorithm module 5 for an acquired video file, specifically comprising image graying 8, image equalization 9, a wavelet filtering algorithm 10 and edge detection 11, after the image preprocessing is completed, analyzing the characteristics of the image, specifically adopting ellipse detection, further adopting an interested region extraction algorithm module 7 to respectively extract an interested region and a hand characteristic region of the face of a driver, adopting a skin color model algorithm building module 6 to respectively build a skin color model YCrCb of the face and a self-adaptive skin color characteristic model of the hand, further adopting a heart rate detection algorithm to calculate the heart rate by utilizing the skin color model of the face, judging whether the heart rate is abnormal or not, if so, possibly having drunk driving behavior, and further processing an abnormal result; and classifying the driving behaviors by using the self-adaptive skin color characteristic model of the hand and adopting a classifier design module 8, judging whether the driving behaviors are abnormal or not, and if so, processing an abnormal result.
FIG. 4 is a flow chart of the software design of the present invention. Firstly, a machine vision-based abnormal driving behavior detection system preprocesses a face image and a posture image of a driver through an image acquisition module 4, removes redundant noise, establishes a skin color model of the face and a skin color model of hands through an interested trend extraction algorithm module 7, judges whether extraction of an interested region is finished, if not, continuously acquires the image, extracts and analyzes face features, judges whether a heart rate is normal, and if not, judges drunk driving and gives an alarm; if the driving behavior is normal, the features of the hands are extracted, the driving behavior with abnormal driving is classified through the classifier design module 8, whether the driving behavior violates rules is judged, if the driving behavior violates the rules, the alarm is given, if the driving behavior does not violate the rules, whether the detection needs to be stopped is judged, if the detection needs to be stopped, the detection is finished, otherwise, the feature extraction is carried out again for judgment.
FIG. 5 illustrates the exception result handling module 15 of the present invention. The abnormal result processing mainly comprises the steps of processing abnormal detection behaviors output by the abnormal driving behavior detection system, giving an alarm when the abnormal heart rate of the driver is detected to be possible to drive drunk, and uploading the driving condition and the driving position of the driver to a traffic department in a specific alarm mode through a cloud end or other modes so as to facilitate the interference and tracking processing of the behaviors by the traffic department; after the abnormal driving behaviors are classified, when the driving behaviors are possibly driven in danger, safe braking is carried out, so that the vehicle is safely stopped.
In the invention, the heart rate detection is based on the acquisition of face information of a human face by a camera, the software is based on LabVIEW software for programming, and a software flow chart is shown in figure 6. The hand gesture detection of the driver is to acquire images of the hand of the driver through a Kinect camera and extract and classify gesture features of the hand. The abnormal result processing module 15 specifically comprises a language prompt warning part and a remote alarm part, wherein the abnormal result is designed based on LabVIEW software G language, and the abnormal driving behavior is reminded in a voice broadcast mode.
The invention provides a design method of an abnormal driving behavior detection system based on machine vision, which aims at detecting abnormal driving behaviors, can reduce the occurrence of the abnormal driving behaviors to a certain extent and improve traffic conditions.
In summary, the design method of the abnormal driving behavior detection system based on the machine vision is used for detecting the abnormal driving behavior of the driver in the driving process so as to take effective measures, and mainly comprises three major parts, namely algorithm design, hardware type selection and software design. Algorithms mainly researched by algorithm design comprise an image preprocessing algorithm, an interesting region extraction algorithm, a skin color model building algorithm and classifier design research. The original video image obtained by the camera contains a complex background and a plurality of image noises which cause interference to video detection, and the acquired image needs to be preprocessed in order to improve the identification and detection precision of abnormal driving behaviors. The skin color clustering characteristics in a specific color space are researched, a skin color sample is selected to establish a skin color model YCrCb, the coordinates of the center point of a human face are positioned, the radius of an interested region is calculated, the area of the interested region of the face is further calculated by utilizing the length-width ratio of the popular face, and the interested region of the hand gesture of a driver is detected and positioned by utilizing a direct least square ellipse fitting algorithm to the region of the image of the steering wheel. The method comprises the steps of analyzing human face characteristic information and hand posture characteristic information to finish heart rate detection and hand posture detection, integrating two detection results, classifying driving behaviors of the obtained driver motion characteristics through a design classifier, recognizing abnormal driving behaviors, and processing abnormal results through an abnormal result processing module.
Claims (8)
1. A method for detecting abnormal driving behaviors based on machine vision is characterized in that: the method comprises the following steps: on the hardware model selection (2), the image acquisition module (4) is selected to acquire face information and hand posture information of a driver, then heart rate detection and hand posture detection of the driver are completed on the face of the driver through algorithm design (1), and then the abnormal result processing module (15) integrates heart rate results of the heart rate detection and hand posture abnormal detection results, so that abnormal behavior detection of the driver is realized, and corresponding measures are taken.
2. The abnormal driving behavior detection method based on machine vision according to claim 1, characterized in that: the algorithm design (1) mainly comprises the steps of an image preprocessing algorithm, a skin color model building algorithm, an interesting region extraction algorithm and a classifier design; the algorithm steps are respectively realized by an image preprocessing algorithm module (5), a skin color model establishing algorithm module (6), an interesting region extracting algorithm module (7) and a classifier design module (8);
the image preprocessing algorithm module (5) is mainly used for preprocessing images, and mainly comprises the steps of preprocessing the acquired face images and hand posture images of the driver so as to extract the characteristics of the images, and specifically comprises the steps of graying (9) the images, equalizing (10) the images and denoising and filtering (11) the images; the skin color model establishing algorithm module (6) is used for researching the skin color clustering characteristics in a specific color space, selecting skin color samples to establish a skin color model, establishing a YCrCb skin color model based on the non-difference of the human face in brightness information and the obvious clustering distribution of the human face in chrominance information, researching a model area of a Cr-Cb plane, and reconstructing a binary image of the human face by using a LabVIEW tool function to obtain the skin color area of the human face; the interesting region extraction algorithm module (7) is used for extracting the interesting regions of the human face and the human hand; the method comprises the steps of aiming at the interesting region extraction of a human face, positioning the coordinates of the center point of the human face, calculating the radius of the interesting region, and further calculating the area of the interesting region by utilizing the length-width ratio of a popular face; extracting an interested region aiming at the hand gesture of a driver, firstly, carrying out edge detection on an image by using an operator, then, carrying out least square ellipse fitting to position the position of a steering wheel, and determining the interested region; the classifier design module (8) analyzes the extracted features based on a Bayesian classification model, designs a classifier, trains the acquired images by taking the positions, the movement speeds and the movement angles of the two hands of the driver as movement feature parameters, further establishes a model, classifies the acquired movement features of the driver for driving behaviors, and judges whether the driving behaviors are abnormal or not;
after the face image and the gesture image of the driver are preprocessed through the image preprocessing algorithm module (5), redundant noise is removed, a skin color model of the face and a skin color model of the hand are respectively established through the interested trend extraction algorithm module (7), whether the extraction of an interested area is finished or not is judged, if not, the image is continuously obtained, if so, the face feature is extracted and analyzed, whether the heart rate is normal or not is judged, if not, drunk driving is judged, and an alarm is given; if the driving behavior is normal, the features of the hands are extracted, the abnormal driving behavior is classified through a classifier design module (8), whether the driving behavior violates rules is judged, if the driving behavior violates the rules, the alarm is given, if the driving behavior does not violate the rules, whether the detection needs to be stopped is judged, if the detection needs to be stopped, the detection is ended, otherwise, the feature extraction is carried out again for judgment.
3. The abnormal driving behavior detection method based on machine vision according to claim 2, characterized in that: the image preprocessing algorithm module (5), the skin color model establishing algorithm module (6), the region-of-interest extracting algorithm module (7) and the classifier design module (8) in the algorithm design (1) are all realized through a software design (3), and mainly comprise a system development environment, a development language and a system software design process, and are developed and designed based on LabVIEW and G language.
4. The abnormal driving behavior detection method based on machine vision according to claim 2, characterized in that: the graying (9) of the image is to convert the color image of the driver acquired by the image acquisition module into an 8-bit grayscale image, and by means of an image processing tool of vision of LabVIEW, the weights of r, g and b of the image are acquired by adopting a weighted average method; the image equalization (10) is used for avoiding the influence of factors such as ambient light on the image and further influencing the stability of image detection, and the contrast of the image is enhanced through a histogram equalization algorithm to avoid the loss of image information; the image denoising and filtering (11) is used for inhibiting and removing other information in the image, a median filtering method is selected for carrying out filtering and denoising on the equalized image, redundant noise points are removed, and the details of the image are well reserved.
5. The abnormal driving behavior detection method based on machine vision according to claim 1, characterized in that: the heart rate detection is based on a photoelectric volume pulse wave notation method to carry out non-contact measurement on the face of a driver, when abnormal conditions such as drunk driving or heart attack exist in the driver, the skin color and the heart rate of the face are changed, a face skin color model YCrCb of the driver is established, a feature area is extracted, a heart rate signal QRS is further detected based on a wavelet algorithm to extract and analyze the features of the face skin color model YCrCb, the heart rate signal QRS is further subjected to wavelet classification by means of detecting Q, R, S points of the signal by an advanced signal tool kit of LabVIEW, the period calculation of the R-R points is further carried out, and the heart rate is calculated.
6. The abnormal driving behavior detection method based on machine vision according to claim 1, characterized in that: the image acquisition module (4) selects a Kinect camera for reading a video image file of a driver, and development is carried out based on a LabVIEW development environment and a G language.
7. The abnormal driving behavior detection method based on machine vision according to claim 6, characterized in that: the hand gesture detection of the driver is to acquire images of the hand of the driver through a Kinect camera and extract gesture features of the hand so as to classify the hand.
8. The abnormal driving behavior detection method based on machine vision according to claim 1, characterized in that: the abnormal result processing module (15) is mainly used for processing abnormal detection behaviors output by the abnormal driving behavior detection system, when the abnormal heart rate of the driver is detected to be abnormal and the possibility of drunk driving is high, the driver gives an alarm, and a specific alarm mode can upload the driving condition and the driving position of the driver to a traffic department through a cloud end or other modes so as to facilitate the interference and tracking processing of the behavior by the traffic department; after the abnormal driving behaviors are classified, when the driving behaviors are possibly driven in danger, safe braking is carried out, so that the vehicle is safely stopped.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899783.4A CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910899783.4A CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110751051A true CN110751051A (en) | 2020-02-04 |
| CN110751051B CN110751051B (en) | 2024-03-19 |
Family
ID=69276902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910899783.4A Active CN110751051B (en) | 2019-09-23 | 2019-09-23 | Abnormal driving behavior detection method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751051B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310730A (en) * | 2020-03-17 | 2020-06-19 | 扬州航盛科技有限公司 | Driving behavior early warning system based on facial expressions |
| CN111637610A (en) * | 2020-06-24 | 2020-09-08 | 山东建筑大学 | Indoor environment health degree adjusting method and system based on machine vision |
| CN111666818A (en) * | 2020-05-09 | 2020-09-15 | 大连理工大学 | Driver abnormal posture detection method |
| CN111914707A (en) * | 2020-07-22 | 2020-11-10 | 上海大学 | System and method for detecting drunkenness behavior |
| WO2021218696A1 (en) * | 2020-04-27 | 2021-11-04 | 华为技术有限公司 | Skin color detection method and apparatus, terminal and storage medium |
| CN113792663A (en) * | 2021-09-15 | 2021-12-14 | 东北大学 | Detection method and device for drunk driving and fatigue driving of driver and storage medium |
| CN113879317A (en) * | 2020-07-02 | 2022-01-04 | 丰田自动车株式会社 | Driver monitoring device |
| CN114469024A (en) * | 2021-12-23 | 2022-05-13 | 广东建采网科技有限公司 | Construction worker safety early warning method and system based on smart band |
| CN114663863A (en) * | 2022-02-24 | 2022-06-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN117495384A (en) * | 2023-11-07 | 2024-02-02 | 广州准捷电子科技有限公司 | KTV face brushing payment method based on AI face recognition technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN106651910A (en) * | 2016-11-17 | 2017-05-10 | 北京蓝天多维科技有限公司 | Intelligent image analysis method and alarm system for abnormal driver behavior state |
| CN109034132A (en) * | 2018-09-03 | 2018-12-18 | 深圳市尼欧科技有限公司 | A kind of detection method driving abnormal behaviour |
-
2019
- 2019-09-23 CN CN201910899783.4A patent/CN110751051B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN106651910A (en) * | 2016-11-17 | 2017-05-10 | 北京蓝天多维科技有限公司 | Intelligent image analysis method and alarm system for abnormal driver behavior state |
| CN109034132A (en) * | 2018-09-03 | 2018-12-18 | 深圳市尼欧科技有限公司 | A kind of detection method driving abnormal behaviour |
Non-Patent Citations (2)
| Title |
|---|
| 曾鸿军;沈燕飞;王毅;: "基于感兴趣区域的头像视频前处理方法", 计算机工程与应用, no. 06, 31 December 2017 (2017-12-31) * |
| 邸巍;王荣本;: "基于红外光源的驾驶员眼睛特征提取", 交通信息与安全, no. 01, 15 February 2009 (2009-02-15) * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310730A (en) * | 2020-03-17 | 2020-06-19 | 扬州航盛科技有限公司 | Driving behavior early warning system based on facial expressions |
| WO2021218696A1 (en) * | 2020-04-27 | 2021-11-04 | 华为技术有限公司 | Skin color detection method and apparatus, terminal and storage medium |
| CN111666818A (en) * | 2020-05-09 | 2020-09-15 | 大连理工大学 | Driver abnormal posture detection method |
| CN111666818B (en) * | 2020-05-09 | 2023-06-16 | 大连理工大学 | Driver abnormal posture detection method |
| CN111637610A (en) * | 2020-06-24 | 2020-09-08 | 山东建筑大学 | Indoor environment health degree adjusting method and system based on machine vision |
| CN113879317A (en) * | 2020-07-02 | 2022-01-04 | 丰田自动车株式会社 | Driver monitoring device |
| CN113879317B (en) * | 2020-07-02 | 2023-08-22 | 丰田自动车株式会社 | driver monitoring device |
| CN111914707A (en) * | 2020-07-22 | 2020-11-10 | 上海大学 | System and method for detecting drunkenness behavior |
| CN113792663A (en) * | 2021-09-15 | 2021-12-14 | 东北大学 | Detection method and device for drunk driving and fatigue driving of driver and storage medium |
| CN113792663B (en) * | 2021-09-15 | 2024-05-14 | 东北大学 | Method, device and storage medium for detecting drunk driving and fatigue driving of driver |
| CN114469024A (en) * | 2021-12-23 | 2022-05-13 | 广东建采网科技有限公司 | Construction worker safety early warning method and system based on smart band |
| CN114469024B (en) * | 2021-12-23 | 2023-12-22 | 广东智云城建科技有限公司 | Intelligent bracelet-based construction worker safety early warning method and system |
| CN114663863A (en) * | 2022-02-24 | 2022-06-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN117495384A (en) * | 2023-11-07 | 2024-02-02 | 广州准捷电子科技有限公司 | KTV face brushing payment method based on AI face recognition technology |
| CN117495384B (en) * | 2023-11-07 | 2024-04-26 | 广州准捷电子科技有限公司 | KTV face brushing payment method based on AI face recognition technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110751051B (en) | 2024-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110751051B (en) | Abnormal driving behavior detection method based on machine vision | |
| CN107292251B (en) | Driver fatigue detection method and system based on human eye state | |
| CN106846734B (en) | A kind of fatigue driving detection device and method | |
| CN101950355B (en) | Method for detecting fatigue state of driver based on digital video | |
| Omidyeganeh et al. | Intelligent driver drowsiness detection through fusion of yawning and eye closure | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| CN102289660B (en) | Method for detecting illegal driving behavior based on hand gesture tracking | |
| CN100462047C (en) | Safe driving auxiliary device based on omnidirectional computer vision | |
| CN110728241A (en) | Driver fatigue detection method based on deep learning multi-feature fusion | |
| US9646215B2 (en) | Eye part detection apparatus | |
| EP1589485A2 (en) | Object tracking and eye state identification method | |
| CN102938058A (en) | Method and system for video driving intelligent perception and facing safe city | |
| CN110443102B (en) | Living body face detection method and device | |
| CN104013414A (en) | Driver fatigue detecting system based on smart mobile phone | |
| CN102054163A (en) | Method for testing driver fatigue based on monocular vision | |
| CN111553214B (en) | Method and system for detecting smoking behavior of driver | |
| Paone et al. | Baseline face detection, head pose estimation, and coarse direction detection for facial data in the SHRP2 naturalistic driving study | |
| CN104881956A (en) | Fatigue driving early warning system | |
| CN107563346A (en) | One kind realizes that driver fatigue sentences method for distinguishing based on eye image processing | |
| CN111832461A (en) | Non-motor vehicle riding personnel helmet wearing detection method based on video stream | |
| CN112241647B (en) | Dangerous driving behavior early warning device and method based on depth camera | |
| CN112401857A (en) | Driver drunk driving detection method | |
| CN106446822A (en) | Blink detection method based on circle fitting | |
| CN110992629A (en) | Method for detecting static human body based on video monitoring | |
| Liu et al. | Design and implementation of multimodal fatigue detection system combining eye and yawn information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |