CN103366506A - Device and method for automatically monitoring telephone call behavior of driver when driving - Google Patents

Abstract

The invention relates to a device and a method for automatically monitoring a telephone call behavior of a driver when driving and belongs to the fields of intelligent traffic and aided driving. The device comprises an image acquisition device, a calculating device, a warning device and a wireless transmission device. The method comprises the following steps of firstly acquiring a head and nearby area image of the driver, secondly, obtaining the positions of the face and both hands of the driver in the image by skin color detection, and classifying by a support vector machine to determine whether the driver makes a telephone call or not; warning the driver if the driver makes the telephone call; uploading the image that the driver makes the telephone call to the monitoring center of the road transport bureau through a wireless network if the warning is useless to be used as an evidence that the driver breaks the law. According to the device and the method for automatically monitoring the telephone call behavior of the driver when driving disclosed by the invention, the unlawful act of the driver can be effectively monitored, the working strength of law enforcement personnel is relieved, the working efficiency is improved, and the occurrence rate of traffic accidents is reduced.

Description

Automatic monitoring device and method for mobile phone call receiving behavior of driver in driving process

Technical Field

The invention relates to the field of intelligent transportation and the field of auxiliary driving.

Background

With the popularization of the use of the mobile phone, the behavior of dialing or answering the mobile phone while driving is seen everywhere. Because a driver usually operates a steering wheel with one hand when connecting and striking a mobile phone, the vehicle cannot be well controlled; more importantly, the driver's attention can be distracted by the connecting and tripping device, and once an emergency happens, the driver often has no time to respond, and traffic accidents are easily caused. Research shows that when the mobile phone is used for driving, the reaction speed of the brain is 30% slower than that of drunk driving, and the risk of traffic accidents caused by using the mobile phone is more than 4 times higher than that of normal driving. 70% of fatal traffic accidents are caused by inattention of drivers, and the mobile phone is one of the main causes of inattention. In view of the harm of driving and connecting with a mobile phone to traffic safety, about 50 countries and regions in the world have been in the clear to prohibit drivers from connecting with the mobile phone during driving. The latest 'implementation regulations of road traffic safety laws' in China also clearly stipulate that '3 points are paid for when a driver dials or answers a mobile phone in the driving process, and the fine is 100 Yuan'.

However, since this behavior occurs in a vehicle traveling at high speed, it is difficult to capture the behavior outside the vehicle. At present, there are three methods for monitoring the behavior of a driver to turn on or off a mobile phone. The first is that law enforcement personnel are on site, but law enforcement personnel are generally only present in a few areas, such as intersections, and therefore this approach has significant limitations. The second is the monitoring of the camera outside the vehicle. In this way, the behavior of the driver using the mobile phone when driving can be captured, and although the monitoring range is expanded compared with the first method, the monitoring range is limited only to the range that can be monitored by the camera, and the behavior cannot be monitored in most areas. In addition, the method also needs the monitoring personnel to stare at the monitor to judge the illegal behaviors, and has the advantages of high working strength, low efficiency and lower efficiency at night. The third is that a directional antenna is arranged in a fixed area to monitor a mobile phone signal from a running vehicle, and if the mobile phone signal is a signal in a call, an imaging device is started to carry out on-site evidence collection. Although the method can play the function of automatic monitoring, all vehicles which detect the mobile phone signals in conversation can be shot, and at the moment, a driver does not necessarily make a call, and other people in the vehicles can also make a conversation, so that a plurality of false detection situations exist. In the latter two methods, the camera is installed outside the vehicle, so that sometimes the condition that the shot picture cannot be clearly seen in the vehicle due to weather or shielding and the like occurs, and missing detection and false detection occur.

Based on the reasons, the behavior of connecting and calling the mobile phone during driving cannot be effectively monitored, so that the behavior is visible everywhere, and great hidden danger is caused to traffic safety. At present, no device for automatically monitoring the behavior of connecting and connecting a mobile phone during driving of a driver based on a camera in a vehicle exists.

Disclosure of Invention

In order to solve the problems, the invention provides an automatic monitoring device and method for the behavior of connecting and disconnecting a mobile phone during driving of a driver. The method comprises the steps of collecting images of the head of a driver and the area nearby the head of the driver through an in-vehicle camera, automatically identifying the behavior of the driver for getting on or off the mobile phone during driving by using an image processing technology and a mode identification method, taking an illegal picture of the illegal picture, and uploading the illegal picture to a monitoring center of a traffic bureau in an internet manner to serve as an illegal evidence. The invention provides a method for automatically monitoring the behavior of a driver for connecting and disconnecting a mobile phone during driving. The method can greatly reduce the law enforcement intensity, improve the law enforcement efficiency and effectively avoid traffic accidents caused by the behavior of dialing or answering the mobile phone by a driver in the driving process. In addition, the camera is arranged on the front windshield in the automobile, so that the behavior of a driver can be clearly shot, the assistance of other electronic devices is not needed, and the normal driving of the driver is not influenced.

An automatic monitoring device for the behavior of a driver to answer or make a call to a mobile phone during driving comprises: the device comprises an image acquisition device, a calculation device, a warning device and a wireless transmission device; wherein,

the image acquisition device acquires images of the head and the nearby area of the driver by adopting a camera fixed on a front windshield in the vehicle and transmits the images to the computing device;

the calculating device obtains the positions of the face and hands of the driver in the image through skin color detection, calculates the gradient histogram characteristics of the face and hands and the lip shape characteristics of the driver, obtains the characteristics of the driver in a mobile phone using state and a mobile phone not using state, and classifies the characteristics through a support vector machine so as to determine whether the driver takes a call or not in the driving process and transmit the information of whether the driver takes the call or not to the warning device;

the warning device sends a warning to the driver after receiving the information that the driver is calling the mobile phone sent by the computing device, and sends a signal to the wireless transmission device if the information that the driver is calling the mobile phone sent by the computing device is still received after the warning is sent;

the wireless transmission device uploads the image of the driver connecting the mobile phone to the monitoring center of the traffic bureau through the wireless network as the evidence of the law violation.

An automatic monitoring method for a driver to take a mobile phone call during driving comprises the following steps:

the method comprises the following steps: when the vehicle engine is started, the monitoring device is automatically started, the device performs self-checking operation, whether a face exists in a shot video picture or not and whether the face is in a specific area or not (the device is prevented from being moved or shielded by a driver) are checked, and if the conditions are met, the steering wheel is unlocked; otherwise the steering wheel is in a locked state and the driver is unable to drive the vehicle.

Step two: after the device is normally started, monitoring the head area image is started, and if the situation that a driver takes a cellphone receiving posture for more than a certain time and simultaneously accompanies lip movement is found in the running process of the vehicle, the driver is judged to take the cellphone receiving posture.

Step three: the alarm device gives an alarm, if the driver still does not stop using the mobile phone after the alarm is sent out for 3 seconds, the camera uploads an image of the mobile phone used by the driver to a monitoring center of a traffic bureau through a network to serve as an evidence of law violation, and law enforcement personnel can conveniently handle the law violation.

Step four: the device is started only during the running of the vehicle, and when the vehicle stops, the device automatically stops working.

The invention has the advantages that:

1. the illegal behavior of the driver for getting on or off the mobile phone during driving can be monitored in real time.

2. The violation records can be automatically transmitted to the monitoring center, the workload of law enforcement personnel is reduced, and the working efficiency is improved.

3. The device is automatic operation, does not need driver's operation, and is simple easy-to-use.

4. The device is installed forcibly, can ensure the realization of the function, can effectively monitor the illegal behaviors of the driver, and reduces the illegal behaviors, thereby reducing the occurrence rate of traffic accidents.

5. The camera is installed in the car, does not receive external environment like weather, shelters from the influence of factors such as, can shoot clear image.

Drawings

FIG. 1 is a flow chart of monitoring, warning and automatic evidence collection of a driver's behavior of connecting and disconnecting a mobile phone while driving;

fig. 2 is a diagram of rectangular features for driver face detection in the present invention.

Detailed Description

The core thought of the invention is as follows: when a driver connects and beats the mobile phone, the driver is sure to have the action of holding the mobile phone close to the ear part and accompanied with the lip movement. And judging whether the behaviors are generated or not through video analysis so as to judge whether the behaviors are connected or disconnected with the mobile phone. Once the illegal behaviors are determined, the violation images are immediately transmitted to the monitoring center under the condition that the warning is invalid, and a violation record is formed, so that law enforcement personnel can collect evidence conveniently.

The automatic monitoring device of the invention comprises: the device comprises an image acquisition device, a calculation device, a warning device and a wireless transmission device; wherein,

the image acquisition device adopts a camera fixed on a front windshield in the automobile to acquire images of the head of the driver and the nearby area;

the calculating device obtains the positions of the face and hands of the driver in the image through skin color detection, calculates the gradient histogram characteristics of the face and hands and the lip shape characteristics of the driver, obtains the characteristics of the driver in a mobile phone using state and a mobile phone not using state, and classifies the characteristics through a support vector machine so as to determine whether the driver takes a call or not in the driving process;

the warning device gives a warning to the driver after the computing device determines that the driver is answering the mobile phone;

the wireless transmission device uploads the image of the driver connecting the mobile phone to the monitoring center of the traffic bureau through the wireless network as the evidence of the law violation.

The invention discloses a flow chart of monitoring, warning and automatic evidence obtaining of a behavior of a driver who makes a call during driving based on a camera in a vehicle, which is shown in figure 1 and comprises the following specific implementation steps:

the method comprises the following steps: when the vehicle engine is started, the monitoring device is automatically started, the device performs self-checking operation, whether a face exists in a shot video picture or not and whether the face is in a specific area or not (the device is prevented from being moved or shielded by a driver) are checked, and if the conditions are met, the steering wheel is unlocked; otherwise the steering wheel is in a locked state and the driver is unable to drive the vehicle.

And positioning the face of the driver by adopting a face detection method based on AdaBoost. According to the method, a weak classifier is constructed based on rectangular features, a small number of key features are selected by an AdaBoost method, the corresponding weak classifier is subjected to weighted summation to construct a strong classifier, and the strong classifier is used as a final classifier for face detection. Where each rectangular feature is composed of 2-3 rectangles, as shown in fig. 2, whose value is the sum of the pixel values in the white rectangle minus the sum of the pixel values in the black rectangle.

Each weak classifier is composed of a rectangular feature, and the training process of the strong classifier is as follows:

(1) given training data (x)₁,y₁),...(x_n,y_n) Wherein y is_i=0 for negative sample, y_i=1 represents a positive sample, and n is the number of training samples.

(2) Initialization weight, y_iWhen =0

y_iWhen =1

And m and l are the numbers of the negative samples and the positive samples respectively.

(3) For T =1,.., T:

A. normalized weight $w_{t,i}\←\frac{w_{t,i}}{{\mathrm{\Σ}}_{j=1}^n{w}_{t,j}}$

B. For each feature j, training weak classifier h_jIf the error of the classifier is epsilon_j=∑_iw_i|h_j(x_i)-y_i|

C. Select classifier h with the smallest error_t

D. Updating the weight

Wherein e_i=0 if x_iIs correctly classified, otherwise e_i=1， ${\mathrm{\β}}_t=\frac{{\mathrm{\ϵ}}_t}{1-{\mathrm{\ϵ}}_t}$

(4) The final strong classifier is

$h\left(x\right)=\left\{\begin{array}{cc}1& {\mathrm{\Σ}}_{t=1}^T{\mathrm{\α}}_t{h}_t\left(x\right)\≥\frac{1}{2}{\mathrm{\Σ}}_{t=1}^T{\mathrm{\α}}_t\\ 0& \mathrm{otherwise}\end{array}\right.,$ Wherein ${\mathrm{\α}}_t=\log \frac{1}{{\mathrm{\β}}_t}$

And after obtaining the strong classifier, scanning the image by using the strong classifier to obtain the face position of the driver, and if the face is in the central area of the image, normally operating the device. The self-checking is successful, the steering wheel is unlocked, and the driver can start the vehicle.

Step two: after the device is normally started, the head area image is automatically monitored in the running process of the vehicle, and if the situation that the posture of a driver for connecting and striking the mobile phone exceeds a certain time and is accompanied with lip movement, the mobile phone is judged to be used.

The driver has a great difference between the normal driving state and the state that the hand is placed on the ear when the mobile phone is used. Therefore, whether the driver is on the mobile phone or not is judged by adopting a mode of training the SVM classifier. Because the lip state has obvious difference when the driver speaks and does not speak, the mode of training the SVM classifier is also adopted for judging whether the lip of the driver acts or not. And combining the two clues, if the posture of the driver for receiving and playing the mobile phone exceeds a certain time and the lip action is accompanied in the posture, judging that the driver is receiving and playing the mobile phone.

1) HOG feature extraction for driver head region

The features used for training the classifier are all HOG features. The HOG, namely histogram of oriented gradients, is a feature descriptor for target detection, and calculates the number of directional gradients that occur locally in an image. The method generates a histogram of directional gradients for each cell by dividing the entire image into small connected regions (called cells), the combination of which represents a descriptor of the detected object. To improve accuracy, all cells in a larger area (called a block) in the image can be normalized. Better illumination/shadow invariance is achieved by the normalization process.

The HOG feature extraction method comprises the following steps:

(1) the image is converted into a grayscale image.

(2) The image is divided into small cells (eg: 2 x 2).

(3) The gradient of each pixel in each cell is calculated.

Gradient size: $R(x,y)=\sqrt{{(I(x+1,y)-I(x-1,y))}^2+{(I(x,y+1)-I(x,y-1))}^2}$

gradient direction: ang (x, y) = arctan ((I (x +1, y) -I (x-1, y))/(I (x, y +1) -I (x, y-1)))

(4) All cells in each block are normalized. Normalization was performed using L2-norm:

L2-norm: $v\→v/\sqrt{{\left|\right|v\left|\right|}_2^2+{\mathrm{\ϵ}}^2}$

(5) HOG features are generated. And calculating HOG features in all blocks in the detection window, and connecting the HOG features into a final feature vector for classification.

2) Classifying HOG features using SVM

After the HOG characteristics of the head area of the driver are extracted, the HOG characteristics are used as the input of an SVM classifier to classify so as to judge the head posture of the driver.

The SVM (Support Vector Machine) is a supervised learning method and has a plurality of specific advantages in solving small sample, nonlinear and high-dimensional pattern recognition. The SVM constructs an optimal segmentation hyperplane in a feature space based on a structural risk minimization theory, so that a classifier is globally optimized, and the expected risk in the whole sample space meets a certain upper bound with a certain probability.

The mathematical form of the hyperplane is assumed to be: w · x-b = 0. Where x is a point on the hyperplane and w is a vector perpendicular to the hyperplane. (x)⁽ⁱ⁾,y⁽ⁱ⁾) Are training samples. x is the number of⁽ⁱ⁾Inputting features of the training samples; y is⁽ⁱ⁾Is its corresponding tag, wherein y⁽ⁱ⁾E {1, -1 }. Then, in the case of linear divisibility, the optimal hyperplane can be obtained by solving the following quadratic optimization problem:

$\min \frac{1}{2}{\left|\right|w\left|\right|}^2$

and the constraint conditions are met: y is⁽ⁱ⁾(w^Tx⁽ⁱ⁾+b)≥1,i=1,2,...,m

3) Determining the lip status of the driver

The process of judging the lip state of the driver is as follows:

1. face position detection through Adaboost learning algorithm

2. And coarsely positioning the position of the lips. According to the priori knowledge, the lips are located in the lower half of the face.

3. Lip shape and position determination. Training skin color and lip color, respectively determining the color range, and separating the skin color and the lip color by using a Fisher linear classifier to segment out a mouth region.

4. And eliminating noise. The small communicated area is eliminated by a method of firstly corroding and then expanding the mouth area divided in the previous step, so that the purpose of removing noise is achieved. The noise-removed region is the mouth region.

5. Mouth contour determination. And horizontally and vertically projecting the mouth region detected in the previous step to obtain a mouth outline.

6. And (5) feature extraction. Calculating the maximum width W of the mouth region from the mouth contour_maxMaximum height H_maxAnd the height H between the upper lip and the lower lip_m. These values are significantly different when the person is speaking and not speaking, and are therefore characterized. F = (W)_amx,H_max,H_m).

7. And training an SVM classifier to classify the lip features so as to judge the mouth state of the driver. 4) Judging whether the driver is connecting or not

When the HOG characteristics of the head area of the driver are classified by using the SVM, the posture of the driver belonging to the mobile phone call receiving is judged, meanwhile, the lip state of the driver is judged to be a speaking state, and when the duration time of the situation exceeds a given threshold value, the driver is judged to be calling the mobile phone call receiving.

Step three: the alarm device gives an alarm, and if the driver still does not stop using the mobile phone after the alarm is given for 3 seconds, the wireless transmission device uploads the image of the mobile phone used by the driver to a monitoring center of a law enforcement officer through a network as evidence of law violation. And the contents of the illegal act, time and the like are recorded, so that law enforcement personnel can conveniently process the illegal act.

Step four: the device is started only during the running of the vehicle, and when the vehicle stops, the device automatically stops working.

Claims (2)

1. The utility model provides a driver connects automatic monitoring device who makes a call to cell-phone action on the way that drives a vehicle which characterized in that: the automatic monitoring device comprises an image acquisition device, a computing device, a warning device and a wireless transmission device; wherein,

the image acquisition device acquires images of the head and the nearby area of the driver by adopting a camera fixed on a front windshield in the vehicle and transmits the images to the computing device;

the computing device obtains the characteristics of the driver in the state of connecting and calling the mobile phone and the state of not connecting and calling the mobile phone through training samples, and can classify each image acquired by the image acquisition device, so that whether the driver connects and calls the mobile phone in the driving process can be further determined, and the information of whether the driver connects and calls the mobile phone is transmitted to the warning device;

the warning device sends a warning to the driver after receiving the information that the driver is calling the mobile phone sent by the computing device, and sends a signal to the wireless transmission device if the information that the driver is calling the mobile phone sent by the computing device is still received after the warning is sent;

the wireless transmission device uploads the image of the driver connecting the mobile phone to the monitoring center of the traffic bureau through the wireless network as the evidence of the law violation.

2. An automatic monitoring method for a driver to take a mobile phone call during driving is characterized in that:

the method comprises the following steps: when the vehicle engine is started, the monitoring device is automatically started, the device performs self-checking operation, whether a face exists in a shot video picture or not and whether the face is in a specific area or not are checked, and if the conditions are met, the steering wheel is unlocked; otherwise, the steering wheel is in a locked state, and the driver can not drive the vehicle;

step two: after the device is normally started, monitoring of the head region image is started. The calculating device obtains the positions of the face and the hands of the driver in the image through skin color detection, calculates the gradient histogram characteristics of the face and the hand regions and the lip shape characteristics of the driver, and classifies the face and the hands by a support vector machine so as to determine whether the driver is connecting or disconnecting the mobile phone. If the gesture of the driver for connecting and calling the mobile phone is found to occur for more than a certain time in the running process of the vehicle, the driver is judged to be connecting and calling the mobile phone;

step three: the alarm device gives an alarm, if the driver still does not stop using the mobile phone after the alarm is given out for 3 seconds, the camera uploads an image of the mobile phone used by the driver to a monitoring center of a traffic bureau through a network to serve as an evidence of the law violation, and law enforcement personnel can conveniently process the law violation;

step four: the device is started only during the running of the vehicle, and when the vehicle stops, the device automatically stops working.

Images