CN103366506A - Device and method for automatically monitoring telephone call behavior of driver when driving - Google Patents

Abstract

The invention relates to a device and a method for automatically monitoring a telephone call behavior of a driver when driving and belongs to the fields of intelligent traffic and aided driving. The device comprises an image acquisition device, a calculating device, a warning device and a wireless transmission device. The method comprises the following steps of firstly acquiring a head and nearby area image of the driver, secondly, obtaining the positions of the face and both hands of the driver in the image by skin color detection, and classifying by a support vector machine to determine whether the driver makes a telephone call or not; warning the driver if the driver makes the telephone call; uploading the image that the driver makes the telephone call to the monitoring center of the road transport bureau through a wireless network if the warning is useless to be used as an evidence that the driver breaks the law. According to the device and the method for automatically monitoring the telephone call behavior of the driver when driving disclosed by the invention, the unlawful act of the driver can be effectively monitored, the working strength of law enforcement personnel is relieved, the working efficiency is improved, and the occurrence rate of traffic accidents is reduced.

Description

Connect automatic monitoring device and the method for the behavior of phoning with mobile telephone in a kind of driver drives vehicle way

Technical field

The present invention relates to intelligent transportation field and auxiliary driving field.

Background technology

Along with popularizing that mobile phone uses, while the driver is driving to dial or the behavior of answering cell phone is seen everywhere.Because the driver is connecing when phoning with mobile telephone, the one-hand control bearing circle can not be controlled vehicle well often; The more important thing is, connecing phones with mobile telephone can disperse driver's notice, in case run into accident, often reacts too late, very easily causes traffic hazard.Studies show that, use mobile phone during driving, slow 30% when the reaction velocity ratio of brain is driven when intoxicated, high more than 4 times when using risk that mobile phone gets into an accident than normal driving during driving.It is that driver attention is not concentrated and caused that 70% fatal traffic hazard is arranged, and mobile phone is to cause scatterbrained one of the main reasons.Connect the harmfulness of phoning with mobile telephone to traffic safety in view of driving, existing approximately 50 countries and regions are prohibited the driver and are phoned with mobile telephone connecing on the run in the world at present.Up-to-date " law on road traffic safety implementing regulations " of China are clear " driver dials in the way of travelling or answering cell phone, remembers 3 minutes, imposes a fine 100 yuan " also.

But because the behavior occurs on the automobile of running at high speed, outside car, be difficult to catch.The monitoring that at present driver's driving connects the behavior of phoning with mobile telephone mainly contains three kinds of methods.The first is law enfrocement official's on-site law-enforcing, but the law enfrocement official generally only appears at a few regions such as crossing, and therefore this method has significant limitation.The second is the outer camera head monitor of car.This mode can photograph the driver uses the behavior of mobile phone when driving, and enlarges although monitoring range is compared with first method to some extent, also is only limited in the scope that the camera monitoring is arrived, in most of zone or monitor less than the behavior.In addition, the method also needs the monitor staff constantly to stare at watch-dog to judge this illegal activities, and working strength is large, and efficient is low, and night efficient lower.The third is the oriented antenna monitoring to be installed from the mobile phone signal on the vehicle in travelling in fixed area, carries out scene evidence taking if the signal of conversing then starts image documentation equipment.Although the method can play the function of automatic monitoring, can take the vehicle that all detect the mobile phone signal of conversing, and this moment not necessarily the driver making a phone call, also may be that others has the situation of a lot of flase drops in conversation in the car.Camera all is positioned on outside the car in rear two kinds of methods, sometimes because weather or the reason such as block so that the picture that photographs can not be seen situation in the car very clearly, has undetected and flase drop occurs.

Based on above reason, " connect in the driving way and phone with mobile telephone " behavior can't effectively be monitored, so that this behavior is seen everywhere, traffic safety has been caused very large hidden danger.At present, also not based on camera in the car to connecing the device that automatic monitoring is carried out in the behavior of phoning with mobile telephone in the driver drives vehicle way.

Summary of the invention

For addressing the above problem, the invention provides the automatic monitoring device and the method that connect the behavior of phoning with mobile telephone in a kind of driver drives vehicle way.By camera collection driver head in the car and near zone image, utilize image processing techniques and mode identification method automatically to identify connecing the behavior of phoning with mobile telephone in the driver drives vehicle way, and with its in violation of rules and regulations picture shooting networking of getting off upload to the Surveillance center of Department of Communications, as the violation evidence.The invention provides a kind of to connecing the method that automatic monitoring is carried out in the behavior of phoning with mobile telephone in the driver drives vehicle way.Can greatly reduce law enforcement intensity, improve law enforcement efficiency, effectively avoid because of the traffic hazard of dialing in the driver drives vehicle way or the answering cell phone behavior causes.And the present invention is installed in camera on the front windshield in the car, can photograph very clearly driver's behavior, and not need other electron devices auxiliary, can not have influence on driver's normal driving.

Connect the automatic monitoring device of the behavior of phoning with mobile telephone in a kind of driver drives vehicle way, comprising: image acquiring device, calculation element, warning device and radio transmitting device; Wherein,

Image acquiring device adopts the video camera that is fixed on the interior front windshield of car to obtain driver's head and near zone image, and sends image to calculation element;

Calculation element obtains driver's face and the position of hand in image by Face Detection, calculate face and the histogram of gradients feature of hand region and driver's lip shape feature, obtaining the driver uses under the mobile phone state and does not use feature under the mobile phone state, classify by support vector machine, thereby determining whether the driver connects in driving procedure phones with mobile telephone, and whether the driver is connect the information of phoning with mobile telephone passes to warning device;

Warning device is after receiving driver that calculation element sends and connecing the information of phoning with mobile telephone, the driver is given a warning, connect the information of phoning with mobile telephone if still receive the driver that calculation element sends after giving a warning, then transmitting a signal to radio transmitting device;

Radio transmitting device meets the driver image of phoning with mobile telephone uploads to Department of Communications by wireless network Surveillance center, as its illegal evidence.

Connect the automatic monitoring method of the behavior of phoning with mobile telephone in a kind of driver drives vehicle way, performing step is as follows:

Step 1: supervising device is opened automatically when vehicle motor starts, device carries out self check operation, checks in the video pictures that photographs whether people's face is arranged, and whether people's face is in the specific zone (prevent the driver from moving or radical occlusion device), if above condition all satisfies, then bearing circle release; Otherwise bearing circle is in the lock state, and the driver can not steering vehicle.

Step 2: device is normal start after, begin to monitor the head zone image, if find have the driver to connect and phone with mobile telephone posture in the process of Vehicle Driving Cycle certain hour occurs to surpass, and simultaneously with the lip action, judge that then it phones with mobile telephone connecing.

Step 3: alarm device generation alarm, if after alarm sent for 3 seconds, the driver does not still stop using mobile phone, then camera uses the image of mobile phone to upload to the Surveillance center of Department of Communications by network it, as its illegal evidence, make things convenient for the law enfrocement official that these illegal activities are processed.

Step 4: this device is only opened in the Vehicle Driving Cycle way, and when vehicle stop, device quits work automatically.

Advantage of the present invention:

1. can in driver's driving way, connect the unlawful practice of phoning with mobile telephone by Real Time Monitoring.

2. can automatically be delivered to Surveillance center with recording in violation of rules and regulations, alleviate law enfrocement official's work load, increase work efficiency.

3. this device is automatically operation, does not need driver's operation, is simple and easy to use.

4. this device is solar obligation, and can guarantee the realization of this function, can effective monitoring driver illegal activities, and reduce this illegal activities, thereby reduce the incidence of traffic hazard.

5. camera is mounted in the car, is not subjected to external environment such as weather, the impact of factor such as blocks, and can photograph clearly image.

Description of drawings

Fig. 1 is the process flow diagram that connects monitoring, warning and the automatic evidence-collecting of the behavior of phoning with mobile telephone among the present invention in the driver drives vehicle way;

Fig. 2 is used for the rectangular characteristic that driver people's face detects among the present invention.

Embodiment

Core thinking of the present invention is: the driver is connecing when phoning with mobile telephone, and certainly has handheld mobile phone near the action of ear, and with lip movement.By video analysis, judge whether to produce these behaviors, thereby judge that whether it phone with mobile telephone connecing.In case determine its illegal activities, in the situation that warning is invalid, send its violation image to Surveillance center at once, and form an in violation of rules and regulations record, be convenient to the law enfrocement official and collect evidence.

Automatic monitoring device of the present invention comprises: image acquiring device, calculation element, warning device and radio transmitting device; Wherein,

Image acquiring device adopts the video camera that is fixed on the interior front windshield of car to obtain driver's head and near zone image;

Calculation element obtains driver's face and the position of hand in image by Face Detection, calculate face and the histogram of gradients feature of hand region and driver's lip shape feature, obtaining the driver uses under the mobile phone state and does not use feature under the mobile phone state, classify by support vector machine, thereby determining whether the driver connects in driving procedure phones with mobile telephone;

Warning device calculation element determine the driver connecing phone with mobile telephone after, the driver is given a warning;

Radio transmitting device meets the driver image of phoning with mobile telephone uploads to Department of Communications by wireless network Surveillance center, as its illegal evidence.

Connect the process flow diagram of the behavior monitoring of phoning with mobile telephone, warning and automatic evidence-collecting in the driver drives vehicle way based on camera in the car among the present invention as shown in Figure 1, the implementation step is as follows:

Step 1: supervising device is opened automatically when vehicle motor starts, device carries out self check operation, checks in the video pictures that photographs whether people's face is arranged, and whether people's face is in the specific zone (prevent the driver from moving or radical occlusion device), if above condition all satisfies, then bearing circle release; Otherwise bearing circle is in the lock state, and the driver can not steering vehicle.

The location of driver people's face is carried out in employing based on the method for detecting human face of AdaBoost.The method is picked out a small amount of key feature with the AdaBoost method again take rectangular characteristic as according to constructing Weak Classifier, constructs strong classifier thereby corresponding Weak Classifier is weighted summation, and it is used for the detection of people's face as final sorter.Wherein, each rectangular characteristic is comprised of 2-3 rectangle, and as shown in Figure 2, its value deducts the interior pixel value sum of black rectangle for the pixel value sum in the white rectangle.

Each Weak Classifier is comprised of a rectangular characteristic, strong classifier the training flow process as follows:

(1) given training data (x ₁, y ₁) ... (x _n, y _n), y wherein _i=0 expression negative sample, y _iThe positive sample of=1 expression, n is the number of training sample.

(2) initialization weights, y _i=0 o'clock y _i=1 o'clock M, l are respectively the numbers of negative sample and positive sample.

(3) corresponding t=1 ..., T:

A. normalization weights $w_{t,i}\←\frac{w_{t,i}}{{\mathrm{\Σ}}_{j=1}^n{w}_{t,j}}$

B. to each feature j, training Weak Classifier h _jIf the error of this sorter is ε _j=∑ _iw _i| h _j(x _i)-y _i|

C. select sorter h with least error _t

D. refreshing weight E wherein _iIf=0 x _iCorrectly classified, otherwise e _i=1, ${\mathrm{\β}}_t=\frac{{\mathrm{\ϵ}}_t}{1-{\mathrm{\ϵ}}_t}$

(4) then final strong classifier is

$h\left(x\right)=\left\{\begin{array}{cc}1& {\mathrm{\Σ}}_{t=1}^T{\mathrm{\α}}_t{h}_t\left(x\right)\≥\frac{1}{2}{\mathrm{\Σ}}_{t=1}^T{\mathrm{\α}}_t\\ 0& \mathrm{otherwise}\end{array}\right.,$ Wherein ${\mathrm{\α}}_t=\log \frac{1}{{\mathrm{\β}}_t}$

After obtaining strong classifier, use strong classifier that image is scanned, obtain people's face position of driver, if people's face in the central area of image, then installs normal operation.The self check success, the bearing circle release, the driver can actuating vehicle.

Step 2: after installing normal the startup, automatic monitoring head zone image in the Vehicle Driving Cycle process is phoned with mobile telephone the posture generation above certain hour if find have the driver to connect, and simultaneously with the lip action, judges that then it is using mobile phone.

Because driver's abnormal driving state and the use state that hand is placed on ear during mobile phone is compared very large difference is arranged.Therefore we adopt the mode of training svm classifier device to judge that whether the driver phones with mobile telephone connecing.Because the driver is speaking and when silent, the lip state has obvious difference, judge therefore whether driver's lip moves the mode that also adopts training svm classifier device.Comprehensive above two clues, the driver has made and has connect the posture of phoning with mobile telephone and surpass certain hour, and therebetween with the lip action, judges that then the driver phones with mobile telephone connecing.

1) the HOG feature of extraction driver head zone

The employed feature of training classifier is the HOG feature.HOG is histogram of orientedgradient(gradient orientation histogram), be the Feature Descriptor for target detection, it calculates the direction gradient number of times that image local occurs.The method is by being divided into entire image little join domain (being called cells), and each cell generates a histograms of oriented gradients, and these histogrammic combinations can be expressed the descriptor of the target that detects.For improving accuracy rate, can be with all the cells normalization in the larger zone (being called block) in the image.Finished better irradiation/shade unchangeability by the normalization process.

The HOG feature extracting method is as follows:

(1) image is converted into gray level image.

(2) image is divided into little cells(eg:2*2).

(3) calculate the gradient of each pixel among each cell.

Gradient magnitude: $R(x,y)=\sqrt{{(I(x+1,y)-I(x-1,y))}^2+{(I(x,y+1)-I(x,y-1))}^2}$

Gradient direction: Ang (x, y)=arctan ((I (x+1, y)-I (x-1, y))/(I (x, y+1)-I (x, y-1)))

(4) all cells among each block of normalization.Adopt L2-norm to carry out normalization:

L2-norm: $v\→v/\sqrt{{\left|\right|v\left|\right|}_2^2+{\mathrm{\ϵ}}^2}$

(5) generate the HOG feature.Calculate the HOG feature among all block in the detection window, and connect into final proper vector for classifying.

2) use SVM that the HOG feature is classified

After having extracted the HOG feature of driver's head zone, with its input as the svm classifier device, classify, to judge driver's head pose.

SVM is Support Vector Machine(support vector machine), it is a kind of method of supervising formula study, shows many distinctive advantages in solving small sample, non-linear and higher-dimension pattern-recognition.SVM based on the structural risk minimization theory in feature space construction optimum segmentation lineoid so that sorter obtains global optimization, and satisfy certain upper bound in the expected risk of whole sample space with certain probability.

The mathematical form of supposing lineoid is: wx-b=0.Wherein x is the point on the lineoid, and w is perpendicular to the vector of lineoid.(x ⁽ⁱ⁾, y ⁽ⁱ⁾) be training sample.x ⁽ⁱ⁾Feature input for training sample; y ⁽ⁱ⁾Be its corresponding label, wherein y ⁽ⁱ⁾∈ 1 ,-1}.Then in the situation that linear separability, optimum lineoid can obtain by separating following double optimization problem:

$\min \frac{1}{2}{\left|\right|w\left|\right|}^2$

Satisfy constraint condition: y ⁽ⁱ⁾(w ^Tx ⁽ⁱ⁾+ b) 〉=1, and i=1,2 ..., m

3) judge driver's lip state

The process of judging driver's lip state is:

1. detect people's face position by the Adaboost learning algorithm

2. lip position coarse positioning.According to priori, lip is positioned at the latter half of people's face.

3. lip shape and location positioning.Training skin color and lip color are determined respectively its color gamut, utilize the Fisher linear classifier that skin color and lip chromatic zones are separated, and are partitioned into mouth region.

4. elimination noise.Corrode first the method for rear expansion by the mouth region that previous step is partitioned into, eliminate little connected region, reach the purpose of removing noise.Zone behind the removal noise is exactly mouth region.

5. the mouth profile is determined.The detected mouth region of previous step is carried out the horizontal and vertical projection, obtain the mouth profile.

6. feature extraction.Calculate the breadth extreme W of mouth region according to the mouth profile _Max, maximum height H _Max, and the height H between upper lip and the lower lip _mThese several values have obvious different when the people speaks and be silent, so are taken as feature.F=(W _amx,H _max,H _m).

7. training svm classifier device is classified to the lip feature, differentiates driver's mouth state.4) judge that whether the driver phones with mobile telephone connecing

When using SVM that the HOG feature of driver's head zone is classified, judging that it belongs to connects the posture of phoning with mobile telephone, judge simultaneously driver's lip state for speaking state, and this type of situation duration when surpassing given threshold value, judge that then the driver phones with mobile telephone connecing.

Step 3: alarm device gives the alarm, if after alarm sent for 3 seconds, the driver does not still stop using mobile phone, and then radio transmitting device uploads to the image of its use mobile phone law enfrocement official's Surveillance center by network, as its illegal evidence.And with its illegal activities, the content records such as time get off, and make things convenient for the law enfrocement official that these illegal activities are processed.

Step 4: this device is only opened in the Vehicle Driving Cycle way, and when vehicle stop, device quits work automatically.

Claims (2)

1. connect the automatic monitoring device of the behavior of phoning with mobile telephone in the driver drives vehicle way, it is characterized in that: this automatic monitoring device comprises image acquiring device, calculation element, warning device and radio transmitting device; Wherein,

Image acquiring device adopts the video camera that is fixed on the interior front windshield of car to obtain driver's head and near zone image, and sends image to calculation element;

Calculation element passes through training sample, obtaining the driver connects under the state of phoning with mobile telephone and the non-feature that connects under the state of phoning with mobile telephone, can classify to each sub-picture that image acquiring device obtains, thereby can determine further whether the driver connects in driving procedure phones with mobile telephone, and whether the driver is connect the information of phoning with mobile telephone passes to warning device;

Warning device is after receiving driver that calculation element sends and connecing the information of phoning with mobile telephone, the driver is given a warning, connect the information of phoning with mobile telephone if still receive the driver that calculation element sends after giving a warning, then transmitting a signal to radio transmitting device;

Radio transmitting device meets the driver image of phoning with mobile telephone uploads to Department of Communications by wireless network Surveillance center, as its illegal evidence.

2. connect the automatic monitoring method of the behavior of phoning with mobile telephone in the driver drives vehicle way, it is characterized in that:

Step 1: supervising device is opened automatically when vehicle motor starts, and device carries out self check operation, checks in the video pictures that photographs whether people's face is arranged, and whether people's face be in the specific zone, if above condition all satisfies, and then bearing circle release; Otherwise bearing circle is in the lock state, and the driver can not steering vehicle;

Step 2: after the normal startup of device, begin to monitor the head zone image.Calculation element obtains driver's face and the position of hand in image by Face Detection, calculate face and the histogram of gradients feature of hand region and driver's lip shape feature, classify by support vector machine, thereby determine that whether the driver phones with mobile telephone connecing.Occur to surpass certain hour if in the process of Vehicle Driving Cycle, find have the driver to connect and phone with mobile telephone posture, judge that then it phones with mobile telephone connecing;

Step 3: alarm device generation alarm, if after alarm sent for 3 seconds, the driver does not still stop using mobile phone, then camera uses the image of mobile phone to upload to the Surveillance center of Department of Communications by network it, as its illegal evidence, make things convenient for the law enfrocement official that these illegal activities are processed;

Step 4: this device is only opened in the Vehicle Driving Cycle way, and when vehicle stop, device quits work automatically.