CN106056071A - Method and device for detection of driver' behavior of making call - Google Patents
Method and device for detection of driver' behavior of making call Download PDFInfo
- Publication number
- CN106056071A CN106056071A CN201610368797.XA CN201610368797A CN106056071A CN 106056071 A CN106056071 A CN 106056071A CN 201610368797 A CN201610368797 A CN 201610368797A CN 106056071 A CN106056071 A CN 106056071A
- Authority
- CN
- China
- Prior art keywords
- border
- phone call
- image
- area
- ksi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Neurology (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a method for detection of the driver's behavior of making a call. The method comprises: selecting the color image of a label as a sample image, employing a convolution nerve network to perform repeat training of the sample image, obtaining call-up detection module with good training; obtaining the car window candidate area according to the license plate area, employing Hough to convert the detection line in the car window candidate area, performing cluster processing and extracting a car window area; employing a face detection algorithm to detect in the car window area, and extracting the face area; obtaining the interest area of making a call according to the face area; and employing the call-up detection module with good training to detect the interest area, and outputting the detection result. Compared with the prior art, the method and device for detection of the driver' behavior of making a call can accurately detect the driver's behavior of making a call, and the robustness is good.
Description
Technical field
The present invention relates to image procossing, video monitoring and intelligent transportation, the inspection of behavior of making a phone call particularly to driver
Survey method and device.
Background technology
Along with the development of transportation, vehicle accident become current harm human life's safety main public hazards it
One, the serious social problem that Ye Shi our times various countries are faced simultaneously.In the occurrence cause of vehicle accident, drive
Member is absent minded is one of the main reasons.Report display according to statistics, the note of the meeting severe jamming driver that makes a phone call during driving
Meaning power so that high more than 4 times during the Hazard ratio normal driving got into an accident.
Research currently for behavioral value of making a phone call in driver drives vehicle way is the most fewer, is concentrated mainly on based on hands
Machine signal detects.Making a phone call or passenger is making a phone call, side based on mobile phone signal owing to very difficult resolution is driver
Formula has a lot of flase drop.Along with computer hardware and software engineering, image processing techniques and the skill such as computer vision, pattern recognition
The development of art, behavioral value of making a phone call based on image procossing is studied in recent years.
Existing behavioral value of making a phone call based on image procossing mostly based on grader, such as Publication No.
The Chinese invention patent application of CN104573659A and CN102567743A is based on SVM (Support Vector
Machines, support vector machine) grader, the Chinese invention patent application of Publication No. CN104966059A be based on
Cascade cascade classifier.The feature extracted yet with grader is limited, therefore have impact on the inspection of behavioral value of making a phone call
Survey accuracy rate.
In sum, make a phone call the detection side of behavior in the urgent need to proposing the higher driver of a kind of Detection accuracy at present
Method and device.
Summary of the invention
In view of this, present invention is primarily targeted at and realize driver and make a phone call the detection of behavior, and Detection accuracy
Higher.
For reaching above-mentioned purpose, according to the first aspect of the invention, it is provided that a kind of driver makes a phone call the inspection of behavior
Survey method, the method includes:
First step, the coloured image choosing label is sample image, uses convolutional neural networks to carry out sample image
Repetition training, obtains the detection model of making a phone call trained；
Second step, obtains the candidate region of vehicle window according to license plate area；
Third step, uses Hough transform detection of straight lines in the candidate region of vehicle window, straight line is carried out clustering processing,
Extract vehicle window region；
4th step, uses Face datection algorithm to detect in vehicle window region, extracts human face region；
5th step, obtains the area-of-interest made a phone call according to human face region；
6th step, utilizes the detection model of making a phone call trained to detect area-of-interest, output detections result.
Described first step farther includes:
Sample selecting step, chooses coloured image that label driver makes a phone call, colour that label driver does not makes a phone call
The coloured image that image, label obscure is as sample image；
Initial training step, utilizes convolutional neural networks that sample image is carried out features training, it is thus achieved that the mould of initial training
Type；
Second training step, chooses test image, according to the model of initial training, test image is carried out repetition training, directly
Restrain to model；
Model output step, makes a phone call the model of convergence detection model exporting as the driver trained.
In described initial training step, convolutional neural networks includes: input layer, Th_Con convolutional layer, Th_Pool pond
Change layer, Th_Full full articulamentum.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*
CKSi, step-length are 1.The size of the core of every layer of pond layer is KSi*KSi, step-length is KSi.Last layer of described full articulamentum is complete
The quantity of the neuron of articulamentum output is 3, is 3 drivers and makes a phone call to detect classification.
Further, described convolutional neural networks includes:
Input layer, the image of input Width*Height；
Ground floor convolutional layer, exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length is 1；
Ground floor pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Second layer convolutional layer, exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length is 1；
Second layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Third layer convolutional layer, exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length is 1；
Third layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Full articulamentum, uses ReLU as activation primitive, exports Th_Neur neuron；
Full articulamentum, exports 3 neurons, makes a phone call for i.e. 3 to detect classification.
Described second training step farther includes:
Training characteristics extraction step, tests the feature of image according to the model extraction of initial training；
Training classification determination step, calculating this feature and each driver make a phone call to detect the similarity of category feature
Simik, k represents kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training step, calculates the error of result of determination and legitimate reading, utilizes back-propagation algorithm to carry out training pattern,
Repetition training characteristic extraction step and training classification determination step, until the convergence of this model.
Described second step farther includes:
License plate area positioning step, obtains license plate area according to algorithm of locating license plate of vehicle from the coloured image gathered；
Border, the candidate region obtaining step of vehicle window, obtains left boundary x=pl of license plate area, the right side according to license plate area
Border, limit x=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor car plate district
The width in territory, W is the width gathering image, λ 3 ＜ λ 2；
The candidate region output step of vehicle window, according to the left boundary of candidate region of vehicle window, border, the right, limit, top
Boundary, following border, determine rectangular area, and this rectangular area is the candidate region of vehicle window.
Described third step farther includes:
Vertically edge obtaining step, carries out gray processing process by the candidate region of vehicle window, obtains the candidate region of gray scale, adopts
WithWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtaining step, uses threshold value Th_F logarithm value edge image to split, obtains bianry image；
Straight-line detection step, uses Hough transform line detection algorithm to process bianry image, obtains the straight of detection
Line sequence row y=kix+bi, i=1,2 ..., N1, N1Quantity for straight line；
Straight line screening step, if arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, thus obtain
Remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary obtaining step, scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bj's
First pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-directionStraight line is entered
Row segmentation, obtains the length on corresponding x direction, and puts it in LineHist array, uses clustering algorithm pair
LineHist array clusters, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary obtaining step, scans coboundary, lower boundary respectively, by the first of coboundary pixel and lower boundary
The straight line that constitutes of first pixel as left margin, by last of last pixel of coboundary and lower boundary
The straight line that pixel is constituted is as right margin；
Vehicle window area acquisition step, coboundary, left margin, right margin, lower boundary the region surrounded is vehicle window region.
Described 5th step farther includes:
Region of interest border obtaining step, obtains left boundary x=fl of human face region, limit, the right according to human face region
Boundary x=fr, border, top y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isBorder, top isBorder is belowWfFor face district
The width in territory, W and H is respectively width and the height gathering image；
Area-of-interest obtaining step, according to the left boundary of the area-of-interest made a phone call, border, the right, limit, top
Boundary, following border, determine rectangular area, and this rectangular area is area-of-interest.
Described 6th step farther includes:
Identify characteristic extraction step, utilize the detection model of making a phone call trained to extract the feature of area-of-interest；
Identify classification determination step, calculate the feature of area-of-interest and similarity Simi of each category featurek, k table
Show kth classification, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result exporting as driver.
According to another aspect of the present invention, it is provided that a kind of driver makes a phone call the detection device of behavior, this device bag
Include:
Make a phone call detection model acquisition module, be sample image for choosing the coloured image of label, use convolutional Neural
Network carries out repetition training to sample image, obtains the detection model of making a phone call trained；
The candidate region extraction module of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
Vehicle window region extraction module, for using Hough transform detection of straight lines in the candidate region of vehicle window, enters straight line
Row clustering processing, extracts vehicle window region；
Human face region extraction module, is used for using Face datection algorithm to detect in vehicle window region, extracts face district
Territory；
Region of interesting extraction module, for obtaining the area-of-interest made a phone call according to human face region；
Make a phone call detection model detection module, for utilizing the detection model of making a phone call trained that area-of-interest is carried out
Detection, output detections result.
Described detection model acquisition module of making a phone call farther includes:
Module chosen by sample, and for choosing coloured image that label driver makes a phone call, label driver does not makes a phone call
The coloured image that coloured image, label obscure is as sample image；
Initial training module, is used for utilizing convolutional neural networks that sample image is carried out features training, it is thus achieved that initial training
Model；
Second training module, is used for choosing test image, repeatedly instructs test image according to the model of initial training
Practice, until model convergence；
Model output module, for making a phone call the model of convergence detection model exporting as the driver trained.
Described second training module farther includes:
Training characteristics extraction module, tests the feature of image for the model extraction according to initial training；
Training classification determination module, makes a phone call to detect the similarity of category feature for calculating this feature and each driver
Simik, k represents kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training module, for calculating the error of result of determination and legitimate reading, utilizes back-propagation algorithm to train
Model, repetition training characteristic extracting module and training classification determination module, until the convergence of this model.
The candidate region extraction module of described vehicle window farther includes:
License plate area locating module, for obtaining license plate area according to algorithm of locating license plate of vehicle from the coloured image gathered；
Border, the candidate region acquisition module of vehicle window, for obtaining left boundary x=of license plate area according to license plate area
Pl, border, the right x=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor car plate district
The width in territory, W is the width gathering image, λ 3 ＜ λ 2；
The candidate region acquisition module of vehicle window, for the left boundary of candidate region according to vehicle window, border, the right, top
Border, following border, determine rectangular area, and this rectangular area is the candidate region of vehicle window.
Described vehicle window region extraction module farther includes:
Vertically edge acquisition module, for the candidate region of vehicle window carries out gray processing process, obtains the candidate regions of gray scale
Territory, usesWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image acquisition module, is used for using threshold value Th_F logarithm value edge image to split, and obtains bianry image；
Straight-line detection module, is used for using Hough transform line detection algorithm to process bianry image, obtains detection
Linear order y=kix+bi, i=1,2 ..., N1, N1Quantity for straight line；
Straight line shaker modeling block, if for arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, thus
Obtain remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary acquisition module, is used for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx
+bjFirst pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-directionWill be straight
Line carries out segmentation, obtains the length on corresponding x direction, and puts it in LineHist array, uses clustering algorithm pair
LineHist array clusters, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary acquisition module, for scanning coboundary, lower boundary respectively, by the first of coboundary pixel and under
The straight line that first pixel on border is constituted as left margin, last by last pixel of coboundary and lower boundary
The straight line of one pixel composition is as right margin；
Vehicle window region acquisition module, is vehicle window for the region surrounded by coboundary, left margin, right margin, lower boundary
Region.
Described region of interesting extraction module farther includes:
Region of interest border acquisition module, for obtaining left boundary x=fl of human face region, the right side according to human face region
Border, limit x=fr, border, top y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isBorder, top isBorder is belowWfFor face
The width in region, W and H is respectively width and the height gathering image；
Area-of-interest acquisition module, for according to the left boundary of area-of-interest made a phone call, border, the right, top
Border, following border, determine rectangular area, and this rectangular area is area-of-interest.
Described detection model detection module of making a phone call farther includes:
Identify characteristic extracting module, for utilizing the detection model of making a phone call trained to extract the feature of area-of-interest；
Identify classification determination module, for calculating the feature of area-of-interest and similarity Simi of each category featurek,
K represents kth classification, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result defeated as driver
Go out.
Compared with the detection technique of behavior of making a phone call with existing driver, a kind of driver of the present invention makes a phone call behavior
Detection method and device use convolutional neural networks, can detect the behavior that driver makes a phone call accurately, and robustness is relatively
Good；It addition, the convolutional neural networks in the present invention trained, driver makes a phone call, driver does not makes a phone call, fuzzy three classifications,
Fuzzy situation can be told.
Accompanying drawing explanation
Fig. 1 shows the flow chart of detection method of behavior of making a phone call according to a kind of driver of the present invention.
Fig. 2 shows the frame diagram detecting device of behavior of making a phone call according to a kind of driver of the present invention.
Detailed description of the invention
For making your auditor can further appreciate that the structure of the present invention, feature and other purposes, in conjunction with appended preferable reality
Executing example and describe in detail as follows, illustrated preferred embodiment is merely to illustrate technical scheme, and the non-limiting present invention.
Fig. 1 gives the flow chart of detection method of behavior of making a phone call according to a kind of driver of the present invention.Such as Fig. 1 institute
Show, include according to the make a phone call detection method of behavior of a kind of driver of the present invention:
First step S1, the coloured image choosing label is sample image, uses convolutional neural networks to enter sample image
Row repetition training, obtains the detection model of making a phone call trained；
Second step S2, obtains the candidate region of vehicle window according to license plate area；
Third step S3, uses Hough transform detection of straight lines in the candidate region of vehicle window, carries out straight line at cluster
Reason, extracts vehicle window region；
4th step S4, uses Face datection algorithm to detect in vehicle window region, extracts human face region；
5th step S5, obtains the area-of-interest made a phone call according to human face region；
6th step S6, utilizes the detection model of making a phone call trained to detect area-of-interest, and output detections is tied
Really.
Described first step S1 farther includes:
Sample selecting step S11, chooses coloured image that label driver makes a phone call, coloured silk that label driver does not makes a phone call
The coloured image that color image, label obscure is as sample image；
Initial training step S12, utilizes convolutional neural networks that sample image is carried out features training, it is thus achieved that initial training
Model；
Second training step S13, chooses test image, repeatedly instructs test image according to the model of initial training
Practice, until model convergence；
Model output step S14, makes a phone call the model of convergence detection model exporting as the driver trained.
Wherein, in described sample selecting step S11, the width of sample image is Width, height is for Height.Width∈
[64,192], Height ∈ [64,192].Preferably, Width elects 128 as, and Height elects 128 as.The label driver chosen
The coloured image quantity made a phone call can be more than 500, and the coloured image quantity that the label driver chosen does not makes a phone call can be big
In 500, the coloured image quantity that the label chosen is fuzzy can be more than 500.Preferably, 1000～5000000 are chosen respectively
Label driver makes a phone call, driver does not makes a phone call, fuzzy coloured image is sample image.
In described initial training step S12, convolutional neural networks includes: input layer, Th_Con convolutional layer, Th_Pool
Pond layer, Th_Full full articulamentum.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*
CKSi, step-length are 1.The size of the core of every layer of pond layer is KSi*KSi, step-length is KSi.Last layer of described full articulamentum is complete
The quantity of the neuron of articulamentum output is 3, is 3 drivers and makes a phone call to detect classification.
Wherein, described Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,
Th_CKmax], Th_CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
Further, described convolutional neural networks includes:
Input layer, the image of input Width*Height；
Ground floor convolutional layer, exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length is 1；
Ground floor pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Second layer convolutional layer, exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length is 1；
Second layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Third layer convolutional layer, exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length is 1；
Third layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Full articulamentum, uses ReLU as activation primitive, exports Th_Neur neuron；
Full articulamentum, exports 3 neurons, makes a phone call for i.e. 3 to detect classification.
Wherein, Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,
5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64,10000].
Preferably, Th_CK1 is set to 8, and CKSi1 is set to 5, and KSi is set to 2, and Th_CK2 is set to 8, and CKSi2 is set to 5, Th_CK3
Being set to 8, CKSi3 is set to 5, and Th_Neur is set to 128.
Described ground floor, the second layer, third layer pond layer in maximum pond method could alternatively be average pond method.
In described full articulamentum, ReLU full name is Rectified Linear Units, and Chinese is translated into correction linear unit,
It is referred to document " Taming the ReLU with Parallel Dither in a Deep Neural
Network.AJR Simpson.Computer Science,2015”。
In described full articulamentum, ReLU could alternatively be sigmoid function or tanh function as activation primitive.
Described second training step S13 farther includes:
Training characteristics extraction step S131, tests the feature of image according to the model extraction of initial training；
Training classification determination step S132, calculating this feature and each driver make a phone call to detect the similarity of category feature
Simik, k represents kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training step S133, calculates the error of result of determination and legitimate reading, utilizes back-propagation algorithm to train
Model, repetition training characteristic extraction step S131 and training classification determination step S132, until the convergence of this model.
Described second step S2 farther includes:
License plate area positioning step S21, obtains license plate area according to algorithm of locating license plate of vehicle from the coloured image gathered；
Border, the candidate region obtaining step S22 of vehicle window, obtains left boundary x=of license plate area according to license plate area
Pl, border, the right x=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor car plate district
The width in territory, W is the width gathering image, λ 3 ＜ λ 2；
Candidate region output step S23 of vehicle window, according to the left boundary of candidate region of vehicle window, border, the right, top
Border, following border, determine rectangular area, and this rectangular area is the candidate region of vehicle window.
In described license plate area positioning step S21, algorithm of locating license plate of vehicle is existing algorithm of locating license plate of vehicle.Such as, " Li Wen
Lift, Liang Dequn, Zhang Qi, Fan Xin. new location method of vehicle license plate based on edge color pair. " Chinese journal of computers ", 2004,27 (2):
204-208”。
Described λ 1 ∈ [1.2,1.8], λ 2 ∈ [4.2,4.8], λ 3 ∈ [0.3,0.8].Preferably, λ 1 elects 1.5 as, and λ 2 elects as
4.5, λ 3 elect 0.5 as.
Described third step S3 farther includes:
Vertically edge obtaining step S31, carries out gray processing process by the candidate region of vehicle window, obtains the candidate regions of gray scale
Territory, usesWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtaining step S32, uses threshold value Th_F logarithm value edge image to split, obtains bianry image；
Straight-line detection step S33, uses Hough transform line detection algorithm to process bianry image, obtains detection
Linear order y=kix+bi, i=1,2 ..., N1, N1Quantity for straight line；
Straight line screening step S34, if arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, thus obtain
To remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary obtaining step S35, scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+
bjFirst pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-directionWill be straight
Line carries out segmentation, obtains the length on corresponding x direction, and puts it in LineHist array, uses clustering algorithm pair
LineHist array clusters, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary obtaining step S36, scans coboundary, lower boundary respectively, by the first of coboundary pixel and under
The straight line that first pixel on border is constituted as left margin, last by last pixel of coboundary and lower boundary
The straight line of one pixel composition is as right margin；
Vehicle window area acquisition step S37, coboundary, left margin, right margin, lower boundary the region surrounded is vehicle window district
Territory.
Wherein, Th_F ∈ [10,30] in described bianry image obtaining step S32.Preferably, Th_F elects 20 as.
In described straight-line detection step S33, Hough transform line detection algorithm is realized by existing technology.Such as, " section
You spoil, Zhao Wei, yellow pine ridge, Chen Jianye. a kind of straight line fast algorithm of detecting based on Improved Hough Transform. and " instrument and meter
Report ", 2010,31 (12): 2774-2780 ".
Th_ θ ∈ [5 °, 15 °] in described straight line screening step S34.Preferably.Th_ θ elects 10 ° as.
In described up-and-down boundary obtaining step S35, Th_S is the sampling interval, Th_S ∈ [5,20].Preferably, Th_S elects as
12.Described clustering algorithm is realized by existing clustering algorithm.
Face datection algorithm in described 4th step S4 can be realized by existing technology.Such as, " Guo Zhibo, China
Continue and encourage, Yan Yunyang, old ability button, Yang Jingyu. face based on dual threshold succession type AdaBoost algorithm quickly detects. " data acquisition
Collection and process ", 2008,23 (3): 306-310 ".
Described 5th step S5 farther includes:
Region of interest border obtaining step S51, obtains left boundary x=fl of human face region, the right side according to human face region
Border, limit x=fr, border, top y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isBorder, top isBorder is belowWfFor face
The width in region, W and H is respectively width and the height gathering image；
Area-of-interest obtaining step S52, according to the left boundary of the area-of-interest made a phone call, border, the right, top
Border, following border, determine rectangular area, and this rectangular area is area-of-interest.
Wherein, described λ 4 ∈ [0.8,1.3], λ 5 ∈ [0.3,0.8], λ 6 ∈ [0.3,0.8].Preferably, λ 4 elects 1 as, λ 5
Electing 0.5 as, λ 6 elects 0.5 as.
Described 6th step S6 farther includes:
Identify characteristic extraction step S61, utilize the detection model of making a phone call trained to extract the feature of area-of-interest；
Identify classification determination step S62, calculate the feature of area-of-interest and similarity Simi of each category featurek, k
Expression kth classification, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result exporting as driver.
Fig. 2 gives the frame diagram detecting device of behavior of making a phone call according to a kind of driver of the present invention.Such as Fig. 2 institute
Show, include according to the make a phone call detection device of behavior of a kind of driver of the present invention:
Make a phone call detection model acquisition module 1, be sample image for choosing the coloured image of label, use convolutional Neural
Network carries out repetition training to sample image, obtains the detection model of making a phone call trained；
The candidate region extraction module 2 of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
Vehicle window region extraction module 3, for using Hough transform detection of straight lines in the candidate region of vehicle window, to straight line
Carry out clustering processing, extract vehicle window region；
Human face region extraction module 4, is used for using Face datection algorithm to detect in vehicle window region, extracts face district
Territory；
Region of interesting extraction module 5, for obtaining the area-of-interest made a phone call according to human face region；
Make a phone call detection model detection module 6, for utilizing the detection model of making a phone call trained that area-of-interest is entered
Row detection, output detections result.
Described detection model acquisition module 1 of making a phone call farther includes:
Module 11 chosen by sample, and for choosing coloured image that label driver makes a phone call, label driver does not makes a phone call
Coloured image, the fuzzy coloured image of label is as sample image；
Initial training module 12, is used for utilizing convolutional neural networks that sample image is carried out features training, it is thus achieved that tentatively instruct
The model practiced；
Second training module 13, is used for choosing test image, carries out test image repeatedly according to the model of initial training
Training, until model convergence；
Model output module 14, for making a phone call the model of convergence detection model defeated as the driver trained
Go out.
Wherein, during module 11 chosen by described sample, the width of sample image is Width, height is for Height.Width∈
[64,192], Height ∈ [64,192].Preferably, Width elects 128 as, and Height elects 128 as.The label driver chosen
The coloured image quantity made a phone call can be more than 500, and the coloured image quantity that the label driver chosen does not makes a phone call can be big
In 500, the coloured image quantity that the label chosen is fuzzy can be more than 500.Preferably, 1000～5000000 are chosen respectively
Label driver makes a phone call, driver does not makes a phone call, fuzzy coloured image is sample image.
In described initial training module 12, convolutional neural networks includes: input layer, Th_Con convolutional layer, Th_Pool
Pond layer, Th_Full full articulamentum.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*
CKSi, step-length are 1.The size of the core of every layer of pond layer is KSi*KSi, step-length is KSi.Last layer of described full articulamentum is complete
The quantity of the neuron of articulamentum output is 3, is 3 drivers and makes a phone call to detect classification.
Wherein, described Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,
Th_CKmax], Th_CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
Further, described convolutional neural networks includes:
Input layer, the image of input Width*Height；
Ground floor convolutional layer, exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length is 1；
Ground floor pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Second layer convolutional layer, exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length is 1；
Second layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Third layer convolutional layer, exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length is 1；
Third layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Full articulamentum, uses ReLU as activation primitive, exports Th_Neur neuron；
Full articulamentum, exports 3 neurons, makes a phone call for i.e. 3 to detect classification.
Wherein, Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,
5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64,10000].
Preferably, Th_CK1 is set to 8, and CKSi1 is set to 5, and KSi is set to 2, and Th_CK2 is set to 8, and CKSi2 is set to 5, Th_CK3
Being set to 8, CKSi3 is set to 5, and Th_Neur is set to 128.
Described ground floor, the second layer, third layer pond layer in maximum pond method could alternatively be average pond method.
In described full articulamentum, ReLU full name is Rectified Linear Units, and Chinese is translated into correction linear unit,
It is referred to document " Taming the ReLU with Parallel Dither in a Deep Neural
Network.AJR Simpson.Computer Science,2015”。
In described full articulamentum, ReLU could alternatively be sigmoid function or tanh function as activation primitive.
Described second training module 13 farther includes:
Training characteristics extraction module 131, tests the feature of image for the model extraction according to initial training；
Training classification determination module 132, makes a phone call the phase of detection category feature for calculating this feature and each driver
Seemingly spend Simik, k represents kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training module 133, for calculating the error of result of determination and legitimate reading, utilizes back-propagation algorithm to instruct
Practice model, repetition training characteristic extracting module 131 and training classification determination module 132, until the convergence of this model.
The candidate region extraction module 2 of described vehicle window farther includes:
License plate area locating module 21, for obtaining car plate district according to algorithm of locating license plate of vehicle from the coloured image gathered
Territory；
Border, the candidate region acquisition module 22 of vehicle window, for obtaining left boundary x of license plate area according to license plate area
=pl, border, the right x=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor car plate district
The width in territory, W is the width gathering image, λ 3 ＜ λ 2；
The candidate region acquisition module 23 of vehicle window, for according to the left boundary of candidate region of vehicle window, border, the right,
Border, limit, following border, determine rectangular area, and this rectangular area is the candidate region of vehicle window.
In described license plate area locating module 21, algorithm of locating license plate of vehicle is existing algorithm of locating license plate of vehicle.Such as, " Li Wen
Lift, Liang Dequn, Zhang Qi, Fan Xin. new location method of vehicle license plate based on edge color pair. " Chinese journal of computers ", 2004,27 (2):
204-208”。
Described λ 1 ∈ [1.2,1.8], λ 2 ∈ [4.2,4.8], λ 3 ∈ [0.3,0.8].Preferably, λ 1 elects 1.5 as, and λ 2 elects as
4.5, λ 3 elect 0.5 as.
Described vehicle window region extraction module 3 farther includes:
Vertically edge acquisition module 31, for the candidate region of vehicle window carries out gray processing process, obtains the candidate of gray scale
Region, usesWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image acquisition module 32, is used for using threshold value Th_F logarithm value edge image to split, and obtains binary map
Picture；
Straight-line detection module 33, is used for using Hough transform line detection algorithm to process bianry image, obtains inspection
The linear order y=k surveyedix+bi, i=1,2 ..., N1, N1Quantity for straight line；
Straight line shaker modeling block 34, if for arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, by
This obtains remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary acquisition module 35, is used for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=
kjx+bjFirst pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-direction
Straight line is carried out segmentation, obtains the length on corresponding x direction, and put it in LineHist array, use clustering algorithm
LineHist array is clustered, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary acquisition module 36, for scanning coboundary, lower boundary respectively, by the first of coboundary pixel and
The straight line that first pixel of lower boundary is constituted is as left margin, by last pixel of coboundary and lower boundary
The straight line that later pixel is constituted is as right margin；
Vehicle window region acquisition module 37, is car for the region surrounded by coboundary, left margin, right margin, lower boundary
Window region.
Wherein, Th_F ∈ [10,30] in described bianry image acquisition module 32.Preferably, Th_F elects 20 as.
In described straight-line detection module 33, Hough transform line detection algorithm is realized by existing technology.Such as, " section
You spoil, Zhao Wei, yellow pine ridge, Chen Jianye. a kind of straight line fast algorithm of detecting based on Improved Hough Transform. and " instrument and meter
Report ", 2010,31 (12): 2774-2780 ".
Th_ θ ∈ [5 °, 15 °] in described straight line shaker modeling block 34.Preferably.Th_ θ elects 10 ° as.
In described up-and-down boundary acquisition module 35, Th_S is the sampling interval, Th_S ∈ [5,20].Preferably, Th_S elects as
12.Described clustering algorithm is realized by existing clustering algorithm.
Face datection algorithm in described human face region extraction module 4 can be realized by existing technology.Such as, " Guo
Will ripple, Hua Jizhao, Yan Yunyang, old ability button, Yang Jingyu. face based on dual threshold succession type AdaBoost algorithm quickly detects.
" data acquisition and procession ", 2008,23 (3): 306-310 ".
Described region of interesting extraction module 5 farther includes:
Region of interest border acquisition module 51, for according to human face region obtain human face region left boundary x=fl,
The right border x=fr, border, top y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isLimit, top
Boundary isBorder is belowWfFor
The width of human face region, W and H is respectively width and the height gathering image；
Area-of-interest acquisition module 52, for according to make a phone call the left boundary of area-of-interest, border, the right,
Border, limit, following border, determine rectangular area, and this rectangular area is area-of-interest.
Wherein, described λ 4 ∈ [0.8,1.3], λ 5 ∈ [0.3,0.8], λ 6 ∈ [0.3,0.8].Preferably, λ 4 elects 1 as, λ 5
Electing 0.5 as, λ 6 elects 0.5 as.
Described detection model detection module 6 of making a phone call farther includes:
Identify characteristic extracting module 61, for utilizing the detection model of making a phone call trained to extract the spy of area-of-interest
Levy；
Identify classification determination module 62, for calculating the feature of area-of-interest and the similarity of each category feature
Simik, k represents kth classification, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result as driver
And export.
Compared with the detection technique of behavior of making a phone call with existing driver, a kind of driver of the present invention makes a phone call behavior
Detection method and device use convolutional neural networks, can detect the behavior that driver makes a phone call accurately, and robustness is relatively
Good；It addition, the convolutional neural networks in the present invention trained, driver makes a phone call, driver does not makes a phone call, fuzzy three classifications,
Fuzzy situation can be told.
The above, only presently preferred embodiments of the present invention, it is not intended to limit protection scope of the present invention, it should
Understanding, the present invention is not limited to implementation as described herein, and the purpose that these implementations describe is to help this area
In technical staff put into practice the present invention.Any those of skill in the art are easy to without departing from spirit and scope of the invention
In the case of be further improved and perfect, therefore the present invention is only by the content of the claims in the present invention and limiting of scope
System, its be intended to contain the alternative in all spirit and scope of the invention being included in and being defined by the appended claims and etc.
Same scheme.
Claims (18)
1. a driver makes a phone call the detection method of behavior, it is characterised in that the method includes:
First step, the coloured image choosing label is sample image, uses convolutional neural networks to carry out sample image repeatedly
Training, obtains the detection model of making a phone call trained；
Second step, obtains the candidate region of vehicle window according to license plate area；
Third step, uses Hough transform detection of straight lines in the candidate region of vehicle window, straight line carries out clustering processing, extracts
Vehicle window region；
4th step, uses Face datection algorithm to detect in vehicle window region, extracts human face region；
5th step, obtains the area-of-interest made a phone call according to human face region；
6th step, utilizes the detection model of making a phone call trained to detect area-of-interest, output detections result.
2. the method for claim 1, it is characterised in that described first step includes:
Sample selecting step, choose coloured image that label driver makes a phone call, coloured image that label driver does not makes a phone call,
The coloured image that label obscures is as sample image；
Initial training step, utilizes convolutional neural networks that sample image is carried out features training, it is thus achieved that the model of initial training；
Second training step, chooses test image, according to the model of initial training, test image is carried out repetition training, until mould
Type restrains；
Model output step, makes a phone call the model of convergence detection model exporting as the driver trained.
3. method as claimed in claim 2, it is characterised in that described convolutional neural networks includes: input layer, Th_Con volume
Lamination, Th_Pool pond layer, Th_Full full articulamentum；
Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*CKSi, step-length is 1；Every layer of pond
The size of the core of layer is KSi*KSi, step-length is KSi；The neuron of last layer of full articulamentum output of described full articulamentum
Quantity is 3, is 3 drivers and makes a phone call to detect classification；
Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,Th_CKmax], Th_
CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
4. method as claimed in claim 3, it is characterised in that described convolutional neural networks includes:
Input layer, the image of input Width*Height；
Ground floor convolutional layer, exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length is 1；
Ground floor pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Second layer convolutional layer, exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length is 1；
Second layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Third layer convolutional layer, exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length is 1；
Third layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Full articulamentum, uses ReLU as activation primitive, exports Th_Neur neuron；
Full articulamentum, exports 3 neurons, makes a phone call for i.e. 3 to detect classification；
Wherein, Width ∈ [64,192], Height ∈ [64,192]；Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈
[2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64,
10000]。
5. method as claimed in claim 2, described second training step includes:
Training characteristics extraction step, tests the feature of image according to the model extraction of initial training；
Training classification determination step, calculating this feature and each driver make a phone call to detect similarity Simi of category featurek, k table
Show kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training step, calculates the error of result of determination and legitimate reading, utilizes back-propagation algorithm to carry out training pattern, repeats
Training characteristics extraction step and training classification determination step, until the convergence of this model.
6. the method for claim 1, it is characterised in that described second step includes:
License plate area positioning step, obtains license plate area according to algorithm of locating license plate of vehicle from the coloured image gathered；
Border, the candidate region obtaining step of vehicle window, obtains left boundary x=pl of license plate area, limit, the right according to license plate area
Boundary x=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor license plate area
Width, W is the width gathering image, λ 3 ＜ λ 2；
The candidate region output step of vehicle window, according to the left boundary of candidate region of vehicle window, border, the right, border, top, under
Border, limit, determines rectangular area, and this rectangular area is the candidate region of vehicle window；
Wherein, λ 1 ∈ [1.2,1.8], λ 2 ∈ [4.2,4.8], λ 3 ∈ [0.3,0.8].
7. the method for claim 1, it is characterised in that described third step includes:
Vertically edge obtaining step, carries out gray processing process by the candidate region of vehicle window, obtains the candidate region of gray scale, usesWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtaining step, uses threshold value Th_F logarithm value edge image to split, obtains bianry image；Straight-line detection
Step, uses Hough transform line detection algorithm to process bianry image, obtains the linear order y=k of detectionix+bi, i
=1,2 ..., N1, N1Quantity for straight line；
Straight line screening step, if arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, thus obtain remaining
Linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary obtaining step, scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bjFirst
Individual pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-directionStraight line is carried out point
Section, obtains the length on corresponding x direction, and puts it in LineHist array, uses clustering algorithm to LineHist number
Group clusters, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary obtaining step, scans coboundary, lower boundary respectively, by the first of coboundary pixel and the of lower boundary
The straight line of one pixel composition is as left margin, by last pixel and last pixel of lower boundary of coboundary
The straight line that point is constituted is as right margin；
Vehicle window area acquisition step, coboundary, left margin, right margin, lower boundary the region surrounded is vehicle window region；
Wherein, Th_F ∈ [10,30], Th_ θ ∈ [5 °, 15 °], Th_S ∈ [5,20].
8. the method for claim 1, it is characterised in that described 5th step includes:
Region of interest border obtaining step, obtains left boundary x=fl of human face region, border, the right x according to human face region
=fr, border, top y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isBorder, top isBorder is belowWfFor face
The width in region, W and H is respectively width and the height gathering image；
Area-of-interest obtaining step, according to the left boundary of the area-of-interest made a phone call, border, the right, border, top, under
Border, limit, determines rectangular area, and this rectangular area is area-of-interest；
Wherein, λ 4 ∈ [0.8,1.3], λ 5 ∈ [0.3,0.8], λ 6 ∈ [0.3,0.8].
9. the method for claim 1, it is characterised in that described 6th step includes:
Identify characteristic extraction step, utilize the detection model of making a phone call trained to extract the feature of area-of-interest；
Identify classification determination step, calculate the feature of area-of-interest and similarity Simi of each category featurek, k represents kth
Individual classification, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result exporting as driver.
10. a driver makes a phone call the detection device of behavior, it is characterised in that this device includes:
Make a phone call detection model acquisition module, be sample image for choosing the coloured image of label, use convolutional neural networks
Sample image is carried out repetition training, obtains the detection model of making a phone call trained；
The candidate region extraction module of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
Vehicle window region extraction module, for using Hough transform detection of straight lines in the candidate region of vehicle window, gathers straight line
Class processes, and extracts vehicle window region；
Human face region extraction module, is used for using Face datection algorithm to detect in vehicle window region, extracts human face region；
Region of interesting extraction module, for obtaining the area-of-interest made a phone call according to human face region；
Make a phone call detection model detection module, for utilizing the detection model of making a phone call trained that area-of-interest is examined
Survey, output detections result.
11. devices as claimed in claim 10, it is characterised in that described in detection model acquisition module of making a phone call include:
Module chosen by sample, for the colour choosing coloured image that label driver makes a phone call, label driver does not makes a phone call
The coloured image that image, label obscure is as sample image；
Initial training module, is used for utilizing convolutional neural networks that sample image is carried out features training, it is thus achieved that the mould of initial training
Type；
Second training module, is used for choosing test image, according to the model of initial training, test image is carried out repetition training, directly
Restrain to model；
Model output module, for making a phone call the model of convergence detection model exporting as the driver trained.
12. devices as claimed in claim 11, it is characterised in that described convolutional neural networks includes: input layer, Th_Con
Convolutional layer, Th_Pool pond layer, Th_Full full articulamentum；
Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*CKSi, step-length is 1；Every layer of pond
The size of the core of layer is KSi*KSi, step-length is KSi；The neuron of last layer of full articulamentum output of described full articulamentum
Quantity is 3, is 3 drivers and makes a phone call to detect classification；
Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,Th_CKmax], Th_
CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
13. devices as claimed in claim 12, it is characterised in that described convolutional neural networks includes:
Input layer, the image of input Width*Height；
Ground floor convolutional layer, exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length is 1；
Ground floor pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Second layer convolutional layer, exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length is 1；
Second layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Third layer convolutional layer, exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length is 1；
Third layer pond layer, using maximum pond method output KSi*KSi, step-length is the core of KSi；
Full articulamentum, uses ReLU as activation primitive, exports Th_Neur neuron；
Full articulamentum, exports 3 neurons, makes a phone call for i.e. 3 to detect classification；
Wherein, Width ∈ [64,192], Height ∈ [64,192]；Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈
[2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64,
10000]。
14. devices as claimed in claim 11, described second training module includes:
Training characteristics extraction module, tests the feature of image for the model extraction according to initial training；
Training classification determination module, makes a phone call to detect the similarity of category feature for calculating this feature and each driver
Simik, k represents kth classification, k={1,2,3}, choose SimikThe classification of value maximum is as couple candidate detection classification；
Repetition training module, for calculating the error of result of determination and legitimate reading, utilizes back-propagation algorithm to carry out training pattern,
Repetition training characteristic extracting module and training classification determination module, until the convergence of this model.
15. devices as claimed in claim 10, it is characterised in that the candidate region extraction module of described vehicle window includes:
License plate area locating module, for obtaining license plate area according to algorithm of locating license plate of vehicle from the coloured image gathered；Vehicle window
Border, candidate region acquisition module, for according to license plate area obtain left boundary x=pl of license plate area, border, the right x
=pr, border, top y=pt, following border y=pb, then the left boundary of the candidate region of vehicle window isBorder, the right isBorder, top isBorder is belowWpFor license plate area
Width, W is the width gathering image, λ 3 ＜ λ 2；
The candidate region acquisition module of vehicle window, for the left boundary of candidate region according to vehicle window, border, the right, limit, top
Boundary, following border, determine rectangular area, and this rectangular area is the candidate region of vehicle window；
Wherein, λ 1 ∈ [1.2,1.8], λ 2 ∈ [4.2,4.8], λ 3 ∈ [0.3,0.8].
16. devices as claimed in claim 10, it is characterised in that described vehicle window region extraction module includes:
Vertically edge acquisition module, for the candidate region of vehicle window carries out gray processing process, obtains the candidate region of gray scale,
UseWave filter, obtains the vertical edge image of the candidate region of gray scale；
Bianry image acquisition module, is used for using threshold value Th_F logarithm value edge image to split, and obtains bianry image；
Straight-line detection module, is used for using Hough transform line detection algorithm to process bianry image, obtains the straight of detection
Line sequence row y=kix+bi, i=1,2 ..., N1, N1Quantity for straight line；
Straight line shaker modeling block, if for arctan | ki|≤Th_ θ, then retain this straight line, otherwise delete this straight line, thus remained
Remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For remaining the quantity of straight line；
Up-and-down boundary acquisition module, is used for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bj's
First pixel (xj1,yj1) and last pixel (xj2,yj2), press in y-directionStraight line is entered
Row segmentation, obtains the length on corresponding x direction, and puts it in LineHist array, uses clustering algorithm pair
LineHist array clusters, using two maximum for the cluster value of acquisition straight lines as coboundary and lower boundary；
Right boundary acquisition module, for scanning coboundary, lower boundary respectively, by the first of coboundary pixel and lower boundary
The straight line that constitutes of first pixel as left margin, by last of last pixel of coboundary and lower boundary
The straight line that pixel is constituted is as right margin；
Vehicle window region acquisition module, is vehicle window region for the region surrounded by coboundary, left margin, right margin, lower boundary；
Wherein, Th_F ∈ [10,30], Th_ θ ∈ [5 °, 15 °], Th_S ∈ [5,20].
17. devices as claimed in claim 10, it is characterised in that described region of interesting extraction module includes: region of interest
Border, territory acquisition module, for obtaining left boundary x=fl of human face region, border, the right x=fr, top according to human face region
Border y=ft, following border y=fb, then the left boundary of the area-of-interest made a phone call isBorder, the right isBorder, top isBorder is belowWfFor human face region
Width, W and H be respectively gather image width and height；
Area-of-interest acquisition module, for according to the left boundary of area-of-interest made a phone call, border, the right, limit, top
Boundary, following border, determine rectangular area, and this rectangular area is area-of-interest；
Wherein, λ 4 ∈ [0.8,1.3], λ 5 ∈ [0.3,0.8], λ 6 ∈ [0.3,0.8].
18. devices as claimed in claim 10, it is characterised in that described in detection model detection module of making a phone call include:
Identify characteristic extracting module, for utilizing the detection model of making a phone call trained to extract the feature of area-of-interest；Identify
Classification determination module, for calculating the feature of area-of-interest and similarity Simi of each category featurek, k represents kth class
Not, k={1,2,3}, choose SimikThe maximum classification of value is made a phone call testing result exporting as driver.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610368797.XA CN106056071B (en) | 2016-05-30 | 2016-05-30 | A kind of driver makes a phone call the detection method and device of behavior |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610368797.XA CN106056071B (en) | 2016-05-30 | 2016-05-30 | A kind of driver makes a phone call the detection method and device of behavior |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106056071A true CN106056071A (en) | 2016-10-26 |
| CN106056071B CN106056071B (en) | 2019-05-10 |
Family
ID=57171292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610368797.XA Active CN106056071B (en) | 2016-05-30 | 2016-05-30 | A kind of driver makes a phone call the detection method and device of behavior |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106056071B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106503673A (en) * | 2016-11-03 | 2017-03-15 | 北京文安智能技术股份有限公司 | A kind of recognition methodss of traffic driving behavior, device and a kind of video acquisition device |
| CN106530730A (en) * | 2016-11-02 | 2017-03-22 | 重庆中科云丛科技有限公司 | Traffic violation detection method and system |
| CN106778583A (en) * | 2016-12-07 | 2017-05-31 | 北京理工大学 | Vehicle attribute recognition methods and device based on convolutional neural networks |
| CN107832728A (en) * | 2017-11-21 | 2018-03-23 | 天津英田视讯科技有限公司 | A kind of judge based on video makes a phone call Activity recognition method |
| CN108205649A (en) * | 2016-12-20 | 2018-06-26 | 浙江宇视科技有限公司 | Driver drives to take the state identification method and device of phone |
| CN108509902A (en) * | 2018-03-30 | 2018-09-07 | 湖北文理学院 | A kind of hand-held telephone relation behavioral value method during driver drives vehicle |
| CN108710857A (en) * | 2018-05-22 | 2018-10-26 | 深圳前海华夏智信数据科技有限公司 | People's vehicle recognition methods based on infrared light filling and device |
| CN108764034A (en) * | 2018-04-18 | 2018-11-06 | 浙江零跑科技有限公司 | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera |
| CN108875812A (en) * | 2018-06-01 | 2018-11-23 | 宁波工程学院 | A kind of driving behavior classification method based on branch's convolutional neural networks |
| CN109376634A (en) * | 2018-10-15 | 2019-02-22 | 北京航天控制仪器研究所 | A kind of Bus driver unlawful practice detection system neural network based |
| CN109410318A (en) * | 2018-09-30 | 2019-03-01 | 先临三维科技股份有限公司 | Threedimensional model generation method, device, equipment and storage medium |
| CN109583442A (en) * | 2018-11-07 | 2019-04-05 | 北京智芯原动科技有限公司 | False detection method of license plate and device based on Line segment detection |
| CN110059541A (en) * | 2019-02-28 | 2019-07-26 | 长江大学 | A kind of mobile phone usage behavior detection method and device in driving |
| CN110309764A (en) * | 2019-06-27 | 2019-10-08 | 浙江工业大学 | A kind of multistage driver based on deep learning makes a phone call behavioral value method |
| CN110839128A (en) * | 2018-08-16 | 2020-02-25 | 杭州海康威视数字技术股份有限公司 | Photographing behavior detection method and device and storage medium |
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2442538A1 (en) * | 2010-10-07 | 2012-04-18 | Research in Motion Limited | Method and system for preventing device operation when driving |
| CN102567743A (en) * | 2011-12-20 | 2012-07-11 | 东南大学 | Automatic identification method of driver gestures based on video images |
| CN103366506A (en) * | 2013-06-27 | 2013-10-23 | 北京理工大学 | Device and method for automatically monitoring telephone call behavior of driver when driving |
| CN104156717A (en) * | 2014-08-31 | 2014-11-19 | 王好贤 | Method for recognizing rule breaking of phoning of driver during driving based on image processing technology |
| CN104966059A (en) * | 2015-06-15 | 2015-10-07 | 安徽创世科技有限公司 | Method for detecting phoning behavior of driver during driving based on intelligent monitoring system |
| CN105005759A (en) * | 2015-05-04 | 2015-10-28 | 南京理工大学 | Multi-characteristic fused monitoring image front vehicle window positioning and extracting method |
-
2016
- 2016-05-30 CN CN201610368797.XA patent/CN106056071B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2442538A1 (en) * | 2010-10-07 | 2012-04-18 | Research in Motion Limited | Method and system for preventing device operation when driving |
| CN102567743A (en) * | 2011-12-20 | 2012-07-11 | 东南大学 | Automatic identification method of driver gestures based on video images |
| CN103366506A (en) * | 2013-06-27 | 2013-10-23 | 北京理工大学 | Device and method for automatically monitoring telephone call behavior of driver when driving |
| CN104156717A (en) * | 2014-08-31 | 2014-11-19 | 王好贤 | Method for recognizing rule breaking of phoning of driver during driving based on image processing technology |
| CN105005759A (en) * | 2015-05-04 | 2015-10-28 | 南京理工大学 | Multi-characteristic fused monitoring image front vehicle window positioning and extracting method |
| CN104966059A (en) * | 2015-06-15 | 2015-10-07 | 安徽创世科技有限公司 | Method for detecting phoning behavior of driver during driving based on intelligent monitoring system |
Non-Patent Citations (5)
| Title |
|---|
| DAN WANG ET AL；: "《Detecting Driver Use of Mobile Phone Based on In-car Camera》", 《2014 10TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND SECURITY》 * |
| 张波 等；: "《基于机器视觉的驾驶人使用手持电话行为检测》", 《吉林大学学报(工学版)》 * |
| 王丹 等；: "《基于机器视觉的驾驶员打电话行为检测》", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 * |
| 魏民国 等；: "《基于机器视觉的驾驶人使用手持电话行为检测方法》", 《2014 中国汽车安全技术学术会议》 * |
| 黎华东；: "《智能交通中的违章识别算法研究》", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530730A (en) * | 2016-11-02 | 2017-03-22 | 重庆中科云丛科技有限公司 | Traffic violation detection method and system |
| CN106503673A (en) * | 2016-11-03 | 2017-03-15 | 北京文安智能技术股份有限公司 | A kind of recognition methodss of traffic driving behavior, device and a kind of video acquisition device |
| CN106778583B (en) * | 2016-12-07 | 2019-12-17 | 北京理工大学 | Vehicle attribute identification method and device based on convolutional neural network |
| CN106778583A (en) * | 2016-12-07 | 2017-05-31 | 北京理工大学 | Vehicle attribute recognition methods and device based on convolutional neural networks |
| CN108205649A (en) * | 2016-12-20 | 2018-06-26 | 浙江宇视科技有限公司 | Driver drives to take the state identification method and device of phone |
| CN107832728A (en) * | 2017-11-21 | 2018-03-23 | 天津英田视讯科技有限公司 | A kind of judge based on video makes a phone call Activity recognition method |
| CN108509902A (en) * | 2018-03-30 | 2018-09-07 | 湖北文理学院 | A kind of hand-held telephone relation behavioral value method during driver drives vehicle |
| CN108509902B (en) * | 2018-03-30 | 2020-07-03 | 湖北文理学院 | Method for detecting call behavior of handheld phone in driving process of driver |
| CN108764034A (en) * | 2018-04-18 | 2018-11-06 | 浙江零跑科技有限公司 | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera |
| CN108710857A (en) * | 2018-05-22 | 2018-10-26 | 深圳前海华夏智信数据科技有限公司 | People's vehicle recognition methods based on infrared light filling and device |
| CN108875812A (en) * | 2018-06-01 | 2018-11-23 | 宁波工程学院 | A kind of driving behavior classification method based on branch's convolutional neural networks |
| CN108875812B (en) * | 2018-06-01 | 2021-10-22 | 宁波工程学院 | Driver behavior classification method based on branch convolutional neural network |
| CN110839128B (en) * | 2018-08-16 | 2021-04-27 | 杭州海康威视数字技术股份有限公司 | Photographing behavior detection method and device and storage medium |
| CN110839128A (en) * | 2018-08-16 | 2020-02-25 | 杭州海康威视数字技术股份有限公司 | Photographing behavior detection method and device and storage medium |
| US11978157B2 (en) | 2018-09-30 | 2024-05-07 | Shining 3D Tech Co., Ltd. | Method and apparatus for generating three-dimensional model, device, and storage medium |
| CN109410318A (en) * | 2018-09-30 | 2019-03-01 | 先临三维科技股份有限公司 | Threedimensional model generation method, device, equipment and storage medium |
| CN109376634A (en) * | 2018-10-15 | 2019-02-22 | 北京航天控制仪器研究所 | A kind of Bus driver unlawful practice detection system neural network based |
| CN109583442B (en) * | 2018-11-07 | 2020-12-11 | 北京智芯原动科技有限公司 | False license plate detection method and device based on line segment detection |
| CN109583442A (en) * | 2018-11-07 | 2019-04-05 | 北京智芯原动科技有限公司 | False detection method of license plate and device based on Line segment detection |
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
| CN111325058B (en) * | 2018-12-14 | 2023-12-01 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device, system and storage medium |
| CN110059541A (en) * | 2019-02-28 | 2019-07-26 | 长江大学 | A kind of mobile phone usage behavior detection method and device in driving |
| CN110309764A (en) * | 2019-06-27 | 2019-10-08 | 浙江工业大学 | A kind of multistage driver based on deep learning makes a phone call behavioral value method |
| CN110309764B (en) * | 2019-06-27 | 2021-06-01 | 浙江工业大学 | Multi-stage driver call-making behavior detection method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106056071B (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106056071A (en) | Method and device for detection of driver' behavior of making call | |
| CN102163284B (en) | Chinese environment-oriented complex scene text positioning method | |
| CN106022285A (en) | Vehicle type identification method and vehicle type identification device based on convolutional neural network | |
| CN103258204B (en) | A kind of automatic micro-expression recognition method based on Gabor and EOH feature | |
| Aich et al. | Global sum pooling: A generalization trick for object counting with small datasets of large images | |
| CN103996047B (en) | Hyperspectral image classification method based on squeezed spectra clustering ensemble | |
| CN106446811A (en) | Deep-learning-based driver's fatigue detection method and apparatus | |
| CN108491077A (en) | A kind of surface electromyogram signal gesture identification method for convolutional neural networks of being divided and ruled based on multithread | |
| CN107945153A (en) | A kind of road surface crack detection method based on deep learning | |
| CN105809173B (en) | A kind of image RSTN invariable attribute feature extraction and recognition methods based on bionical object visual transform | |
| CN101930549B (en) | Second generation curvelet transform-based static human detection method | |
| CN105608446A (en) | Video stream abnormal event detection method and apparatus | |
| CN106485224A (en) | A kind of seatbelt wearing recognition methodss and device | |
| CN106228166B (en) | The recognition methods of character picture | |
| CN107358260A (en) | A kind of Classification of Multispectral Images method based on surface wave CNN | |
| CN101383008A (en) | Image classification method based on visual attention model | |
| Chen et al. | Adaptive convolution for object detection | |
| CN107092884A (en) | Rapid coarse-fine cascade pedestrian detection method | |
| CN104008404B (en) | Pedestrian detection method and system based on significant histogram features | |
| CN109460787A (en) | IDS Framework method for building up, device and data processing equipment | |
| CN107633229A (en) | Method for detecting human face and device based on convolutional neural networks | |
| CN104834909B (en) | A kind of new image representation method based on Gabor comprehensive characteristics | |
| CN104298974A (en) | Human body behavior recognition method based on depth video sequence | |
| CN106778646A (en) | Model recognizing method and device based on convolutional neural networks | |
| CN104112138A (en) | Object color classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |