CN106056071B - A kind of driver makes a phone call the detection method and device of behavior - Google Patents

A kind of driver makes a phone call the detection method and device of behavior Download PDF

Info

Publication number
CN106056071B
CN106056071B CN201610368797.XA CN201610368797A CN106056071B CN 106056071 B CN106056071 B CN 106056071B CN 201610368797 A CN201610368797 A CN 201610368797A CN 106056071 B CN106056071 B CN 106056071B
Authority
CN
China
Prior art keywords
boundary
phone call
region
vehicle window
ksi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610368797.XA
Other languages
Chinese (zh)
Other versions
CN106056071A (en
Inventor
李志国
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zhi Xinyuandong Science And Technology Ltd
Original Assignee
Beijing Zhi Xinyuandong Science And Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zhi Xinyuandong Science And Technology Ltd filed Critical Beijing Zhi Xinyuandong Science And Technology Ltd
Priority to CN201610368797.XA priority Critical patent/CN106056071B/en
Publication of CN106056071A publication Critical patent/CN106056071A/en
Application granted granted Critical
Publication of CN106056071B publication Critical patent/CN106056071B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Neurology (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

It makes a phone call the detection method of behavior the present invention provides a kind of driver, this method comprises: the color image for choosing label is sample image, repetition training is carried out to sample image using convolutional neural networks, obtains trained detection model of making a phone call；The candidate region of vehicle window is obtained according to license plate area；Straight line is detected in the candidate region of vehicle window using Hough transform, clustering processing is carried out to straight line, extracts vehicle window region；It is detected in vehicle window region using Face datection algorithm, extracts human face region；The area-of-interest made a phone call is obtained according to human face region；Area-of-interest is detected using trained detection model of making a phone call, output test result.Compared with prior art, the present invention can accurately detect the behavior of making a phone call of driver, and robustness is preferable.

Description

A kind of driver makes a phone call the detection method and device of behavior
Technical field
The present invention relates to image procossing, video monitoring and intelligent transportation, in particular to driver's inspection for making a phone call behavior Survey method and device.
Background technique
With the development of transportation, traffic accident have become now endanger human life's safety main public hazards it One, while being also the serious social concern that our times various countries are faced.In traffic accident reason, drive Absent minded member is one of the main reasons.Report display according to statistics, when driving make a phone call can severe jamming driver note Anticipate power so that have a car accident Hazard ratio normal driving when high 4 times or more.
Currently, the research for behavioral value of making a phone call on the way for driver drives vehicle is also fewer, it is concentrated mainly on based on hand Machine signal is detected.Due to being difficult to differentiate, to be driver making a phone call or passenger is making a phone call, the side based on mobile phone signal Formula has many erroneous detections.With computer hardware and the skills such as software technology, image processing techniques and computer vision, pattern-recognition The development of art, the behavioral value of making a phone call based on image procossing is studied in recent years.
The existing behavioral value of making a phone call based on image procossing is based on classifier, such as Publication No. mostly The Chinese invention patent application of CN104573659A and CN102567743A is based on SVM (Support Vector Machines, support vector machines) classifier, the Chinese invention patent application of Publication No. CN104966059A is to be based on Cascade cascade classifier.However since the feature that classifier extracts is limited, the inspection for behavioral value of making a phone call is affected Survey accuracy rate.
In conclusion there is an urgent need to propose that a kind of higher driver of Detection accuracy makes a phone call the detection side of behavior at present Method and device.
Summary of the invention
In view of this, it is a primary object of the present invention to realize that driver makes a phone call the detection of behavior, and Detection accuracy It is higher.
In order to achieve the above objectives, first aspect according to the invention provides a kind of driver and makes a phone call the inspection of behavior Survey method, this method comprises:
First step, the color image for choosing label is sample image, is carried out using convolutional neural networks to sample image Repetition training obtains trained detection model of making a phone call；
Second step obtains the candidate region of vehicle window according to license plate area；
Third step detects straight line in the candidate region of vehicle window using Hough transform, carries out clustering processing to straight line, Extract vehicle window region；
Four steps is detected in vehicle window region using Face datection algorithm, extracts human face region；
5th step obtains the area-of-interest made a phone call according to human face region；
6th step detects area-of-interest using trained detection model of making a phone call, output test result.
The first step further comprises:
Sample selecting step chooses the color image made a phone call of label driver, the colour that label driver does not make a phone call The fuzzy color image of image, label is as sample image；
Initial training step carries out feature training to sample image using convolutional neural networks, obtains the mould of initial training Type；
Second training step chooses test image, carries out repetition training to test image according to the model of initial training, directly It is restrained to model；
Model exports step, makes a phone call detection model and to export using convergent model as trained driver.
Convolutional neural networks include: input layer, Th_Con convolutional layer, Th_Pool pond in the initial training step Change layer, Th_Full full articulamentum.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi* CKSi, step-length 1.The size of the core of every layer of pond layer is KSi*KSi, step-length KSi.The last layer of the full articulamentum is complete The quantity of the neuron of articulamentum output is 3, and as 3 drivers make a phone call to detect classification.
Further, the convolutional neural networks include:
Input layer inputs the image of Width*Height；
First layer convolutional layer exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length 1；
First layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Second layer convolutional layer exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length 1；
Second layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Third layer convolutional layer exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length 1；
Third layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Full articulamentum exports Th_Neur neuron using ReLU as activation primitive；
Full articulamentum exports 3 neurons, i.e., 3 are made a phone call to detect classification.
The second training step further comprises:
Training characteristics extraction step, according to the feature of the model extraction test image of initial training；
Training classification determination step, calculates this feature and each driver makes a phone call to detect the similarity of category feature Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification as couple candidate detection classification；
Repetition training step calculates the error for determining result and legitimate reading, using back-propagation algorithm come training pattern, Repetition training characteristic extraction step and training classification determination step, until the model is restrained.
The second step further comprises:
License plate area positioning step obtains license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle；
The candidate region boundary obtaining step of vehicle window obtains left boundary x=pl, the right side of license plate area according to license plate area Side boundary x=pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop boundary ForFollowing boundary isWpFor license plate The width in region, W are the width for acquiring image, 3 < λ 2 of λ；
The candidate region of vehicle window exports step, according to the left boundary of the candidate region of vehicle window, the right boundary, top side Boundary, following boundary, determine rectangular area, which is the candidate region of vehicle window.
The third step further comprises:
The candidate region of vehicle window is carried out gray processing processing, obtains the candidate region of gray scale, adopt by vertical edge obtaining step WithFilter obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtaining step is split using threshold value Th_F logarithm edge image, obtains bianry image；
Straight-line detection step is handled bianry image using Hough transform line detection algorithm, obtains the straight of detection Line sequence column y=kix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
Straight line screening step, if arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, thus obtains Remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
Up-and-down boundary obtaining step scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bj's First pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-directionBy straight line into Row segmentation, obtains the length on the corresponding direction x, and put it into LineHist array, using clustering algorithm pair LineHist array is clustered, and maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
Right boundary obtaining step scans coboundary, lower boundary, by first pixel and lower boundary of coboundary respectively First pixel constitute straight line as left margin, by the last one of the last one pixel of coboundary and lower boundary The straight line that pixel is constituted is as right margin；
Vehicle window area acquisition step is vehicle window region by the region that coboundary, left margin, right margin, lower boundary surround.
5th step further comprises:
Region of interest border obtaining step obtains left boundary x=fl, the right side of human face region according to human face region Boundary x=fr, top boundary y=ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areThe right boundary isTop boundary ForFollowing boundary isWfFor people The width in face region, W and H are respectively the width and height for acquiring image；
Area-of-interest obtaining step, according to the left boundary for the area-of-interest made a phone call, the right boundary, top side Boundary, following boundary, determine rectangular area, which is area-of-interest.
6th step further comprises:
Identification feature extraction step extracts the feature of area-of-interest using trained detection model of making a phone call；
Identification classification determination step, calculates the feature of area-of-interest and the similarity Simi of each category featurek, k table Show k-th of classification, k={ 1,2,3 } chooses SimikIt is worth maximum classification as driver to make a phone call testing result and to export.
Other side according to the invention provides a kind of driver and makes a phone call the detection device of behavior, the device packet It includes:
Make a phone call detection model obtain module, for choose label color image be sample image, using convolutional Neural Network carries out repetition training to sample image, obtains trained detection model of making a phone call；
The candidate region extraction module of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
Vehicle window region extraction module, for detecting straight line in the candidate region of vehicle window using Hough transform, to straight line into Row clustering processing extracts vehicle window region；
Human face region extraction module extracts face area for being detected in vehicle window region using Face datection algorithm Domain；
Region of interesting extraction module, for obtaining the area-of-interest made a phone call according to human face region；
It makes a phone call detection model detection module, for being carried out using trained detection model of making a phone call to area-of-interest Detection, output test result.
The detection model of making a phone call obtains module and further comprises:
Sample chooses module, and for choosing the color image that label driver makes a phone call, label driver does not make a phone call The fuzzy color image of color image, label is as sample image；
Initial training module obtains initial training for carrying out feature training to sample image using convolutional neural networks Model；
Second training module instructs test image according to the model of initial training for choosing test image repeatedly Practice, until model is restrained；
Model output module, for making a phone call detection model and to export using convergent model as trained driver.
The second training module further comprises:
Training characteristics extraction module, for the feature according to the model extraction test image of initial training；
Training classification determination module, for calculating this feature and each driver makes a phone call to detect the similarity of category feature Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification as couple candidate detection classification；
Repetition training module is trained for calculating the error for determining result and legitimate reading using back-propagation algorithm Model, repetition training characteristic extracting module and training classification determination module, until the model is restrained.
The candidate region extraction module of the vehicle window further comprises:
License plate area locating module, for obtaining license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle；
The candidate region boundary of vehicle window obtains module, for obtaining the left boundary x=of license plate area according to license plate area Pl, the right boundary x=pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop side Boundary isFollowing boundary isWpFor vehicle The width in board region, W are the width for acquiring image, 3 < λ 2 of λ；
The candidate region of vehicle window obtains module, for the left boundary according to the candidate region of vehicle window, the right boundary, top Boundary, following boundary, determine rectangular area, which is the candidate region of vehicle window.
The vehicle window region extraction module further comprises:
Vertical edge obtains module, for the candidate region of vehicle window to be carried out gray processing processing, obtains the candidate regions of gray scale Domain usesFilter obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtains module, for being split using threshold value Th_F logarithm edge image, obtains bianry image；
Straight-line detection module obtains detection for handling using Hough transform line detection algorithm bianry image Linear order y=kix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
Straight line screening module, if being used for arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, thus Obtain remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
Up-and-down boundary obtains module, for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx +bjFirst pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-directionIt will be straight Line is segmented, and the length on the corresponding direction x is obtained, and put it into LineHist array, using clustering algorithm pair LineHist array is clustered, and maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
Right boundary obtains module, for scanning coboundary, lower boundary respectively, by first pixel of coboundary under The straight line that first pixel on boundary is constituted is as left margin, by the last of the last one pixel of coboundary and lower boundary The straight line that one pixel is constituted is as right margin；
Vehicle window region obtains module, and the region for being surrounded by coboundary, left margin, right margin, lower boundary is vehicle window Region.
The region of interesting extraction module further comprises:
Region of interest border obtains module, for obtaining left boundary x=fl, the right side of human face region according to human face region Side boundary x=fr, top boundary y=ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areThe right boundary isTop side Boundary isFollowing boundary isWfFor The width of human face region, W and H are respectively the width and height for acquiring image；
Area-of-interest obtains module, for the left boundary according to the area-of-interest made a phone call, the right boundary, top Boundary, following boundary, determine rectangular area, which is area-of-interest.
The detection model detection module of making a phone call further comprises:
Identification feature extraction module, for extracting the feature of area-of-interest using trained detection model of making a phone call；
Identification classification determination module, for calculating the feature of area-of-interest and the similarity Simi of each category featurek, K indicates k-th of classification, and k={ 1,2,3 } chooses SimikIt is worth maximum classification to make a phone call testing result and defeated as driver Out.
With existing driver make a phone call behavior detection technique compared with, a kind of driver of the invention makes a phone call behavior Detection method and device use convolutional neural networks, can go out the behavior that driver makes a phone call with accurate detection, and robustness compared with It is good；In addition, the convolutional neural networks in the present invention have trained, driver makes a phone call, driver does not make a phone call, obscures three classifications, Fuzzy situation can be told.
Detailed description of the invention
Fig. 1 show a kind of driver according to the invention make a phone call behavior detection method flow chart.
It makes a phone call the frame diagram of the detection device of behavior Fig. 2 shows a kind of driver according to the invention.
Specific embodiment
To enable your auditor to further appreciate that structure of the invention, feature and other purposes, now in conjunction with appended preferable reality Applying example, detailed description are as follows, and illustrated preferred embodiment is only used to illustrate the technical scheme of the present invention, and the non-limiting present invention.
Fig. 1 give a kind of driver according to the invention make a phone call behavior detection method flow chart.Such as Fig. 1 institute Show, a kind of the make a phone call detection method of behavior of driver according to the invention includes:
First step S1, choose label color image be sample image, using convolutional neural networks to sample image into Row repetition training obtains trained detection model of making a phone call；
Second step S2 obtains the candidate region of vehicle window according to license plate area；
Third step S3 detects straight line in the candidate region of vehicle window using Hough transform, carries out at cluster to straight line Reason extracts vehicle window region；
Four steps S4 is detected in vehicle window region using Face datection algorithm, extracts human face region；
5th step S5 obtains the area-of-interest made a phone call according to human face region；
6th step S6 detects area-of-interest using trained detection model of making a phone call, output detection knot Fruit.
The first step S1 further comprises:
Sample selecting step S11 chooses the color image made a phone call of label driver, the coloured silk that label driver does not make a phone call The fuzzy color image of chromatic graph picture, label is as sample image；
Initial training step S12 carries out feature training to sample image using convolutional neural networks, obtains initial training Model；
Second training step S13 chooses test image, is instructed repeatedly according to the model of initial training to test image Practice, until model is restrained；
Model exports step S14, makes a phone call detection model and to export using convergent model as trained driver.
Wherein, the width of sample image is Width, is highly Height in the sample selecting step S11.Width∈ [64,192], Height ∈ [64,192].Preferably, Width is selected as 128, Height and is selected as 128.The label driver of selection The color image quantity made a phone call can be greater than 500, and the color image quantity that the label driver of selection does not make a phone call can be big In 500, the fuzzy color image quantity of the label of selection can be greater than 500.Preferably, 1000~5000000 are chosen respectively The color image that label driver makes a phone call, driver does not make a phone call, obscures is sample image.
Convolutional neural networks include: input layer, Th_Con convolutional layer, Th_Pool in the initial training step S12 Pond layer, Th_Full full articulamentums.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi* CKSi, step-length 1.The size of the core of every layer of pond layer is KSi*KSi, step-length KSi.The last layer of the full articulamentum is complete The quantity of the neuron of articulamentum output is 3, and as 3 drivers make a phone call to detect classification.
Wherein, the Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin, Th_CKmax], Th_CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
Further, the convolutional neural networks include:
Input layer inputs the image of Width*Height；
First layer convolutional layer exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length 1；
First layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Second layer convolutional layer exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length 1；
Second layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Third layer convolutional layer exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length 1；
Third layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Full articulamentum exports Th_Neur neuron using ReLU as activation primitive；
Full articulamentum exports 3 neurons, i.e., 3 are made a phone call to detect classification.
Wherein, [6,20] Th_CK1 ∈, CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3, 5], [6,40] Th_CK3 ∈, CKSi3 ∈ [3,5], Th_Neur ∈ [64,10000].
Preferably, Th_CK1 is set as 8, CKSi1 and is set as 5, KSi being set as 2, Th_CK2 and be set as 8, CKSi2 being set as 5, Th_CK3 It is set as 8, CKSi3 and is set as 5, Th_Neur being set as 128.
The first layer, the second layer, third layer pond layer in maximum Chi Huafa could alternatively be average Chi Huafa.
ReLU full name is Rectified Linear Units in the full articulamentum, and Chinese is translated into amendment linear unit, It can be with bibliography " Taming the ReLU with Parallel Dither in a Deep Neural Network.AJR Simpson.Computer Science,2015”。
ReLU could alternatively be sigmoid function or tanh function as activation primitive in the full articulamentum.
The second training step S13 further comprises:
Training characteristics extraction step S131, according to the feature of the model extraction test image of initial training；
Training classification determination step S132, calculates this feature and each driver makes a phone call to detect the similarity of category feature Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification as couple candidate detection classification；
Repetition training step S133 is calculated the error for determining result and legitimate reading, is trained using back-propagation algorithm Model, repetition training characteristic extraction step S131 and training classification determination step S132, until the model is restrained.
The second step S2 further comprises:
License plate area positioning step S21 obtains license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle；
The candidate region boundary obtaining step S22 of vehicle window obtains the left boundary x=of license plate area according to license plate area Pl, the right boundary x=pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop side Boundary isFollowing boundary isWpFor vehicle The width in board region, W are the width for acquiring image, 3 < λ 2 of λ；
The candidate region of vehicle window exports step S23, according to the left boundary of the candidate region of vehicle window, the right boundary, top Boundary, following boundary, determine rectangular area, which is the candidate region of vehicle window.
Algorithm of locating license plate of vehicle is existing algorithm of locating license plate of vehicle in the license plate area positioning step S21.For example, " Li Wen It lifts, Liang Dequn, Zhang Qi, new location method of vehicle license plate " Chinese journal of computers " of the Fan Xin based on edge color pair, 2004,27 (2): 204-208”。
1 ∈ of λ [1.2,1.8], 2 ∈ of λ [4.2,4.8], 3 ∈ of λ [0.3,0.8].Preferably, λ 1 is selected as 1.5, λ 2 and is selected as 4.5, λ 3 are selected as 0.5.
The third step S3 further comprises:
The candidate region of vehicle window is carried out gray processing processing, obtains the candidate regions of gray scale by vertical edge obtaining step S31 Domain usesFilter obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtaining step S32 is split using threshold value Th_F logarithm edge image, obtains bianry image；
Straight-line detection step S33, is handled bianry image using Hough transform line detection algorithm, obtains detection Linear order y=kix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
Straight line screening step S34, if arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, thus To remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
Up-and-down boundary obtaining step S35 scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+ bjFirst pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-directionIt will be straight Line is segmented, and the length on the corresponding direction x is obtained, and put it into LineHist array, using clustering algorithm pair LineHist array is clustered, and maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
Right boundary obtaining step S36 scans coboundary, lower boundary respectively, by first pixel of coboundary under The straight line that first pixel on boundary is constituted is as left margin, by the last of the last one pixel of coboundary and lower boundary The straight line that one pixel is constituted is as right margin；
Vehicle window area acquisition step S37 is vehicle window area by the region that coboundary, left margin, right margin, lower boundary surround Domain.
Wherein, Th_F ∈ [10,30] in the bianry image obtaining step S32.Preferably, Th_F is selected as 20.
Hough transform line detection algorithm is realized by existing technology in the straight-line detection step S33.For example, " section You is tender, Zhao Wei, yellow pine ridge, a kind of straight line fast algorithm of detecting " instrument and meter based on Improved Hough Transform of Chen Jianye Report ", 2010,31 (12): 2774-2780 ".
Th_ θ ∈ [5 °, 15 °] in the straight line screening step S34.Preferably.Th_ θ is selected as 10 °.
Th_S is sampling interval, Th_S ∈ [5,20] in the up-and-down boundary obtaining step S35.Preferably, Th_S is selected as 12.The clustering algorithm is realized by existing clustering algorithm.
Face datection algorithm in the four steps S4 can be realized by existing technology.For example, " Guo Zhibo, China After encouraging, Yan Yunyang, old ability button, Yang Jingyu based on the face of dual threshold succession type AdaBoost algorithm quickly detects, and " data are adopted Collection and processing ", 2008,23 (3): 306-310 ".
The 5th step S5 further comprises:
Region of interest border obtaining step S51 obtains left boundary x=fl, the right side of human face region according to human face region Side boundary x=fr, top boundary y=ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areThe right boundary isTop side Boundary isFollowing boundary isWfFor The width of human face region, W and H are respectively the width and height for acquiring image；
Area-of-interest obtaining step S52, according to the left boundary for the area-of-interest made a phone call, the right boundary, top Boundary, following boundary, determine rectangular area, which is area-of-interest.
Wherein, 4 ∈ of λ [0.8,1.3], 5 ∈ of λ [0.3,0.8], 6 ∈ of λ [0.3,0.8].Preferably, λ 4 is selected as 1, λ 5 It is selected as 0.5, λ 6 and is selected as 0.5.
The 6th step S6 further comprises:
Identification feature extraction step S61 extracts the feature of area-of-interest using trained detection model of making a phone call；
Identification classification determination step S62, calculates the feature of area-of-interest and the similarity Simi of each category featurek, k Indicate k-th of classification, k={ 1,2,3 } chooses SimikIt is worth maximum classification as driver to make a phone call testing result and to export.
Fig. 2 give a kind of driver according to the invention make a phone call behavior detection device frame diagram.Such as Fig. 2 institute Show, a kind of the make a phone call detection device of behavior of driver according to the invention includes:
Make a phone call detection model obtain module 1, for choose label color image be sample image, using convolutional Neural Network carries out repetition training to sample image, obtains trained detection model of making a phone call；
The candidate region extraction module 2 of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
Vehicle window region extraction module 3, for detecting straight line in the candidate region of vehicle window using Hough transform, to straight line Clustering processing is carried out, vehicle window region is extracted；
Human face region extraction module 4 extracts face area for being detected in vehicle window region using Face datection algorithm Domain；
Region of interesting extraction module 5, for obtaining the area-of-interest made a phone call according to human face region；
Make a phone call detection model detection module 6, for using it is trained make a phone call detection model to area-of-interest into Row detection, output test result.
The detection model of making a phone call obtains module 1 and further comprises:
Sample chooses module 11, and for choosing the color image that label driver makes a phone call, label driver does not make a phone call Color image, the fuzzy color image of label is as sample image；
Initial training module 12 obtains preliminary instruction for carrying out feature training to sample image using convolutional neural networks Experienced model；
Second training module 13 carries out repeatedly test image according to the model of initial training for choosing test image Training, until model is restrained；
Model output module 14, for making a phone call detection model and defeated using convergent model as trained driver Out.
Wherein, the width that the sample chooses sample image in module 11 is Width, is highly Height.Width∈ [64,192], Height ∈ [64,192].Preferably, Width is selected as 128, Height and is selected as 128.The label driver of selection The color image quantity made a phone call can be greater than 500, and the color image quantity that the label driver of selection does not make a phone call can be big In 500, the fuzzy color image quantity of the label of selection can be greater than 500.Preferably, 1000~5000000 are chosen respectively The color image that label driver makes a phone call, driver does not make a phone call, obscures is sample image.
Convolutional neural networks include: input layer, Th_Con convolutional layer, Th_Pool in the initial training module 12 Pond layer, Th_Full full articulamentums.Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi* CKSi, step-length 1.The size of the core of every layer of pond layer is KSi*KSi, step-length KSi.The last layer of the full articulamentum is complete The quantity of the neuron of articulamentum output is 3, and as 3 drivers make a phone call to detect classification.
Wherein, the Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin, Th_CKmax], Th_CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
Further, the convolutional neural networks include:
Input layer inputs the image of Width*Height；
First layer convolutional layer exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length 1；
First layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Second layer convolutional layer exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length 1；
Second layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Third layer convolutional layer exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length 1；
Third layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
Full articulamentum exports Th_Neur neuron using ReLU as activation primitive；
Full articulamentum exports 3 neurons, i.e., 3 are made a phone call to detect classification.
Wherein, [6,20] Th_CK1 ∈, CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3, 5], [6,40] Th_CK3 ∈, CKSi3 ∈ [3,5], Th_Neur ∈ [64,10000].
Preferably, Th_CK1 is set as 8, CKSi1 and is set as 5, KSi being set as 2, Th_CK2 and be set as 8, CKSi2 being set as 5, Th_CK3 It is set as 8, CKSi3 and is set as 5, Th_Neur being set as 128.
The first layer, the second layer, third layer pond layer in maximum Chi Huafa could alternatively be average Chi Huafa.
ReLU full name is Rectified Linear Units in the full articulamentum, and Chinese is translated into amendment linear unit, It can be with bibliography " Taming the ReLU with Parallel Dither in a Deep Neural Network.AJR Simpson.Computer Science,2015”。
ReLU could alternatively be sigmoid function or tanh function as activation primitive in the full articulamentum.
The second training module 13 further comprises:
Training characteristics extraction module 131, for the feature according to the model extraction test image of initial training；
Training classification determination module 132 makes a phone call to detect the phase of category feature for calculating this feature and each driver Like degree Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification as couple candidate detection classification；
Repetition training module 133 is instructed for calculating the error for determining result and legitimate reading using back-propagation algorithm Practice model, repetition training characteristic extracting module 131 and training classification determination module 132, until the model is restrained.
The candidate region extraction module 2 of the vehicle window further comprises:
License plate area locating module 21, for obtaining license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle Domain；
The candidate region boundary of vehicle window obtains module 22, for obtaining the left boundary x of license plate area according to license plate area =pl, the right boundary x=pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop side Boundary isFollowing boundary isWpFor vehicle The width in board region, W are the width for acquiring image, 3 < λ 2 of λ；
The candidate region of vehicle window obtains module 23, for according to the left boundary of the candidate region of vehicle window, the right boundary, Side boundary, following boundary, determine rectangular area, which is the candidate region of vehicle window.
Algorithm of locating license plate of vehicle is existing algorithm of locating license plate of vehicle in the license plate area locating module 21.For example, " Li Wen It lifts, Liang Dequn, Zhang Qi, new location method of vehicle license plate " Chinese journal of computers " of the Fan Xin based on edge color pair, 2004,27 (2): 204-208”。
1 ∈ of λ [1.2,1.8], 2 ∈ of λ [4.2,4.8], 3 ∈ of λ [0.3,0.8].Preferably, λ 1 is selected as 1.5, λ 2 and is selected as 4.5, λ 3 are selected as 0.5.
The vehicle window region extraction module 3 further comprises:
Vertical edge obtains module 31, for the candidate region of vehicle window to be carried out gray processing processing, obtains the candidate of gray scale Region usesFilter obtains the vertical edge image of the candidate region of gray scale；
Bianry image obtains module 32, for being split using threshold value Th_F logarithm edge image, obtains binary map Picture；
Straight-line detection module 33 obtains inspection for handling using Hough transform line detection algorithm bianry image The linear order y=k of surveyix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
Straight line screening module 34, if being used for arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, by This obtains remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
Up-and-down boundary obtains module 35, for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y= kjx+bjFirst pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-direction Straight line is segmented, the length on the corresponding direction x is obtained, and put it into LineHist array, using clustering algorithm LineHist array is clustered, maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
Right boundary obtains module 36, for scanning coboundary, lower boundary respectively, by first pixel of coboundary and The straight line that first pixel of lower boundary is constituted is as left margin, most by the last one pixel and lower boundary of coboundary The straight line that the latter pixel is constituted is as right margin；
Vehicle window region obtains module 37, and the region for being surrounded by coboundary, left margin, right margin, lower boundary is vehicle Window region.
Wherein, the bianry image obtains Th_F ∈ [10,30] in module 32.Preferably, Th_F is selected as 20.
Hough transform line detection algorithm is realized by existing technology in the straight-line detection module 33.For example, " section You is tender, Zhao Wei, yellow pine ridge, a kind of straight line fast algorithm of detecting " instrument and meter based on Improved Hough Transform of Chen Jianye Report ", 2010,31 (12): 2774-2780 ".
Th_ θ ∈ [5 °, 15 °] in the straight line screening module 34.Preferably.Th_ θ is selected as 10 °.
It is sampling interval, Th_S ∈ [5,20] that the up-and-down boundary, which obtains Th_S in module 35,.Preferably, Th_S is selected as 12.The clustering algorithm is realized by existing clustering algorithm.
Face datection algorithm in the human face region extraction module 4 can be realized by existing technology.For example, " Guo Will wave, Hua Jizhao, Yan Yunyang, old ability button, Yang Jingyu quickly detect based on the face of dual threshold succession type AdaBoost algorithm " data acquisition and procession ", 2008,23 (3): 306-310 ".
The region of interesting extraction module 5 further comprises:
Region of interest border obtain module 51, for according to human face region obtain human face region left boundary x=fl, The right boundary x=fr, top boundary y=ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areThe right boundary isTop side Boundary isFollowing boundary isWfFor The width of human face region, W and H are respectively the width and height for acquiring image；
Area-of-interest obtains module 52, for according to the left boundary of the area-of-interest made a phone call, the right boundary, Side boundary, following boundary, determine rectangular area, which is area-of-interest.
Wherein, 4 ∈ of λ [0.8,1.3], 5 ∈ of λ [0.3,0.8], 6 ∈ of λ [0.3,0.8].Preferably, λ 4 is selected as 1, λ 5 It is selected as 0.5, λ 6 and is selected as 0.5.
The detection model detection module 6 of making a phone call further comprises:
Identification feature extraction module 61, for extracting the spy of area-of-interest using trained detection model of making a phone call Sign；
Identification classification determination module 62, for calculating the feature of area-of-interest and the similarity of each category feature Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification to make a phone call testing result as driver And it exports.
With existing driver make a phone call behavior detection technique compared with, a kind of driver of the invention makes a phone call behavior Detection method and device use convolutional neural networks, can go out the behavior that driver makes a phone call with accurate detection, and robustness compared with It is good；In addition, the convolutional neural networks in the present invention have trained, driver makes a phone call, driver does not make a phone call, obscures three classifications, Fuzzy situation can be told.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention, should Understand, the present invention is not limited to implementation as described herein, the purpose of these implementations description is to help this field In technical staff practice the present invention.Any those of skill in the art are easy to do not departing from spirit and scope of the invention In the case of be further improved and perfect, therefore the present invention is only by the content of the claims in the present invention and the limit of range System, intention, which covers, all to be included the alternative in the spirit and scope of the invention being defined by the appended claims and waits Same scheme.

Claims (16)

  1. The detection method of behavior 1. a kind of driver makes a phone call, which is characterized in that this method comprises:
    First step, the color image for choosing label is sample image, is carried out repeatedly using convolutional neural networks to sample image Training obtains trained detection model of making a phone call；
    Second step obtains the candidate region of vehicle window according to license plate area；
    Third step detects straight line in the candidate region of vehicle window using Hough transform, carries out clustering processing to straight line, extracts Vehicle window region；
    Four steps is detected in vehicle window region using Face datection algorithm, extracts human face region；
    5th step obtains the area-of-interest made a phone call according to human face region；
    6th step detects area-of-interest using trained detection model of making a phone call, output test result；
    Wherein, the second step includes:
    License plate area positioning step obtains license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle；
    The candidate region boundary obtaining step of vehicle window obtains left boundary x=pl, the right side of license plate area according to license plate area Boundary x=pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop boundary isFollowing boundary isWpFor license plate area The width in domain, W are the width for acquiring image, 3 < λ 2 of λ；
    The candidate region of vehicle window exports step, according to the left boundary of the candidate region of vehicle window, the right boundary, top boundary, under Side boundary, determines rectangular area, which is the candidate region of vehicle window；
    Wherein, 3 ∈ of 2 ∈ of 1 ∈ of λ [1.2,1.8], λ [4.2,4.8], λ [0.3,0.8].
  2. 2. the method as described in claim 1, which is characterized in that the first step includes:
    Sample selecting step, choose the color image made a phone call of label driver, the color image that label driver does not make a phone call, The fuzzy color image of label is as sample image；
    Initial training step carries out feature training to sample image using convolutional neural networks, obtains the model of initial training；
    Second training step chooses test image, repetition training is carried out to test image according to the model of initial training, until mould Type convergence；
    Model exports step, makes a phone call detection model and to export using convergent model as trained driver.
  3. 3. method according to claim 2, which is characterized in that the convolutional neural networks include: input layer, Th_Con volume Lamination, Th_Pool pond layer, Th_Full full articulamentums；
    Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*CKSi, step-length 1；Every layer of pond Layer core size be KSi*KSi, step-length KSi；The neuron of the full articulamentum output of the last layer of the full articulamentum Quantity is 3, and as 3 drivers make a phone call to detect classification；
    Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,Th_CKmax], Th_ CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
  4. 4. method as claimed in claim 3, which is characterized in that the convolutional neural networks include:
    Input layer inputs the image of Width*Height；
    First layer convolutional layer exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length 1；
    First layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Second layer convolutional layer exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length 1；
    Second layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Third layer convolutional layer exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length 1；
    Third layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Full articulamentum exports Th_Neur neuron using ReLU as activation primitive；
    Full articulamentum exports 3 neurons, i.e., 3 are made a phone call to detect classification；
    Wherein, [64,192] Width ∈, Height ∈ [64,192]；Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64, 10000]。
  5. 5. method according to claim 2, the second training step include:
    Training characteristics extraction step, according to the feature of the model extraction test image of initial training；
    Training classification determination step, calculates this feature and each driver makes a phone call to detect the similarity Simi of category featurek, k table Show k-th of classification, k={ 1,2,3 } chooses SimikIt is worth maximum classification as couple candidate detection classification；
    Repetition training step calculates the error for determining result and legitimate reading, using back-propagation algorithm come training pattern, repeats Training characteristics extraction step and training classification determination step, until the model is restrained.
  6. 6. the method as described in claim 1, which is characterized in that the third step includes:
    The candidate region of vehicle window is carried out gray processing processing, obtains the candidate region of gray scale, used by vertical edge obtaining stepFilter obtains the vertical edge image of the candidate region of gray scale；
    Bianry image obtaining step is split using threshold value Th_F logarithm edge image, obtains bianry image；
    Straight-line detection step is handled bianry image using Hough transform line detection algorithm, obtains the straight line sequence of detection Arrange y=kix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
    Straight line screening step, if arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, thus obtains remaining Linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
    Up-and-down boundary obtaining step scans every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bjFirst A pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-directionStraight line is divided Section, obtains the length on the corresponding direction x, and put it into LineHist array, using clustering algorithm to LineHist number Group is clustered, and maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
    Right boundary obtaining step scans coboundary, lower boundary respectively, by the of first pixel of coboundary and lower boundary The straight line that one pixel is constituted is as left margin, by the last one pixel of the last one pixel of coboundary and lower boundary The straight line that point is constituted is as right margin；
    Vehicle window area acquisition step is vehicle window region by the region that coboundary, left margin, right margin, lower boundary surround；
    Wherein, [10,30] Th_F ∈, Th_ θ ∈ [5 °, 15 °], Th_S ∈ [5,20].
  7. 7. the method as described in claim 1, which is characterized in that the 5th step includes:
    Region of interest border obtaining step obtains left boundary x=fl, the right boundary x of human face region according to human face region =fr, top boundary y=ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areThe right boundary isTop boundary ForFollowing boundary isWfFor people The width in face region, W and H are respectively the width and height for acquiring image；
    Area-of-interest obtaining step, according to the left boundary for the area-of-interest made a phone call, the right boundary, top boundary, under Side boundary, determines rectangular area, which is area-of-interest；
    Wherein, 6 ∈ of 5 ∈ of 4 ∈ of λ [0.8,1.3], λ [0.3,0.8], λ [0.3,0.8].
  8. 8. the method as described in claim 1, which is characterized in that the 6th step includes:
    Identification feature extraction step extracts the feature of area-of-interest using trained detection model of making a phone call；
    Identification classification determination step, calculates the feature of area-of-interest and the similarity Simi of each category featurek, k expression kth A classification, k={ 1,2,3 } choose SimikIt is worth maximum classification as driver to make a phone call testing result and to export.
  9. The detection device of behavior 9. a kind of driver makes a phone call, which is characterized in that the device includes:
    Make a phone call detection model obtain module, for choose label color image be sample image, using convolutional neural networks Repetition training is carried out to sample image, obtains trained detection model of making a phone call；
    The candidate region extraction module of vehicle window, for obtaining the candidate region of vehicle window according to license plate area；
    Vehicle window region extraction module gathers straight line for detecting straight line in the candidate region of vehicle window using Hough transform Class processing, extracts vehicle window region；
    Human face region extraction module extracts human face region for being detected in vehicle window region using Face datection algorithm；
    Region of interesting extraction module, for obtaining the area-of-interest made a phone call according to human face region；
    It makes a phone call detection model detection module, for being examined using trained detection model of making a phone call to area-of-interest It surveys, output test result；
    Wherein, the candidate region extraction module of the vehicle window includes:
    License plate area locating module, for obtaining license plate area from the color image of acquisition according to algorithm of locating license plate of vehicle；Vehicle window Candidate region boundary obtain module, for according to license plate area obtain license plate area left boundary x=pl, the right boundary x =pr, top boundary y=pt, following boundary y=pb, then the left boundary of the candidate region of vehicle window beThe right boundary isTop boundary ForFollowing boundary isWpFor license plate The width in region, W are the width for acquiring image, 3 < λ 2 of λ；
    The candidate region of vehicle window obtains module, for the left boundary according to the candidate region of vehicle window, the right boundary, top side Boundary, following boundary, determine rectangular area, which is the candidate region of vehicle window；
    Wherein, 3 ∈ of 2 ∈ of 1 ∈ of λ [1.2,1.8], λ [4.2,4.8], λ [0.3,0.8].
  10. 10. device as claimed in claim 9, which is characterized in that the detection model of making a phone call obtains module and includes:
    Sample chooses module, the colour for choosing the color image that label driver makes a phone call, label driver does not make a phone call The fuzzy color image of image, label is as sample image；
    Initial training module obtains the mould of initial training for carrying out feature training to sample image using convolutional neural networks Type；
    Second training module carries out repetition training to test image according to the model of initial training, directly for choosing test image It is restrained to model；
    Model output module, for making a phone call detection model and to export using convergent model as trained driver.
  11. 11. device as claimed in claim 10, which is characterized in that the convolutional neural networks include: input layer, Th_Con Convolutional layer, Th_Pool pond layer, Th_Full full articulamentums；
    Wherein, every layer of convolutional layer includes Th_CK convolution kernel, and the size of convolution kernel is CKSi*CKSi, step-length 1；Every layer of pond Layer core size be KSi*KSi, step-length KSi；The neuron of the full articulamentum output of the last layer of the full articulamentum Quantity is 3, and as 3 drivers make a phone call to detect classification；
    Th_Con ∈ [2,8], Th_Pool ∈ [2,8], Th_Full ∈ [1,3], Th_CK ∈ [Th_CKmin,Th_CKmax], Th_ CKmin∈ [6,16], Th_CKmax∈ [30,512], CKSi ∈ [3,7], KSi ∈ [2,4].
  12. 12. device as claimed in claim 11, which is characterized in that the convolutional neural networks include:
    Input layer inputs the image of Width*Height；
    First layer convolutional layer exports Th_CK1 convolution kernel, and the size of convolution kernel is CKSi1*CKSi1, step-length 1；
    First layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Second layer convolutional layer exports Th_CK2 convolution kernel, and the size of convolution kernel is CKSi2*CKSi2, step-length 1；
    Second layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Third layer convolutional layer exports Th_CK3 convolution kernel, and the size of convolution kernel is CKSi3*CKSi3, step-length 1；
    Third layer pond layer uses maximum pond method output KSi*KSi, step-length for the core of KSi；
    Full articulamentum exports Th_Neur neuron using ReLU as activation primitive；
    Full articulamentum exports 3 neurons, i.e., 3 are made a phone call to detect classification；
    Wherein, [64,192] Width ∈, Height ∈ [64,192]；Th_CK1 ∈ [6,20], CKSi1 ∈ [3,5], KSi ∈ [2,4], Th_CK2 ∈ [6,40], CKSi2 ∈ [3,5], Th_CK3 ∈ [6,40], CKSi3 ∈ [3,5], Th_Neur ∈ [64, 10000]。
  13. 13. device as claimed in claim 10, the second training module include:
    Training characteristics extraction module, for the feature according to the model extraction test image of initial training；
    Training classification determination module, for calculating this feature and each driver makes a phone call to detect the similarity of category feature Simik, k k-th of classification of expression, k={ 1,2,3 }, selection SimikIt is worth maximum classification as couple candidate detection classification；
    Repetition training module, for calculating the error for determining result and legitimate reading, using back-propagation algorithm come training pattern, Repetition training characteristic extracting module and training classification determination module, until the model is restrained.
  14. 14. device as claimed in claim 9, which is characterized in that the vehicle window region extraction module includes:
    Vertical edge obtains module, for the candidate region of vehicle window to be carried out gray processing processing, obtains the candidate region of gray scale, adopts WithFilter obtains the vertical edge image of the candidate region of gray scale；
    Bianry image obtains module, for being split using threshold value Th_F logarithm edge image, obtains bianry image；
    Straight-line detection module obtains the straight of detection for handling using Hough transform line detection algorithm bianry image Line sequence column y=kix+bi, i=1,2 ..., N1, N1For the quantity of straight line；
    Straight line screening module, if being used for arctan | ki|≤Th_ θ, then retain the straight line, otherwise deletes the straight line, is thus remained Remaining linear order y=kjx+bj, j=1,2 ..., N2, N2For the quantity of remaining straight line；
    Up-and-down boundary obtains module, for scanning every straight line y=kjx+bj, j=1,2 ..., N2, obtain straight line y=kjx+bj's First pixel (xj1,yj1) and the last one pixel (xj2,yj2), it presses in y-directionBy straight line into Row segmentation, obtains the length on the corresponding direction x, and put it into LineHist array, using clustering algorithm pair LineHist array is clustered, and maximum two straight lines of the cluster value that will acquire are respectively as coboundary and lower boundary；
    Right boundary obtains module, for scanning coboundary, lower boundary respectively, by first pixel and lower boundary of coboundary First pixel constitute straight line as left margin, by the last one of the last one pixel of coboundary and lower boundary The straight line that pixel is constituted is as right margin；
    Vehicle window region obtains module, and the region for being surrounded by coboundary, left margin, right margin, lower boundary is vehicle window region；
    Wherein, [10,30] Th_F ∈, Th_ θ ∈ [5 °, 15 °], Th_S ∈ [5,20].
  15. 15. device as claimed in claim 9, which is characterized in that the region of interesting extraction module includes: region of interest border Module is obtained, for obtaining left boundary x=fl, the right boundary x=fr, the top boundary y=of human face region according to human face region Ft, following boundary y=fb, the then left boundary for the area-of-interest made a phone call areIt is right Side boundary isTop boundary is Following boundary isWfFor the width of human face region, W and H are respectively to acquire figure The width and height of picture；
    Area-of-interest obtains module, for the left boundary according to the area-of-interest made a phone call, the right boundary, top side Boundary, following boundary, determine rectangular area, which is area-of-interest；
    Wherein, 6 ∈ of 5 ∈ of 4 ∈ of λ [0.8,1.3], λ [0.3,0.8], λ [0.3,0.8].
  16. 16. device as claimed in claim 9, which is characterized in that the detection model detection module of making a phone call includes: that identification is special Extraction module is levied, for extracting the feature of area-of-interest using trained detection model of making a phone call；Identification classification determines mould Block, for calculating the feature of area-of-interest and the similarity Simi of each category featurek, k k-th of classification of expression, k=1, 2,3 }, Simi is chosenkIt is worth maximum classification as driver to make a phone call testing result and to export.
CN201610368797.XA 2016-05-30 2016-05-30 A kind of driver makes a phone call the detection method and device of behavior Active CN106056071B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610368797.XA CN106056071B (en) 2016-05-30 2016-05-30 A kind of driver makes a phone call the detection method and device of behavior

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610368797.XA CN106056071B (en) 2016-05-30 2016-05-30 A kind of driver makes a phone call the detection method and device of behavior

Publications (2)

Publication Number Publication Date
CN106056071A CN106056071A (en) 2016-10-26
CN106056071B true CN106056071B (en) 2019-05-10

Family

ID=57171292

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610368797.XA Active CN106056071B (en) 2016-05-30 2016-05-30 A kind of driver makes a phone call the detection method and device of behavior

Country Status (1)

Country Link
CN (1) CN106056071B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530730A (en) * 2016-11-02 2017-03-22 重庆中科云丛科技有限公司 Traffic violation detection method and system
CN106503673A (en) * 2016-11-03 2017-03-15 北京文安智能技术股份有限公司 A kind of recognition methodss of traffic driving behavior, device and a kind of video acquisition device
CN106778583B (en) * 2016-12-07 2019-12-17 北京理工大学 Vehicle attribute identification method and device based on convolutional neural network
CN108205649B (en) * 2016-12-20 2021-08-31 浙江宇视科技有限公司 Method and device for recognizing state of driver for calling and answering
CN107832728A (en) * 2017-11-21 2018-03-23 天津英田视讯科技有限公司 A kind of judge based on video makes a phone call Activity recognition method
CN108509902B (en) * 2018-03-30 2020-07-03 湖北文理学院 Method for detecting call behavior of handheld phone in driving process of driver
CN108764034A (en) * 2018-04-18 2018-11-06 浙江零跑科技有限公司 A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera
CN108710857B (en) * 2018-05-22 2022-05-17 深圳前海华夏智信数据科技有限公司 Method and device for identifying people and vehicles based on infrared supplementary lighting
CN108875812B (en) * 2018-06-01 2021-10-22 宁波工程学院 Driver behavior classification method based on branch convolutional neural network
CN110839128B (en) * 2018-08-16 2021-04-27 杭州海康威视数字技术股份有限公司 Photographing behavior detection method and device and storage medium
CN109410318B (en) * 2018-09-30 2020-09-08 先临三维科技股份有限公司 Three-dimensional model generation method, device, equipment and storage medium
CN109376634A (en) * 2018-10-15 2019-02-22 北京航天控制仪器研究所 A kind of Bus driver unlawful practice detection system neural network based
CN109583442B (en) * 2018-11-07 2020-12-11 北京智芯原动科技有限公司 False license plate detection method and device based on line segment detection
CN111325058B (en) * 2018-12-14 2023-12-01 长沙智能驾驶研究院有限公司 Driving behavior detection method, device, system and storage medium
CN110059541A (en) * 2019-02-28 2019-07-26 长江大学 A kind of mobile phone usage behavior detection method and device in driving
CN110309764B (en) * 2019-06-27 2021-06-01 浙江工业大学 Multi-stage driver call-making behavior detection method based on deep learning

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2442538A1 (en) * 2010-10-07 2012-04-18 Research in Motion Limited Method and system for preventing device operation when driving
CN102567743A (en) * 2011-12-20 2012-07-11 东南大学 Automatic identification method of driver gestures based on video images
CN103366506A (en) * 2013-06-27 2013-10-23 北京理工大学 Device and method for automatically monitoring telephone call behavior of driver when driving
CN104156717A (en) * 2014-08-31 2014-11-19 王好贤 Method for recognizing rule breaking of phoning of driver during driving based on image processing technology
CN104966059A (en) * 2015-06-15 2015-10-07 安徽创世科技有限公司 Method for detecting phoning behavior of driver during driving based on intelligent monitoring system
CN105005759A (en) * 2015-05-04 2015-10-28 南京理工大学 Multi-characteristic fused monitoring image front vehicle window positioning and extracting method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2442538A1 (en) * 2010-10-07 2012-04-18 Research in Motion Limited Method and system for preventing device operation when driving
CN102567743A (en) * 2011-12-20 2012-07-11 东南大学 Automatic identification method of driver gestures based on video images
CN103366506A (en) * 2013-06-27 2013-10-23 北京理工大学 Device and method for automatically monitoring telephone call behavior of driver when driving
CN104156717A (en) * 2014-08-31 2014-11-19 王好贤 Method for recognizing rule breaking of phoning of driver during driving based on image processing technology
CN105005759A (en) * 2015-05-04 2015-10-28 南京理工大学 Multi-characteristic fused monitoring image front vehicle window positioning and extracting method
CN104966059A (en) * 2015-06-15 2015-10-07 安徽创世科技有限公司 Method for detecting phoning behavior of driver during driving based on intelligent monitoring system

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
《Detecting Driver Use of Mobile Phone Based on In-car Camera》;Dan Wang et al；;《2014 10th International Conference on Computational Intelligence and Security》;20141231;第148-151页；
《基于机器视觉的驾驶人使用手持电话行为检测》;张波 等；;《吉林大学学报(工学版)》;20150930;第45卷(第5期);第1688-1695页；
《基于机器视觉的驾驶人使用手持电话行为检测方法》;魏民国 等；;《2014 中国汽车安全技术学术会议》;20141231;第149-158页；
《基于机器视觉的驾驶员打电话行为检测》;王丹 等；;《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》;20160315(第3期);第C035-209页；
《智能交通中的违章识别算法研究》;黎华东；;《中国优秀硕士学位论文全文数据库信息科技辑》;20150215;第C034-255页；

Also Published As

Publication number Publication date
CN106056071A (en) 2016-10-26

Similar Documents

Publication Publication Date Title
CN106056071B (en) A kind of driver makes a phone call the detection method and device of behavior
WO2017190574A1 (en) Fast pedestrian detection method based on aggregation channel features
Lee et al. Adaboost for text detection in natural scene
Pan et al. A robust system to detect and localize texts in natural scene images
CN106485224B (en) A kind of seatbelt wearing recognition methods and device
CN102722712B (en) Multiple-scale high-resolution image object detection method based on continuity
Park et al. Design of a visual perception model with edge-adaptive Gabor filter and support vector machine for traffic sign detection
CN102682287B (en) Pedestrian detection method based on saliency information
CN102938054B (en) Method for recognizing compressed-domain sensitive images based on visual attention models
CN101930549B (en) Second generation curvelet transform-based static human detection method
CN105809205B (en) A kind of classification method and its system of high spectrum image
CN102034107B (en) Unhealthy image differentiating method based on robust visual attention feature and sparse representation
CN106022285A (en) Vehicle type identification method and vehicle type identification device based on convolutional neural network
CN105809173B (en) A kind of image RSTN invariable attribute feature extraction and recognition methods based on bionical object visual transform
CN111126240B (en) Three-channel feature fusion face recognition method
CN112381775A (en) Image tampering detection method, terminal device and storage medium
An et al. An automated airplane detection system for large panchromatic image with high spatial resolution
Pan et al. Fast scene text localization by learning-based filtering and verification
CN108647695A (en) Soft image conspicuousness detection method based on covariance convolutional neural networks
CN111914838A (en) License plate recognition method based on text line recognition
Rigaud et al. Automatic text localisation in scanned comic books
CN102254196A (en) Method for identifying handwritten Chinese character by virtue of computer
CN103605993B (en) Image-to-video face identification method based on distinguish analysis oriented to scenes
Prates et al. Brazilian license plate detection using histogram of oriented gradients and sliding windows
CN104008404B (en) Pedestrian detection method and system based on significant histogram features

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant