CN106874863A - Vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction - Google Patents
Vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction Download PDFInfo
- Publication number
- CN106874863A CN106874863A CN201710059676.1A CN201710059676A CN106874863A CN 106874863 A CN106874863 A CN 106874863A CN 201710059676 A CN201710059676 A CN 201710059676A CN 106874863 A CN106874863 A CN 106874863A
- Authority
- CN
- China
- Prior art keywords
- road surface
- detection
- road
- vehicle
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims description 30
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 9
- 230000033001 locomotion Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 5
- 238000013459 approach Methods 0.000 claims description 4
- 230000003542 behavioural effect Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000009471 action Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 238000012360 testing method Methods 0.000 abstract description 7
- 230000008901 benefit Effects 0.000 abstract description 4
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 9
- 230000004913 activation Effects 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 7
- 210000002569 neuron Anatomy 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000003068 static effect Effects 0.000 description 5
- 239000010426 asphalt Substances 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 3
- 238000009795 derivation Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000001537 neural effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 239000012491 analyte Substances 0.000 description 1
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 210000004218 nerve net Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000010865 sewage Substances 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000000225 synapse Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
Vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction, with mobile terminal test point as road camera, mobile terminal test point obtains image information by camera, deep learning is introduced into road surface event recognition and is improved, to significantly improve road incidents recognition accuracy.The present invention is analyzed using convolutional neural networks to the image for obtaining, and road surface ROI region is divided into multiple networks, road surface-non-road surface identification model is built, by targets such as the parking violation of non-road surface grid reversal of identification highway, vehicle driving in reverse.The present invention is applied to road surface and disobeys stop detection, vehicle and drive in the wrong direction the non real-time tasks such as detection, the characteristics of make full use of mobile Internet and advantage, stops the road surface event detection such as being driven in the wrong direction with vehicle to be realized with a low cost region high coverage rate vehicle and disobey.
Description
Technical field
The invention belongs to the video detection technology field of depth characteristic, it is related to the detection of road surface event, based on Deep-CNN
Road surface reversal of identification model carries out vehicle and disobeys the detection that stops or drive in the wrong direction, and data analysis and data mining to testing result, is
Vehicle of the one kind based on depth convolutional neural networks (Deep-CNN) is disobeyed and stops detection method of driving in the wrong direction.
Background technology
Road accident, traffic congestion, environmental pollution is the generality problem that current Highway Development faces.Road traffic
Safe condition makes people worried, road informationization and intelligent transportation system (Intelligent Transportation System,
ITS it is) lifting road equipment utilization ratio, alleviates traffic jam, reduces the effective means of traffic accident incidence.By calculating
The traffic flow parameters such as machine vision technique real-time perception road speed, flow, there is provided real-time road, and historical data is combined to road network
Current state and travel time are predicted, and artificial supervision is replaced with automatic video frequency analysis, detect that road is different from massive video
Ordinary affair part, including road surface leave analyte detection, highway it is illegal stop detection etc. high-risk event, to improve highway information level and
Public service ability all has very important significance.
The content of the invention
The problem to be solved in the present invention is：As traffic surveillance videos quantity is unprecedented soaring, only relying on cannot manually realize
Existing video resource is effectively managed.Analyzed by computer video and automatically analyze Traffic Surveillance Video, extract traffic parameter, automatically
It was found that and active reporting anomalous event, the human cost of traffic administration can be greatly decreased, raise the management level and event be emergent rings
Should be able to power.
The technical scheme is that：Vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction, based on depth
Road surface area-of-interest is divided into multiple nets by degree convolutional neural networks Deep-CNN network models, Deep-CNN network models
Network, builds road surface-non-road surface identification model, by the parking violation of non-road surface grid reversal of identification express highway pavement, retrograde etc.
Target, comprises the following steps：
Step1：Road surface model train, gather road camera video image, by video window road surface it is interested
Region ROIRoad surfaceGridding is divided into multiple fritters, as the training set of Deep-CNN network models after standardization, during training first
Characteristics of image is obtained using unsupervised approaches training, label is set again after cluster, manual type mark road surface types distinguish road surface
With non-road surface, road surface-non-road surface identification model is obtained；
Step2：Non- road surface foreground model training, the grid picture that will be divided into non-road surface is combined into candidate by connected region
Target adds training storehouse, is trained using Deep-CNN network models again, classification training road surface target, the road surface target
Including vehicle, road surface legacy and pedestrian, prospect identification model is obtained；
Step3：Foreground target is detected, on the basis of the identification model of step1 and step2, real time video image used
Deep-CNN network models and SVM classifier realize the detection and classification of foreground target, first recognize the non-road surface in road surface, then identify
The type of foreground target；
Step4：Behavioural analysis, on the basis of foreground object classification identification, according to foreground target in sequence of video images
Contextual information, carry out road incidents vehicle disobey stop drive in the wrong direction detection identification, ifWithRespectively T0With T0+tMoment
K-th position of prospect candidate target, calculate prospect candidate target image-region displacement Euclidean distance K-th prospect candidate target motion state and direction are obtained, further determines that target with the presence or absence of stopping or retrograde shape
State.
Described vehicle is disobeyed and stopped or detection of driving in the wrong direction is specially：
Step1.1：Prohibited area ROI is set to the road surface in real time video imageProhibit, after video window is image gridding,
Classified according to road surface-non-road surface identification model；
Step1.2：Connection ROIProhibitRegion Nei Fei road surfaces grid pictures Ii,j, generation candidate target Ok；
Step1.3：To candidate target OkClassification and Identification, if car, is then locked as vehicle target；
Step1.4：Calculate initial time T0With T after t0+tBetween vehicle target shift length and direction, and wrap
Road vehicles direction is contrasted；
Step1.5：Judge that vehicle is disobeyed to stop or behavior of driving in the wrong direction.
Further, Step1 and Step2 are specially：
1) region of interest ROI is setRoad surface：The video image of collection road or streetscape watch-dog, obtains road or streetscape is regarded
Frequency two field picture, according to actual road or streetscape situation, extracts diagonal 4 points in border of region-of-interest on current frame image,
Point to being extracted carries out fitting a straight line calculating, forms the interest domain ROI as detection in fork configuration, fork configurationRoad surface,
It is exactly effective detection region；
2) the unrestrained water filling of non-detection region：Non- ROIRoad surfaceIt is non-monitored region, carries out unrestrained water filling treatment, after filling, falls
ROIRoad surfaceGrid picture pixels average outside region is 0, and directly filtering no longer carries out subsequent treatment；
3) detection zone ROIRoad surfaceGridding piecemeal, grid picture is road surface or non-road surface through Deep-CNN classification results, is incited somebody to action
ROIRoad surfacePiecemeal in the grid picture on non-road surface link up, be designated as Ip,q, i.e.,Road surface, Ip,qComposition candidate target, feeding point
Class device, is categorized as vehicle, pedestrian or road surface legacy.
The present invention makes full use of existing video monitoring facility and massive video data, can to greatest extent save hardware input,
More rich intuitively traffic data is obtained, the data/information demand of traffic administration and public service is met.
The present invention establishes the reverse road surface identification model based on Deep-CNN, solves to move using Deep-CNN road surface models
Dynamic video camera detects with image object, is applied to vehicle and disobeys stop detecting, driving in the wrong direction the tasks such as detection.Passed with ground induction coil, radar etc.
System technology is compared, and monitor video contains the image informations such as road surface, vehicle, road surface legacy, extracts above-mentioned by CNN neutral nets
Feature is simultaneously disobeyed based on Deep-CNN detection road vehicles and the abnormal events information such as stops, drives in the wrong direction, and identification road accident simultaneously sends in time
Picture and text are alarmed, there is provided the road incidents information more more rich than traditional car test equipment.
The present invention forms mobile terminal test point in monitored area any wide-area deployment mobile terminal, is set by mobile communication
Standby or road monitoring interacts formula demarcation, after determining camera parameter, using camera collection image, using CNN nerve nets
Network is further analysed to the image for gathering, and obtains the ROI data of current each image, and road surface ROI region is divided into multiple
Network, builds road surface-non-road surface identification model, by targets such as the parking violation of non-road surface grid reversal of identification, reverse drivings.
Mobile terminal test point of the present invention is road camera, and itself is cheap, makes full use of mobile Internet advantage,
Expensive monitoring instrument equipment is not needed, camera ready-made on road is only needed, can any wide-area deployment, Site Detection information of road surface
Transmitted to server by mobile Internet.The characteristics of making full use of mobile Internet and advantage are high to be realized with a low cost region
Coverage rate road surface event detection.
Brief description of the drawings
Fig. 1 is based on Deep-CNN reversal of identification road surface event detection flow chart for the present invention
Fig. 2 is the Deep-CNN modular concepts that the present invention is used.(a) Softplus and ReLU activation primitives (b) Deep-
CNN network structures.
Fig. 3 is that Deep-CNN road surfaces of the present invention identification model detects figure.A () ROI detection zones divide (b) unrestrained water law filling
Remove non-ROI region pixel (c) gridding Deep-CNN inputs and target detection.
Fig. 4 is that mobile terminal single frames photo pavement detection of the present invention trains figure.(a) hand-held mobile terminal photo (b) track
ROI region pavement detection effect.
Fig. 5 is that the selection Jiangsu Province that the present invention is implemented rather is connected highway scene and reversely known based on Deep-CNN road surfaces
Other model forbidden zone disobey stop, drive in the wrong direction Detection results figure (a) road surface ROI (b) disobey stop detection (c) drive in the wrong direction detection (d) drive in the wrong direction detection.
Specific embodiment
Deep learning is introduced road surface event recognition and is improved by the present invention, is remarkably improved road incidents identification accurate
Degree.It is static object in view of vehicle, and traditional background modeling method is not suitable for static object detection, and road surface legacy
It is difficult with prior model construction training set so that the same road surface legacy as static object can be obscured with separated parking
Situation.The present invention establishes the road surface reversal of identification model based on depth convolutional neural networks Deep-CNN, mobile terminal detection
Point be road camera, mobile terminal test point by camera obtain image information, using CNN neutral nets to acquisition figure
As being analyzed, road surface ROI region is divided into multiple networks, road surface-non-road surface identification model is built, by non-road surface net
The parking violation of lattice reversal of identification and reverse driving.
The present invention constructs the layering road incidents identification framework based on Deep-CNN networks：
Its basic thought is layering identification, is recognized by road surface-non-road surface, and then reversal of identification road surface object, road surface
Reversal of identification process is：
Road surface model training is carried out first, by area-of-interest (Region of Interest, ROI) net of video window
Format and be divided into multiple fritters, as the training set of Deep-CNN networks after standardization, it is contemplated that road surface model by pavement asphalt,
Road surface track, guardrail etc. are constituted, and picture appearance differs greatly, and pressure is labeled as an independent class and easily causes over-fitting, so adopt
Trained with unsupervised approaches and obtain characteristics of image, road surface is marked with multiclass label manual type again after cluster, it is ensured that accurately distinguish
Road surface and non-road surface.
Then non-road surface foreground model training is carried out, the grid picture block that will be divided into non-road surface is combined into by connected region
Candidate target adds training storehouse, Deep-CNN network trainings again, classification training road surface target, including vehicle, road surface legacy,
Pedestrian etc..
When carrying out real-time detection to road, foreground target detection is carried out first, on the basis of road surface identification model before
Realize motion with static foreground target detection and classification.
Behavioural analysis is finally carried out, it is upper and lower in sequence of video images according to target on the basis of foreground target detection
Literary information, carry out vehicle disobey stop detection, vehicle drive in the wrong direction detection.
Described vehicle disobeys the detection that stops detecting and driving in the wrong direction and is embodied in following steps：
Step1.1：Prohibited area ROI is set to the road surface in real time video imageProhibit, after video window is image gridding,
Classified according to road surface-non-road surface identification model；
Step1.2：Connection ROIProhibitRegion Nei Fei road surfaces grid pictures Ii,j, generation candidate target Ok；
Step1.3：To candidate target OkClassification and Identification, if car, is then locked as vehicle target；
Step1.4：Calculate initial time T0With T after t0+tBetween vehicle target shift length and direction, and wrap
Road vehicles direction is contrasted；
Step1.5：Judge that vehicle is disobeyed to stop or behavior of driving in the wrong direction.
Implementation of the invention is further illustrated below by specific embodiment.
Schematic flow sheet of the invention is as shown in figure 1, by mobile terminal road video camera acquisition video pictures, carry out
CNN treatment training classification carries out road surface event detection：
Step1：Road surface model is trained, and the area-of-interest (Region of Interest, ROI) of video window is split
Into multiple fritters, as the training set of Deep-CNN networks after standardization, it is contemplated that road surface model is by pavement asphalt, road surface car
Road, guardrail etc. are constituted, and picture appearance differs greatly, and pressure is labeled as an independent class and easily causes over-fitting, so using without prison
Superintend and direct method training obtain characteristics of image, after cluster again with multiclass label manual type mark road surface, it is ensured that accurately distinguish road surface with
Non- road surface；
Step2：Non- road surface foreground model training, the segmentation fritter that will be divided into non-road surface is combined into candidate by connected region
Target adds training storehouse, again Deep-CNN network trainings, classification training road surface target, including vehicle, road surface legacy, pedestrian
Deng；
Step3：Foreground target detects, realizes moving and static foreground on the basis of the identification model of step1, step2 road surface
Target detection and classification；
Step4：Behavioural analysis, on the basis of foreground object classification identification, contextual information carries out road incidents knowledge
Not, including vehicle disobey stop detection, vehicle drive in the wrong direction detection.
The network model principle of depth convolutional neural networks Deep-CNN as shown in Fig. 2 including：
1) class biological neural activation primitive：
Neuroscientist think people's brain signal receive process nature on be neural synapses to external world input signal activation ring
Should, neuron node excitement is considered as linear function activation, and it is 1 that stimulation responds nodal value to external world, is otherwise 0.It is right
Brain wave found by the energy measurement experiment of input stimulus, the pattern of environmental stimuli and neuron node response be it is sparse, i.e.,
A large amount of lower some neuron node of stimulus signal are to activate, receive the external information same time there was only about 1%~
4% neuron is in active state.2001, neuroscientist Dayan and Abott proposed to simulate from biology angle
Brain neuron signal receives and activation response model.For the ease of calculating, Glorot et al. is proposed and is adapted to machine learning
The correction of Nonlinear activation primitive Softplus of multilayer neural network, approximate substitution neuron models.
Softplus functions are defined as：
SortPlus (x)=log (1+exp (x)) (1)
Correct the linear simplifiation version that linear activation primitive (Rectified Linear Units, ReLU) is Softplus
This, is defined as：
ReLU (x)=max (0, x) (2)
ReLU function derivations are very simple, work as x>Derivative is 1 when 0, is otherwise 0, and derivation formula is to be represented by：
Curve distribution such as Fig. 2 (a) of Softplus and ReLU activation primitives
2) network structure
Fig. 2 (b) is that multi-layer C NN network structures include the basic layers such as input layer, hidden layer and output layer, and hidden layer is by more
Individual convolution-pond beggar layer is formed by stacking, and the output of last layer is used as next layer of input.ReLU is used after each convolutional calculation
Function activates node, due to the derivation feature of ReLU functions, weights quilt of the initial value by part node after ReLU gradient calculations
0 is set to, this is not activated by the part node of point 0 value weight so that network structure has openness.Openness network structure symbol
The essence of biological neural is closed, also there is real advantage in mathematical computations, such as reduce parameter complexity, improve operation efficiency, and
Can effectively prevent over-fitting from occurring.
Whole process includes operating in detail below：
Based on the Deep-CNN road surfaces identification model that Caffe realizes proposing, candidate target is reversely extracted with road surface model, point
Realize disobeying after class stopping the functions such as detection that detect and drive in the wrong direction.
Assuming that input image size is w × h, image lattice is turned into the individual fritters of (w/s) × (h/s) using step-length s, represented
Into matrix sequence Aij(i=1,2,3 ... w/s；J=1,2,3 ... h/s), the pixel wide of step-length s is about 1/4 width in track.
Detection interest region ROIRoad surfaceIt is point { x1, y1；X2, y2；X3, y3；x4.y4；X1, y1 } the closing detection zone that surrounds
Domain, then AROI∈ROIRoad surfaceIt is matrix grid in detection zone.
By the A of road surface modelROIImage block feeding Deep-CNN training, using unsupervised approaches, extracts road surface modular character
After clustered, be the subtype such as asphalt surface, track wire tag, guardrail by road surface cluster, labeled as " road surface 1 "~" road surface
n”(n<4).Be to improve the covering of road surface model sample, gridding image process using downwards, Offset portion pixel sampling side to the right
Method, is obtained the individual road surface samples of 3 × (w/s) × (h/s).
After input picture gridding, construction correspondence classification matrix, the grid picture positioned at road surface non-detection region directly makes
Pixel is set to 0, the non-area's grid matrix that is annihilated with unrestrained water lawIt is initialized as -1, detection zone grid image
After block is classified by Deep-CNN road surface models, output category result matrix
Road surface legacy empirical model is built due to being difficult with priori, the present invention is recognized using road surface-non-road,
Reversely realize foreground detection with classification.Extraction prospect is by CROI=1 connection matrix sub-block constitutes candidate target, uses Deep-
Prospect cluster is vehicle, pedestrian, other targets (may be road surface legacy) by CNN objective networks.
Fig. 3 demonstrates Deep-CNN road surface model implementation process, and interest domain, i.e. detection zone are labelled with Fig. 3 (a), makes
The ROI detection zones of closing are generated with 4 summits, Fig. 3 (b) is the unrestrained water filling in non-detection area, after overflowing water law filling, grid picture block
Ii,jThe grid picture pixels average fallen outside ROI region is 0, can directly be filtered before Deep-CNN is sent into.Detection zone ROI schemes
Tile is road surface or non-road surface through Deep-CNN classification results.By ROIRoad surfacePiecemeal in the grid picture on non-road surface link up, remember
It is Ip,q, i.e.,Road surface, as shown in Fig. 3 (c) overstriking marked regions, Ip,qComposition candidate target, sends into grader, is categorized as car
, pedestrian or other indefinite targets (i.e. road surface legacy).
IfWithRespectively T0With T0+tK-th position of prospect candidate target at moment, calculates prospect candidate
Euclidean distance of the target in image-region displacementObtain k-th prospect candidate target motion state and side
To, further determine that target with the presence or absence of stop or retrograde state.
Fig. 4 demonstrates handheld terminal movement photo pavement detection situation, and Fig. 4 (a) is original photo, uses lane line straight line
Detection and Extraction ROI detection zones, and non-ROI region is filled with black using unrestrained water law, to mitigate grader burden, such as Fig. 4
B shown in (), road surface Sewage well cover founds cone roadblock, and deceleration strip is all detected as non-road surface.To conventional pavement facilities, such as well lid, road
Barrier etc., can extract foreground image to training sample and concentrates directly training detection model.
Based on above-mentioned model, Fig. 5 (b), (c), (d) realize respectively forbidden zone disobey stop detection, vehicle drive in the wrong direction detection, vehicle it is inverse
Row detection.
The behavior such as expressway ramp outlet parking or retrograde, Emergency Vehicle Lane parking easily causes potential safety hazard, and Fig. 5 is demonstrated
Deep-CNN forbidden zones parking detection model is realized.Forbidden zone ROI is marked in road surfaceProhibit, such as exit ramp shunting mouthful front end fish-bone spot
The region such as horse line and Emergency Vehicle Lane, by detection zone image lattice AROIFeeding Deep-CNN road surface model training, generates forbidden zone road
Shown in face die plate, such as Fig. 5 (a).Vehicle is identified into the foreground target region of detection by Deep-CNN foreground classification devices again.With
Vehicle targetAcross grid matrix line number MrowIt is displacement threshold value, it is assumed that targetMotion tracking is small in interval time t
In given threshold, then it is judged to parking, otherwise calculates car displacement and direction, such as direction of motion dirctv and direction of traffic
Dirctflow is conversely, be then judged to drive in the wrong direction, Fig. 5 (b)~(d) is respectively parking and retrograde detection.Wherein disobey and stop threshold value for t>
20S (500 frame), Mrow<3, threshold value of driving in the wrong direction Mrow>=3, the direction of motion is dirctv=-dirctflow.
Claims (3)
1. the vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction, it is characterized in that being based on depth convolutional neural networks
Road surface area-of-interest is divided into multiple networks, structure road surface-non-by Deep-CNN network models, Deep-CNN network models
Road surface identification model, by the parking violation of non-road surface grid reversal of identification or drive in the wrong direction etc. target, comprise the following steps：
Step1：Road surface model is trained, and the video image of road camera is gathered, by the area-of-interest on road surface in video window
ROIRoad surfaceGridding is divided into multiple fritters, as the training set of Deep-CNN network models after standardization, is used first during training
Unsupervised approaches training obtains characteristics of image, sets label after cluster again, manual type mark road surface types, distinguish road surface with it is non-
Road surface, obtains road surface-non-road surface identification model；
Step2：Non- road surface foreground model training, the grid picture that will be divided into non-road surface is combined into candidate target by connected region
Training storehouse is added, is trained using Deep-CNN network models again, classification training road surface target, the road surface target includes
Vehicle, road surface legacy and pedestrian, obtain prospect identification model；
Step3：Foreground target detection, on the basis of the identification model of step1 and step2, Deep- is used to real time video image
CNN network models and SVM classifier realize the detection and classification of foreground target, first recognize the non-road surface in road surface, then identify prospect
The type of target；
Step4：Behavioural analysis, it is upper in sequence of video images according to foreground target on the basis of foreground object classification identification
Context information, carry out road incidents vehicle disobey stop drive in the wrong direction detection identification, ifWithRespectively T0With T0+tThe kth at moment
The position of individual prospect candidate target, calculates Euclidean distance of the prospect candidate target in image-region displacement Obtain
K-th prospect candidate target motion state and direction are taken, further determines that target with the presence or absence of stopping or retrograde state.
2. the vehicle based on depth convolutional neural networks according to claim 1 is disobeyed and stops detection method of driving in the wrong direction, it is characterized in that
Described vehicle is disobeyed and stopped or detection of driving in the wrong direction is specially：
Step1.1：Prohibited area ROI is set to the road surface in real time video imageProhibit, after video window is image gridding, according to
Road surface-non-road surface identification model is classified；
Step1.2：Connection ROIProhibitRegion Nei Fei road surfaces grid pictures Ii,j, generation candidate target Ok；
Step1.3：To candidate target OkClassification and Identification, if car, is then locked as vehicle target；
Step1.4：Calculate initial time T0With T after t0+tBetween vehicle target shift length and direction, and wrap bus
Direction is contrasted；
Step1.5：Judge that vehicle is disobeyed to stop or behavior of driving in the wrong direction.
3. the vehicle based on depth convolutional neural networks according to claim 1 is disobeyed and stops detection method of driving in the wrong direction, it is characterized in that
Step1 and Step2 are specially：
1) region of interest ROI is setRoad surface：The video image of collection road or streetscape watch-dog, obtains road or streetscape frame of video
Image, according to actual road or streetscape situation, extracts diagonal 4 points in border of region-of-interest, to institute on current frame image
The point of extraction carries out fitting a straight line calculating, forms the interest domain ROI as detection in fork configuration, fork configurationRoad surface, that is,
Effective detection region；
2) the unrestrained water filling of non-detection region：Non- ROIRoad surfaceIt is non-monitored region, carries out unrestrained water filling treatment, after filling, falls
ROIRoad surfaceGrid picture pixels average outside region is 0, and directly filtering no longer carries out subsequent treatment；
3) detection zone ROIRoad surfaceGridding piecemeal, grid picture is road surface or non-road surface through Deep-CNN classification results, is incited somebody to action
ROIRoad surfacePiecemeal in the grid picture on non-road surface link up, be designated as Ip,q, i.e.,Ip,qComposition candidate target, feeding point
Class device, is categorized as vehicle, pedestrian or road surface legacy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710059676.1A CN106874863B (en) | 2017-01-24 | 2017-01-24 | Vehicle illegal parking and reverse running detection method based on deep convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710059676.1A CN106874863B (en) | 2017-01-24 | 2017-01-24 | Vehicle illegal parking and reverse running detection method based on deep convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106874863A true CN106874863A (en) | 2017-06-20 |
| CN106874863B CN106874863B (en) | 2020-02-07 |
Family
ID=59159242
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710059676.1A Active CN106874863B (en) | 2017-01-24 | 2017-01-24 | Vehicle illegal parking and reverse running detection method based on deep convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106874863B (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491765A (en) * | 2017-08-29 | 2017-12-19 | 张羽 | A kind of shared bicycle and its parking identification device and system |
| CN108256554A (en) * | 2017-12-20 | 2018-07-06 | 深圳市金溢科技股份有限公司 | Vehicle reverse stopping judgment method, server and system based on deep learning |
| CN109063612A (en) * | 2018-07-19 | 2018-12-21 | 中智城信息技术有限公司 | City intelligent red line management method and machine readable storage medium |
| CN109493606A (en) * | 2017-09-11 | 2019-03-19 | 中国移动通信集团浙江有限公司 | The recognition methods and system of parking are disobeyed on a kind of highway |
| CN109523570A (en) * | 2017-09-20 | 2019-03-26 | 杭州海康威视数字技术股份有限公司 | Beginning parameter transform model method and device |
| CN109670404A (en) * | 2018-11-23 | 2019-04-23 | 江苏理工学院 | A kind of road ponding image detection method for early warning based on mixed model |
| CN109815831A (en) * | 2018-12-28 | 2019-05-28 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of vehicle is towards acquisition methods and relevant apparatus |
| CN109887281A (en) * | 2019-03-01 | 2019-06-14 | 北京云星宇交通科技股份有限公司 | A kind of method and system monitoring traffic events |
| CN109919053A (en) * | 2019-02-24 | 2019-06-21 | 太原理工大学 | A kind of deep learning vehicle parking detection method based on monitor video |
| CN109949578A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal automatic auditing method of vehicle crimping based on deep learning |
| WO2019175686A1 (en) * | 2018-03-12 | 2019-09-19 | Ratti Jayant | On-demand artificial intelligence and roadway stewardship system |
| CN110399803A (en) * | 2019-07-01 | 2019-11-01 | 北京邮电大学 | A kind of vehicle checking method and device |
| CN110555347A (en) * | 2018-06-01 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | Vehicle target identification method and device with dangerous cargo carrying behavior and electronic equipment |
| CN110619750A (en) * | 2019-08-15 | 2019-12-27 | 重庆特斯联智慧科技股份有限公司 | Intelligent aerial photography identification method and system for illegal parking vehicle |
| CN110929676A (en) * | 2019-12-04 | 2020-03-27 | 浙江工业大学 | Deep learning-based real-time detection method for illegal turning around |
| CN111126171A (en) * | 2019-12-04 | 2020-05-08 | 江西洪都航空工业集团有限责任公司 | Vehicle reverse running detection method and system |
| CN111178215A (en) * | 2019-12-23 | 2020-05-19 | 深圳成谷科技有限公司 | Sensor data fusion processing method and device |
| CN111209880A (en) * | 2020-01-13 | 2020-05-29 | 南京新一代人工智能研究院有限公司 | Vehicle behavior identification method and device |
| CN111368687A (en) * | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| CN112085952A (en) * | 2020-09-07 | 2020-12-15 | 平安科技（深圳）有限公司 | Vehicle data monitoring method and device, computer equipment and storage medium |
| CN112270668A (en) * | 2020-11-06 | 2021-01-26 | 南京斌之志网络科技有限公司 | Suspended cable detection method and system and electronic equipment |
| WO2021027569A1 (en) * | 2019-08-14 | 2021-02-18 | 平安国际智慧城市科技股份有限公司 | Law enforcement analysis method and apparatus, electronic device, and storage medium |
| CN113327414A (en) * | 2020-02-28 | 2021-08-31 | 深圳市丰驰顺行信息技术有限公司 | Vehicle reverse running detection method and device, computer equipment and storage medium |
| CN113378642A (en) * | 2021-05-12 | 2021-09-10 | 三峡大学 | Method for detecting illegal occupation buildings in rural areas |
| CN114360209A (en) * | 2022-01-17 | 2022-04-15 | 常州信息职业技术学院 | Video behavior identification security system based on deep learning |
| CN115984221A (en) * | 2023-01-03 | 2023-04-18 | 广州新粤交通技术有限公司 | Road marking repairing and identifying method, device, equipment and storage medium thereof |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937614A (en) * | 2010-06-12 | 2011-01-05 | 北京中科卓视科技有限责任公司 | Plug and play comprehensive traffic detection system |
| CN102184547A (en) * | 2011-03-28 | 2011-09-14 | 长安大学 | Video-based vehicle reverse driving event detecting method |
| US20120148092A1 (en) * | 2010-12-09 | 2012-06-14 | Gorilla Technology Inc. | Automatic traffic violation detection system and method of the same |
| CN104809470A (en) * | 2015-04-23 | 2015-07-29 | 杭州中威电子股份有限公司 | Vehicle converse running detection device and method based on SVM |
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105185108A (en) * | 2015-08-06 | 2015-12-23 | 苏州市世跃智能科技有限公司 | Automatic snapshot system of illegal parking at yellow grid lines |
| US20160307071A1 (en) * | 2015-04-20 | 2016-10-20 | Xerox Corporation | Fisher vectors meet neural networks: a hybrid visual classification architecture |
-
2017
- 2017-01-24 CN CN201710059676.1A patent/CN106874863B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937614A (en) * | 2010-06-12 | 2011-01-05 | 北京中科卓视科技有限责任公司 | Plug and play comprehensive traffic detection system |
| US20120148092A1 (en) * | 2010-12-09 | 2012-06-14 | Gorilla Technology Inc. | Automatic traffic violation detection system and method of the same |
| CN102184547A (en) * | 2011-03-28 | 2011-09-14 | 长安大学 | Video-based vehicle reverse driving event detecting method |
| US20160307071A1 (en) * | 2015-04-20 | 2016-10-20 | Xerox Corporation | Fisher vectors meet neural networks: a hybrid visual classification architecture |
| CN104809470A (en) * | 2015-04-23 | 2015-07-29 | 杭州中威电子股份有限公司 | Vehicle converse running detection device and method based on SVM |
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105185108A (en) * | 2015-08-06 | 2015-12-23 | 苏州市世跃智能科技有限公司 | Automatic snapshot system of illegal parking at yellow grid lines |
Non-Patent Citations (2)
| Title |
|---|
| CHRISTOPHER J.HOLDER ET AL.: "From On-Road to Off:Transfer Learning Within a Deep Convolutional Neural Network for Segmentation and Classification of Off-Road Scenes", 《EUROPEAN CONFERENCE ON COMPUTER VISION》 * |
| 刘占文 等: "基于图模型与卷积神经网络的交通标志识别方法", 《交通运输工程学报》 * |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491765A (en) * | 2017-08-29 | 2017-12-19 | 张羽 | A kind of shared bicycle and its parking identification device and system |
| CN109493606A (en) * | 2017-09-11 | 2019-03-19 | 中国移动通信集团浙江有限公司 | The recognition methods and system of parking are disobeyed on a kind of highway |
| CN109523570A (en) * | 2017-09-20 | 2019-03-26 | 杭州海康威视数字技术股份有限公司 | Beginning parameter transform model method and device |
| CN109523570B (en) * | 2017-09-20 | 2021-01-22 | 杭州海康威视数字技术股份有限公司 | Motion parameter calculation method and device |
| CN108256554A (en) * | 2017-12-20 | 2018-07-06 | 深圳市金溢科技股份有限公司 | Vehicle reverse stopping judgment method, server and system based on deep learning |
| WO2019175686A1 (en) * | 2018-03-12 | 2019-09-19 | Ratti Jayant | On-demand artificial intelligence and roadway stewardship system |
| CN110555347A (en) * | 2018-06-01 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | Vehicle target identification method and device with dangerous cargo carrying behavior and electronic equipment |
| CN109063612A (en) * | 2018-07-19 | 2018-12-21 | 中智城信息技术有限公司 | City intelligent red line management method and machine readable storage medium |
| CN109670404A (en) * | 2018-11-23 | 2019-04-23 | 江苏理工学院 | A kind of road ponding image detection method for early warning based on mixed model |
| CN109670404B (en) * | 2018-11-23 | 2023-07-11 | 江苏理工学院 | Road ponding image detection early warning method based on hybrid model |
| CN109815831A (en) * | 2018-12-28 | 2019-05-28 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of vehicle is towards acquisition methods and relevant apparatus |
| CN109815831B (en) * | 2018-12-28 | 2021-03-23 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle orientation obtaining method and related device |
| CN109949578A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal automatic auditing method of vehicle crimping based on deep learning |
| CN109919053A (en) * | 2019-02-24 | 2019-06-21 | 太原理工大学 | A kind of deep learning vehicle parking detection method based on monitor video |
| CN109887281A (en) * | 2019-03-01 | 2019-06-14 | 北京云星宇交通科技股份有限公司 | A kind of method and system monitoring traffic events |
| CN110399803A (en) * | 2019-07-01 | 2019-11-01 | 北京邮电大学 | A kind of vehicle checking method and device |
| WO2021027569A1 (en) * | 2019-08-14 | 2021-02-18 | 平安国际智慧城市科技股份有限公司 | Law enforcement analysis method and apparatus, electronic device, and storage medium |
| CN110619750B (en) * | 2019-08-15 | 2020-09-11 | 重庆特斯联智慧科技股份有限公司 | Intelligent aerial photography identification method and system for illegal parking vehicle |
| CN110619750A (en) * | 2019-08-15 | 2019-12-27 | 重庆特斯联智慧科技股份有限公司 | Intelligent aerial photography identification method and system for illegal parking vehicle |
| CN111126171A (en) * | 2019-12-04 | 2020-05-08 | 江西洪都航空工业集团有限责任公司 | Vehicle reverse running detection method and system |
| CN110929676A (en) * | 2019-12-04 | 2020-03-27 | 浙江工业大学 | Deep learning-based real-time detection method for illegal turning around |
| CN111178215A (en) * | 2019-12-23 | 2020-05-19 | 深圳成谷科技有限公司 | Sensor data fusion processing method and device |
| CN111178215B (en) * | 2019-12-23 | 2024-03-08 | 深圳成谷科技有限公司 | Sensor data fusion processing method and device |
| CN111209880A (en) * | 2020-01-13 | 2020-05-29 | 南京新一代人工智能研究院有限公司 | Vehicle behavior identification method and device |
| CN113327414A (en) * | 2020-02-28 | 2021-08-31 | 深圳市丰驰顺行信息技术有限公司 | Vehicle reverse running detection method and device, computer equipment and storage medium |
| CN111368687A (en) * | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| CN112085952A (en) * | 2020-09-07 | 2020-12-15 | 平安科技（深圳）有限公司 | Vehicle data monitoring method and device, computer equipment and storage medium |
| CN112085952B (en) * | 2020-09-07 | 2022-06-03 | 平安科技（深圳）有限公司 | Method and device for monitoring vehicle data, computer equipment and storage medium |
| CN112270668A (en) * | 2020-11-06 | 2021-01-26 | 南京斌之志网络科技有限公司 | Suspended cable detection method and system and electronic equipment |
| CN112270668B (en) * | 2020-11-06 | 2021-09-21 | 威海世一电子有限公司 | Suspended cable detection method and system and electronic equipment |
| CN113378642A (en) * | 2021-05-12 | 2021-09-10 | 三峡大学 | Method for detecting illegal occupation buildings in rural areas |
| CN114360209B (en) * | 2022-01-17 | 2023-06-23 | 常州信息职业技术学院 | Video behavior recognition security system based on deep learning |
| CN114360209A (en) * | 2022-01-17 | 2022-04-15 | 常州信息职业技术学院 | Video behavior identification security system based on deep learning |
| CN115984221A (en) * | 2023-01-03 | 2023-04-18 | 广州新粤交通技术有限公司 | Road marking repairing and identifying method, device, equipment and storage medium thereof |
| CN115984221B (en) * | 2023-01-03 | 2023-08-04 | 广州新粤交通技术有限公司 | Road marking restoration and identification method, device, equipment and storage medium thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106874863B (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106874863A (en) | Vehicle based on depth convolutional neural networks is disobeyed and stops detection method of driving in the wrong direction | |
| CN106845424A (en) | Road surface remnant object detection method based on depth convolutional network | |
| KR102197946B1 (en) | object recognition and counting method using deep learning artificial intelligence technology | |
| WO2022126669A1 (en) | Subway pedestrian flow network fusion method based on video pedestrian recognition, and pedestrian flow prediction method | |
| CN106023605B (en) | A kind of method for controlling traffic signal lights based on depth convolutional neural networks | |
| CN101577812B (en) | Method and system for post monitoring | |
| CN109326124A (en) | A kind of urban environment based on machine vision parks cars Activity recognition system | |
| Ren et al. | Detecting and positioning of traffic incidents via video‐based analysis of traffic states in a road segment | |
| CN109670404A (en) | A kind of road ponding image detection method for early warning based on mixed model | |
| CN103985182B (en) | A kind of bus passenger flow automatic counting method and automatic counter system | |
| CN106373426A (en) | Computer vision-based parking space and illegal lane occupying parking monitoring method | |
| CN103986910A (en) | Method and system for passenger flow statistics based on cameras with intelligent analysis function | |
| CN109637151A (en) | A kind of recognition methods that highway Emergency Vehicle Lane is driven against traffic regulations | |
| CN104809887B (en) | A kind of retrograde detection method of vehicle on expressway and autoalarm | |
| CN105930800A (en) | Lane line detection method and device | |
| CN103208190B (en) | Traffic flow detection method based on object detection | |
| CN105608889B (en) | A kind of stoppage of vehicle analysis method | |
| CN114267082B (en) | Bridge side falling behavior identification method based on depth understanding | |
| CN104320617A (en) | All-weather video monitoring method based on deep learning | |
| CN104134067A (en) | Road vehicle monitoring system based on intelligent visual Internet of Things | |
| CN109410583A (en) | Traffic information collection and analysis system and operation method based on automobile data recorder | |
| CN103632427A (en) | Gate cracking protection method and gate control system | |
| CN105957341A (en) | Wide area traffic jam detection method based on unmanned plane airborne platform | |
| CN113450573A (en) | Traffic monitoring method and traffic monitoring system based on unmanned aerial vehicle image recognition | |
| Bharadwaj et al. | Traffic data analysis using image processing technique on Delhi–Gurgaon expressway |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |