CN109902676A - A kind of separated based on dynamic background stops detection algorithm - Google Patents
A kind of separated based on dynamic background stops detection algorithm Download PDFInfo
- Publication number
- CN109902676A CN109902676A CN201910029090.XA CN201910029090A CN109902676A CN 109902676 A CN109902676 A CN 109902676A CN 201910029090 A CN201910029090 A CN 201910029090A CN 109902676 A CN109902676 A CN 109902676A
- Authority
- CN
- China
- Prior art keywords
- frame
- region
- network
- detection
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of based on dynamic background separated stops detection algorithm, acquired image is screened, retain effective image and marks, the example segmentation network and associated region constructed in the neural network based on dynamic background convolution proposes network, the model parameter of effect optimal network is obtained after training, by effective image input effect optimal network, vehicle after being divided can stop region and disobey to stop region, when vehicle is less than preset threshold with the degree of overlapping that can stop regionaAnd stop region degree of overlapping greater than threshold value with disobeyingbWhen, then it is judged to disobeying parking.The present invention can be also applicable in the collected picture of revocable picture pick-up device, and the pavement background of scene complexity can be precisely partitioned into vehicle and can be stopped and stop region with separated；Object region that may be present is predicted for the connection between the picture that front and back is shot in time sequencing, and model constantly iteration update can enhance its robustness；The detection success rate under circumstance of occlusion is improved, model accuracy greatly promotes.
Description
Technical field
The present invention relates to data identifications；Data indicate；Record carrier；The technical field for recording the processing of carrier, especially relates to
And a kind of separated based on dynamic background stops detection algorithm.
Background technique
With the promotion of urban development and disposal income of Chinese people, the quantity of vehicle rises year by year, and has aggravated road incessantly
The pressure of traffic, simultaneously because the hysteresis quality of parking stall quantity matching aspect, the problem of " parking difficulty ", are becoming increasingly acute, car owner in order to
The convenience of itself often utilizes the blind area in road monitoring region, and vehicle is stopped into manway.It is normal that separated parking hinders pedestrian
Travel region and blind trade, there are the potential risks that No Through Route for Pedestrians enters non-motorized lane or even car lane.In order to prevent this
The separated parking of the generation of kind phenomenon, traffic department's meeting regular visit both sides of the road pavement simultaneously carries out image using mobile device
Acquisition, but artificial patrol supervision is disobeyed and stops there are problems that efficiency is lower and takes long time to collect evidence to take pictures.
Disobey using intelligent image analysis and stop the developing direction that detection is future city traffic administration, disobeys and stop detection calculation
Method is one such.Disobey that stop detection algorithm be that collected reality scene picture is trained modeling and obtains for a kind of basis
Model algorithm, complete training after can detect newly to collect in image automatically automatically according to the obtained model parameter of training
With the presence or absence of separated parking, and precise positioning segmentation is carried out in the picture.
Paper " Rich feature hierarchies for accurate object detection and
Semantic segmentation " is that convolutional neural networks (CNN) method is introduced object detection field, substantially increases mesh
Mark detection effect, it may be said that change the main Research Thinking of object detection field, serial article followed by includes RCNN
(Region-based Convolutional Neural Networks), Fast-RCNN and Faster-RCNN, meanwhile, 2017
Year, the author of Faster-RCNN had also been proposed Mask-RCNN, and this represent the current highest levels in this field.
Existing disobey stops detection algorithm mostly and is to carry out under the premise of having detected that vehicle, and fixed both for road
Monitor camera device carries out frame difference method, background subtraction etc. according to video sequence and vehicle detecting information to obtain vehicle
More specific location information.However, these methods are affected by illumination, monitoring device stability, and rely on to detection scene
Background modeling will affect the effect of vehicle detection when scene is more complicated, vehicle is because when deformation occurs the reason of shooting angle
Fruit, so as to cause the failure for stopping detection algorithm is disobeyed.
In recent years, with the development of deep learning, road traffic detection field, such as Publication No. is had also been introduced in CNN
The patent of invention of CN107609491A proposes a kind of separated based on convolutional neural networks and stops vehicle detecting algorithm, but the algorithm is only
Deep learning is applied to the identification of vehicle, it can not there are have when the case where more vehicles for disobeying parking in image
The positioning of effect is distinguished；Therefore, there are still following deficiencies for the prior art:
1) common CNN image recognition is unable to complete the ability of detection and localization；
2) for disobeying parking, it would be desirable to be directly targeted to specific separated parking from acquired image, and wish to
Vehicle, the license board information of extracting vehicle, this requires the recognition detection functions for high-resolution pictures, and for common
CNN network be typically all to pass through size scaling and handled, carry out precise positioning presence for reverting back high-resolution size
It is difficult；
3) pavement parking is intensive and there is the object mutually blocked, detect classification be not enough accurately, and this for
The accurate segmentation of generic different vehicle also proposed requirement；
4) when acquiring picture, the tandem in picture having time has the connection in content, common target between picture
Convolutional network is detected just for a width picture, does not utilize the related information between the two images of front and back.
Summary of the invention
To solve the above-mentioned problems, the invention proposes a kind of optimizations based on dynamic background separated stops detection algorithm, energy
Continuous iteration updates enhancing robustness.Fusion is based on the improved non-maximum restraining method C- of Soft-NMS on example segmentation network
Soft-NMS promotes the detection success rate under circumstance of occlusion.
The technical scheme adopted by the invention is that a kind of separated based on dynamic background stops detection algorithm, the method includes
Following steps:
Step 1: acquired image being screened, retains effective image, is labeled；
Step 2: example segmentation network and associated region in neural network of the building based on dynamic background convolution propose net
Network, training, setting loss function obtain the model parameter of effect optimal network；
Step 3: by effective image input effect optimal network, vehicle after being divided can stop region and disobey to stop region；
Step 4: detecting the vehicle after dividing and region and the separated degree of overlapping for stopping region can be stopped, when the vehicle and area can be stopped
The degree of overlapping in domain be less than preset threshold a and with disobey stop region degree of overlapping greater than threshold value b when, be judged to disobeying parking；0 < a < 1,0
< b < 1.
Preferably, it in the step 1, after being screened to acquired image, is pre-processed, and retain effectively figure
Picture.
Preferably, in the step 3, effective image in effect optimal network the following steps are included:
Step 3.1: by treated, effective image input example is divided in the residual error network of network, exports residual error characteristic pattern
IRES；
Step 3.2: by residual error characteristic pattern IRESInput area proposes that network, removal low confidence are less than the proposal frame of q, with
Remaining proposal frame WjPropose frame output, W as prospectj=IRES·HRPN, wherein HRPNIndicate the operation of proposing offers frame；0 < q
< 0.5；
Step 3.3: if processed number of image frames is more than or equal to 2, the image that front cross frame has been detected inputs association area
Network is proposed in domain, and associated region is proposed that the prospect frame proposal of network output is denoted as Wr, carry out in next step, otherwise, directly carry out
Step 3.5；
Step 3.4: by WjAnd WrThe anchor frame of overlapping input in improved Soft-NMS algorithm, removal overlapping exports prospect
Propose frame Wf；
Step 3.5: prospect being proposed that frame is unified for identical size by interest pool area, inputs full articulamentum and full volume
Product network；
Step 3.6: the mask that output is classified based on pixel scale.
Preferably, the step 3.3 the following steps are included:
Step 3.3.1: the former frame of present frame and the existing vehicle detection frame of the first two frame are obtained, is denoted as B respectively1, iWith
B2, j, wherein i and j respectively represents ith and jth detection block；
Step 3.3.2: by B1, iAnd B2, jThe shared convolutional network of weight is inputted, feature is extracted；
Step 3.3.3: the feature extracted is unified for identical size by interest pool area；
Step 3.3.4: it will test frame and carry out mutual convolution operation, choose B1, iWith B2All detection blocks carry out convolution operation,
It obtains representing detection block B1, iWith B2, jSimilarity Confidence queue Sj, reject detection of the confidence level less than 0.6 < p < 0.8
Frame records the detection block with highest confidence level, and enabling the index of the detection block with highest confidence level is m；
Step 3.3.5: determine that former frame matches the region A that frame is moved with the first two framem；
Step 3.3.6: according to moving area AmThere is matched corresponding detection block B in former frame1, mDetermine region of search As；
Step 3.3.7: by region of search AsIt is cut in original image, obtains the matching for corresponding to region of search in former frame
Frame B1, m, it is sent to the shared convolutional neural networks of weight and carries out feature extraction, obtains respective characteristic pattern, be denoted as AfWith
B1, f；
Step 3.3.8: it will test the characteristic pattern B of frame1, fIn region of search characteristic pattern AfOn carry out intervolving long-pending operation, obtain
Confidence level figure D chooses the highest region of confidence level as the prospect frame proposal output for proposing that network is proposed in region addition region.
Preferably, the step 3.3.5 the following steps are included:
Step 3.3.5.1: the corresponding upper left corner that two match frame and bottom right angular coordinate are converted to corresponding centroid and sat
Mark (x, y) and rectangle frame length and width h, w；
Step 3.3.5.2: moving area is obtained according to the relative position of two rectangle frame centroidsWherein, coor represents the coordinate (x, y) of certain point, i
The rectangle frame of the first two frame to match is represented, j represents the rectangle frame of former frame；
Step 3.3.5.3: by coormovCoordinate is converted to corresponding centroid coordinate and rectangle frame length and width, with (xm, ym) and
wm、hmIt indicates, the region A as movedm。
Preferably, the step 3.3.6 the following steps are included:
Step 3.3.6.1: amount of movement the Δ x, Δ y that the centroid of frame is accordingly matched in front cross frame are calculated；
Step 3.3.6.2: by moving area AmCentroid coordinate respectively correspond plus amount of movement Δ x and Δ y, searched for
Region As。
Preferably, the step 3.4 the following steps are included:
Step 3.4.1: by WjAnd WrThe anchor frame of overlapping be defined as several detection blocks；
Step 3.4.2: it will test frame according to confidence level and carry out descending arrangement, the queue after sequence is denoted as L1；Initialization
Empty queue is denoted as L2；
Step 3.4.3: queue L is checked1Whether it is sky, then carries out in next step, otherwise carrying out step 3.4.7 if not empty；
Step 3.4.4: by queue L1The middle highest detection block of confidence level is denoted as Wmax, calculate remaining detection block and WmaxWeight
Folded degree, updates the confidence level of detection blockWherein, biIt represents
The current detection block for carrying out operation, iou is degree of overlapping；
Step 3.4.5: in queue L1Middle detection block of the confidence level lower than 0.1 after deleting update；
Step 3.4.6: by WmaxIt is placed in candidate queue L2And from queue L1Middle deletion, return step 3.4.3:
Step 3.4.7: queue L is returned2As final result.
Preferably, the step 3.5 the following steps are included:
Step 3.5.1: the prospect for being unified for identical size is proposed that frame inputs full articulamentum, output prospect proposes the class of frame
Other and prospect proposes the coordinate (x, y) in the frame upper left corner and the lower right corner, wherein y=Xx, y are the vector of n × 1, and n is classification number,
X is the matrix of n × m, and x is the vector of m × 1, and m is dimension when entering full articulamentum；
Step 3.5.2: coordinate is inputted into full convolutional network, is exported z=g (y), wherein g is to have exchanged forward and backward to ask
The convolutional layer of derived function.
Preferably, the step 4 the following steps are included:
Step 4.1: the vehicle example after traversing all segmentations；
Step 4.2: detecting the detection block of vehicle and disobey the IoU for stopping region detection frame, region detection frame being stoppedIt disobeys and stopsWith IoUIt can stop；
Step 4.3: working as IoUIt disobeys and stops> b and IoUIt can stopWhen < a, determines that vehicle example is to disobey parking, indexed and team is added
Arrange Lw, otherwise, non-separated parking；It carries out in next step；
Step 4.4: if traversal is not finished, otherwise return step 1 terminates detection and returns to queue Lw。
The present invention provides a kind of optimizations based on dynamic background it is separated stop detection algorithm, by acquired image into
Row screening retains effective image and marks, and constructs example segmentation network and pass in the neural network based on dynamic background convolution
Join region and propose network, the model parameter that effect optimal network is obtained after training obtains effective image input effect optimal network
Vehicle after to segmentation can stop region and disobey to stop region, the vehicle after detection segmentation with can stop region and separated stop the overlapping of region
Degree, when vehicle be less than with the degree of overlapping that can stop region preset threshold a and with disobey stop region degree of overlapping greater than threshold value b when, then determine
To disobey parking.
The invention has the advantages that playing effective booster action for intelligent city traffic administration:
(1) the phenomenon that stopping is disobeyed for vehicle propose a kind of effective algorithm, it is collected with revocable picture pick-up device
Picture can be also applicable in, and the pavement background of scene complexity can be precisely partitioned into vehicle and can be stopped and stop region with separated；
(2) associated region is devised for vehicle missing inspection situation and propose network, for the figure that front and back is shot in time sequencing
Connection between piece predicts object region that may be present, and model can continuous its robustness of iteration update enhancing；
(3) it has merged in example segmentation network and has been calculated based on the improved non-maximum restraining method C-Soft-NMS of Soft-NMS
Method, improves the detection success rate under circumstance of occlusion, and model accuracy greatly promotes.
Detailed description of the invention
Fig. 1 is flow chart of the invention；
Fig. 2 is the flow chart that associated region proposes network in the present invention；
Fig. 3 is the flow chart of improved Soft-NMS algorithm in the present invention.
Specific embodiment
The present invention is described in further detail below with reference to example and attached drawing, but protection scope of the present invention and unlimited
In this.
The present invention relates to a kind of based on dynamic background it is separated stop detection algorithm, the use of dynamic background modeling is typically all to exist
The interference of complex background is eliminated in video successive frame for specific moving object, and then promotes the precision of detection, but for quiet
Only specific objective, traditional dynamic background method are applied less；And the development of convolutional neural networks is then that have passed through repetition member
The shallow-layer network (VGG) of element arrives the residual error depth network (ResNet) by improvement, can extract foot to different tasks
Pictorial information enough is enriched, while also proposed region on this basis and proposing network (RPN) to be directed to the requirement of target positioning.
It the described method comprises the following steps.
Step 1: acquired image being screened, retains effective image, is labeled.
In the step 1, after being screened to acquired image, pre-processed, and retain effective image.
In the present invention, effective image refers to the image in image there are pavement, when occurring non-pavement element in image,
Effective image is not belonging to if when crossroad.Certainly, in a practical situation, based on disobeying the judgement range stopped, effective image
Criterion can be adjusted according to actual needs.
In the present invention, mark refers to the effect for realizing emphasis position mark, i.e., informs which kind of mark algorithm needs to reach in advance
It is quasi-.
In the present invention, pretreatment generally refers to Image Adjusting size to be detected short side is such as scaled 800, long side
No more than 1300, in any case, it is also necessary to carry out denoising.Those skilled in the art can voluntarily set according to demand
It sets.
Step 2: example segmentation network and associated region in neural network of the building based on dynamic background convolution propose net
Network, training, setting loss function obtain the model parameter of effect optimal network.
In the present invention, it includes that residual error extracts network (RESNet), region is proposed network (RPN), improved that example, which divides network,
Non-maxima suppression method (C-Soft-NMS), full convolutional network (FCN) and the output of full articulamentum when required can be with
According to different task, corresponding loss function is set；And associated region propose network (CPN) include region of search limit (SRR),
Mutual convolutional network (CCN), it is proposed that prospect frame divides network training with secondary instance, and similitude ratio can also be arranged when required
Compared with loss function.Specifically, mode of the invention is to be partitioned into target with example segmentation network, and propose with associated region
Network carries out precision adjustment, so that the result of output is more accurate.
In the present invention, the convolutional layer kernel size that residual error extracts network (RESNet) can be set to a × a, a=1 or 3；
Propose that the convolutional layer kernel size of network (RPN) can be set to b × b, b=3,5 or 7 in region；Full convolutional network uses transposition
Convolution, convolutional layer kernel can be set to 2 × 2.It can be by those skilled in the art's root herein for the size of convolutional layer kernel
Optimal network is obtained according to actual demand self-setting and by training.
In the present invention, the model parameter of effect optimal network refers to the optimal model parameter of effect, i.e., in repetitive exercise mistake
Make the parameter in the smallest convolution kernel of loss in journey, can use ginseng of the cross entropy loss function as loss to model herein
Number asks gradient to be updated, this is the routine techniques of those skilled in the art, and those skilled in the art can select according to demand
And it is arranged.
Step 3: by effective image input effect optimal network, vehicle after being divided can stop region and disobey to stop region.
In the step 3, effective image includes the following steps in effect optimal network.
Step 3.1: by treated, effective image input example is divided in the residual error network of network, exports residual error characteristic pattern
IRES。
Step 3.2: by residual error characteristic pattern IRESInput area proposes that network, removal low confidence are less than the proposal frame of q, with
Remaining proposal frame WjPropose frame output, W as prospectj=IRES·HRPN, wherein HRPNIndicate the operation of proposing offers frame；0 < q
< 0.5.
In the present invention, under normal circumstances, q can take 0.3.
Step 3.3: if processed number of image frames is more than or equal to 2, the image that front cross frame has been detected inputs association area
Network is proposed in domain, and associated region is proposed that the prospect frame proposal of network output is denoted as Wr, carry out in next step, otherwise, directly carry out
Step 3.5.
The step 3.3 includes the following steps.
Step 3.3.1: the former frame of present frame and the existing vehicle detection frame of the first two frame are obtained, is denoted as B respectively1, iWith
B2, j, wherein i and j respectively represents ith and jth detection block.
In the present invention, refer to the frame of the vehicle detected with B, former frame is referred to " 1 ", " 2 " refer to preceding 2 frame.
Step 3.3.2: by B1, iAnd B2, jThe shared convolutional network of weight is inputted, feature is extracted.
In the present invention, step 3.3.2 selects VGG network to extract feature.
Step 3.3.3: the feature extracted is unified for identical size by interest pool area.It is 17 in the present embodiment
×17。
Step 3.3.4: it will test frame and carry out mutual convolution operation, choose B1, iWith B2All detection blocks carry out convolution operation,
It obtains representing detection block B1, iWith B2, jSimilarity Confidence queue Sj, reject detection of the confidence level less than 0.6 < p < 0.8
Frame records the detection block with highest confidence level, and enabling the index of the detection block with highest confidence level is m.
In the present invention, with B1, iWith B2All detection blocks carry out convolution operation be with B1In any one detection block and
B2In all vehicle detection frame operated.Under normal circumstances, desirable 0.8 p, index " m " refer to match.
Step 3.3.5: determine that former frame matches the region A that frame is moved with the first two framem。
The step 3.3.5 includes the following steps.
Step 3.3.5.1: the corresponding upper left corner that two match frame and bottom right angular coordinate are converted to corresponding centroid and sat
Mark (x, y) and rectangle frame length and width h, w.
Step 3.3.5.2: moving area is obtained according to the relative position of two rectangle frame centroidsWherein, coor represents the coordinate (x, y) of certain point, i
The rectangle frame of the first two frame to match is represented, j represents the rectangle frame of former frame.
Step 3.3.5.3: by coormovCoordinate is converted to corresponding centroid coordinate and rectangle frame length and width, with (xm, ym) and
wm、hmIt indicates, the region A as movedm。
In the present invention, the coordinate for matching corresponding four angles of frame can be determined by corresponding centroid coordinate and length and width, turnover zone
Domain is similarly.
Step 3.3.6: according to moving area AmThere is matched corresponding detection block B in former frame1, mDetermine region of search As。
The step 3.3.6 includes the following steps.
Step 3.3.6.1: amount of movement the Δ x, Δ y that the centroid of frame is accordingly matched in front cross frame are calculated.
Step 3.3.6.2: by moving area AmCentroid coordinate respectively correspond plus amount of movement Ax and Ay, obtain the field of search
Domain As。
In the present invention, region of search AsRefer to it is being in the region of search of former frame i.e. moving area AmOn, according to offset
Estimate the regional scope in present frame.
Step 3.3.7: by region of search AsIt is cut in original image, obtains the matching for corresponding to region of search in former frame
Frame B1, m, it is sent to the shared convolutional neural networks of weight and carries out feature extraction, obtains respective characteristic pattern, be denoted as AfWith
B1, f。
It is corresponding with step 3.3.2 in the present invention, select VGG network to extract feature herein.
Step 3.3.8: it will test the characteristic pattern B of frame1, fIn region of search characteristic pattern AfOn carry out intervolving long-pending operation, obtain
Confidence level figure D chooses the highest region of confidence level as the prospect frame proposal output for proposing that network is proposed in region addition region.
Step 3.4: by WjAnd WrThe anchor frame of overlapping input in improved Soft-NMS algorithm, removal overlapping exports prospect
Propose frame Wf。
In the present invention, step 3.4 is with the confidence level of improved non-maximum restraining method (C-Soft-NMS) modification detection block.
Optimal detection can be retained in the case where due to common non-maximum restraining method (NMS) overlapping detection blocks a large amount of for presence
Frame, but the detection leakage phenomenon that can be generated when two cars are overlapped close using common NMS, Soft-NMS is for overlapping vehicle
Although detection has certain effect, the linear attenuation method provided is not accurate enough for practical application, and WjAnd WrIt is middle to exist largely
The anchor frame of overlapping, so when needing for vehicle overlapping using only with improved non-maximum restraining method for the missing inspection that will appear
Situation is handled.
The step 3.4 includes the following steps.
Step 3.4.1: by WjAnd WrThe anchor frame of overlapping be defined as several detection blocks.
Step 3.4.2: it will test frame according to confidence level and carry out descending arrangement, the queue after sequence is denoted as L1；Initialization
Empty queue is denoted as L2。
In the present invention, confidence level is the confidence level for belonging to such for detecting network and calculating.
Step 3.4.3: queue L is checked1Whether it is sky, then carries out in next step, otherwise carrying out step 3.4.7 if not empty.
Step 3.4.4: by queue L1The middle highest detection block of confidence level is denoted as Wmax, calculate remaining detection block and WmaxWeight
Folded degree, updates the confidence level of detection blockWherein, biIt represents
The current detection block for carrying out operation, iou is degree of overlapping.
In the present invention, the confidence level of the detection block to IoU greater than 0.5 carries out round curvature type decaying.
Step 3.4.5: in queue L1Middle detection block of the confidence level lower than 0.1 after deleting update.
In the present invention, when confidence level is lower than 0.1, non-targeted frame can be considered as, therefore deleted.
Step 3.4.6: by WmaxIt is placed in candidate queue L2And from queue L1Middle deletion, return step 3.4.3.
Step 3.4.7: queue L is returned2As final result.
Step 3.5: prospect being proposed that frame is unified for identical size by interest pool area, inputs full articulamentum and full volume
Product network.
In the present invention, because of WfThere is the object for needing further identification classification in frame, need first to be passed through interest
Pool area (ROI Pooling) is unified for same size, and such as 14 × 14.
In the present invention, full convolutional network (FCN) and full articulamentum are all the output objects in order to finally divide example respectively
Mask, rectangle surround frame, object category, wherein full convolutional network has used transposition convolution, and convolutional layer kernel is 2 × 2, Quan Lian
Connect the coordinate that layer output is respectively the specific category of object and the upper left corner of area-encasing rectangle frame and the lower right corner.
The step 3.5 includes the following steps.
Step 3.5.1: the prospect for being unified for identical size is proposed that frame inputs full articulamentum, output prospect proposes the class of frame
Other and prospect proposes the coordinate (x, y) in the frame upper left corner and the lower right corner, wherein y=Xx, y are the vector of n × 1, and n is classification number,
X is the matrix of n × m, and x is the vector of m × 1, and m is dimension when entering full articulamentum.
In the present invention, output prospect propose frame classification include foreground and background, that is, disobey stop region, vehicle, it is non-disobey stop region,
Stopped with this to distinguish vehicle with the presence or absence of separated.
In the present invention, n=4 when the coordinate of output box needs four information when determining a detection block, is left respectively
The centre coordinate of upper and the lower right corner the coordinate either frame lengthens width, both expression require n=4.
In the present invention, m is dimension when entering full articulamentum, such as 2048.
Step 3.5.2: coordinate is inputted into full convolutional network, is exported z=g (y), wherein g is to have exchanged forward and backward to ask
The convolutional layer of derived function.
In the present invention, the input of full convolutional network is the image-region in each detection block, in order in the sky for keeping picture
Between the mask (mask) classified based on pixel scale is exported under the premise of shape, need using full convolutional network.
In the present invention, the core of full convolutional network is transposition convolutional layer, it is assumed that f is a convolutional layer, gives input x, can
It is preceding to output y=f (x) to calculate；In reversed derivationIt is known that z can obtain the output of the shape as x；And
Because the derivative of convolution algorithm is oneself itself, it can be denoted as g, to have exchanged forward and backward with legal definition transposition convolutional layer
The convolutional layer of derivation function.Namely z=g (y), after full convolutional network, the rectangle of object, which surrounds in frame, can export prediction
The mask of object edge.
Step 3.6: the mask that output is classified based on pixel scale.
In the present invention, mask is to divide in detection block to the edge of target.
Step 4: detecting the vehicle after dividing and region and the separated degree of overlapping for stopping region can be stopped, when the vehicle and area can be stopped
The degree of overlapping in domain be less than preset threshold a and with disobey stop region degree of overlapping greater than threshold value b when, be judged to disobeying parking；0 < a < 1,0
< b < 1.
The step 4 the following steps are included:
Step 4.1: the vehicle example after traversing all segmentations；
Step 4.2: detecting the detection block of vehicle and disobey the IoU for stopping region detection frame, region detection frame being stoppedIt disobeys and stopsWith IoUIt can stop；
Step 4.3: working as IoUIt disobeys and stops> b and IoUIt can stopWhen < a, determines that vehicle example is to disobey parking, indexed and team is added
Arrange Lw, otherwise, non-separated parking；It carries out in next step；
Step 4.4: if traversal is not finished, otherwise return step 1 terminates detection and returns to queue Lw。
In the present invention, a can use 0.5, b desirable 0.3.
For the present invention by screening, retaining effective image and marking to acquired image, building is based on dynamic background
Example segmentation network and associated region in the neural network of convolution propose network, and the model of effect optimal network is obtained after training
Parameter, by effective image input effect optimal network, vehicle after being divided can stop region and disobey to stop region, detection segmentation
Rear vehicle stops the degree of overlapping in region with that can stop region and disobeying, when vehicle with can stop the degree of overlapping in region less than preset threshold a and
With disobey stop region degree of overlapping greater than threshold value b when, then be judged to disobeying parking.
The present invention disobeys the phenomenon that stopping for vehicle and proposes a kind of effective algorithm, is collected with revocable picture pick-up device
Picture can also be applicable in, for the pavement background of scene complexity can precisely be partitioned into vehicle and can stop with disobey stop region；
Associated region, which is devised, for vehicle missing inspection situation proposes network, it is pre- for the connection between the picture that front and back is shot in time sequencing
Object region that may be present is surveyed, and model can continuous its robustness of iteration update enhancing；Melt in example segmentation network
The detection success improved under circumstance of occlusion based on the improved non-maximum restraining method C-Soft-NMS algorithm of Soft-NMS is closed
Rate, model accuracy greatly promote.Effective booster action is played for intelligent city traffic administration.
Claims (9)
1. a kind of separated based on dynamic background stops detection algorithm, it is characterised in that: the described method comprises the following steps:
Step 1: acquired image being screened, retains effective image, is labeled；
Step 2: example segmentation network and associated region in neural network of the building based on dynamic background convolution propose network, instruction
Practice, setting loss function obtains the model parameter of effect optimal network；
Step 3: by effective image input effect optimal network, vehicle after being divided can stop region and disobey to stop region；
Step 4: detecting the vehicle after dividing and region and the separated degree of overlapping for stopping region can be stopped, when the vehicle and region can be stopped
Degree of overlapping be less than preset threshold a and with disobey stop region degree of overlapping greater than threshold value b when, be judged to disobeying parking；0 < a <, 1,0 < b
< 1.
2. a kind of separated based on dynamic background according to claim 1 stops detection algorithm, it is characterised in that: the step 1
In, it after being screened to acquired image, is pre-processed, and retain effective image.
3. a kind of separated based on dynamic background according to claim 1 stops detection algorithm, it is characterised in that: the step 3
In, effective image in effect optimal network the following steps are included:
Step 3.1: by treated, effective image input example is divided in the residual error network of network, exports residual error characteristic pattern IRES；
Step 3.2: by residual error characteristic pattern IRESInput area proposes network, and removal low confidence is less than the proposal frame of q, with remaining
Proposal frame WjPropose frame output, W as prospectj=IRES·HRPN, wherein HRPNIndicate the operation of proposing offers frame；0 < q <
0.5；
Step 3.3: if processed number of image frames is more than or equal to 2, the image input associated region that front cross frame has detected being mentioned
Network is discussed, associated region is proposed that the prospect frame proposal of network output is denoted as Wr, carry out in next step, otherwise, directly carry out step
3.5；
Step 3.4: by WjAnd WrThe anchor frame of overlapping input in improved Soft-NMS algorithm, removal overlapping, output prospect is proposed
Frame Wf；
Step 3.5: prospect being proposed that frame is unified for identical size by interest pool area, inputs full articulamentum and full convolution net
Network；
Step 3.6: the mask that output is classified based on pixel scale.
4. a kind of separated based on dynamic background according to claim 3 stops detection algorithm, it is characterised in that: the step
3.3 the following steps are included:
Step 3.3.1: the former frame of present frame and the existing vehicle detection frame of the first two frame are obtained, is denoted as B respectively1,iAnd B2,j,
Wherein, i and j respectively represents ith and jth detection block；
Step 3.3.2: by B1,iAnd B2,jThe shared convolutional network of weight is inputted, feature is extracted；
Step 3.3.3: the feature extracted is unified for identical size by interest pool area；
Step 3.3.4: it will test frame and carry out mutual convolution operation, choose B1,iWith B2All detection blocks carry out convolution operation, obtain
Represent detection block B1,iWith B2,jSimilarity Confidence queue Sj, reject detection block of the confidence level less than 0.6 < p < 0.8, note
The detection block with highest confidence level is recorded, enabling the index of the detection block with highest confidence level is m；
Step 3.3.5: determine that former frame matches the region A that frame is moved with the first two framem；
Step 3.3.6: according to moving area AmThere is matched corresponding detection block B in former frame1,mDetermine region of search As；
Step 3.3.7: by region of search AsIt is cut in original image, obtains the matching frame for corresponding to region of search in former frame
B1,m, it is sent to the shared convolutional neural networks of weight and carries out feature extraction, obtains respective characteristic pattern, be denoted as AfAnd B1,f；
Step 3.3.8: it will test the characteristic pattern B of frame1,fIn region of search characteristic pattern AfOn carry out intervolving long-pending operation, obtain confidence level
Scheme D, chooses the highest region of confidence level as the prospect frame for proposing region addition region proposal network and propose output.
5. a kind of separated based on dynamic background according to claim 4 stops detection algorithm, it is characterised in that: the step
3.3.5 the following steps are included:
Step 3.3.5.1: by two match frame the corresponding upper left corner and bottom right angular coordinate be converted to corresponding centroid coordinate (x,
And rectangle frame length and width h, w y)；
Step 3.3.5.2: moving area is obtained according to the relative position of two rectangle frame centroidsWherein, coor represents the coordinate (x, y) of certain point, i
The rectangle frame of the first two frame to match is represented, j represents the rectangle frame of former frame；
Step 3.3.5.3: by coormovCoordinate is converted to corresponding centroid coordinate and rectangle frame length and width, with (xm,ym) and wm、hmTable
Show, the region A as movedm。
6. a kind of separated based on dynamic background according to claim 4 stops detection algorithm, it is characterised in that: the step
3.3.6 the following steps are included:
Step 3.3.6.1: amount of movement the Δ x, Δ y that the centroid of frame is accordingly matched in front cross frame are calculated；
Step 3.3.6.2: by moving area AmCentroid coordinate respectively correspond plus amount of movement Δ x and Δ y, obtain region of search
As。
7. a kind of separated based on dynamic background according to claim 3 stops detection algorithm, it is characterised in that: the step
3.4 the following steps are included:
Step 3.4.1: by WjAnd WrThe anchor frame of overlapping be defined as several detection blocks；
Step 3.4.2: it will test frame according to confidence level and carry out descending arrangement, the queue after sequence is denoted as L1；Initialize empty team
Column, are denoted as L2；
Step 3.4.3: queue L is checked1Whether it is sky, then carries out in next step, otherwise carrying out step 3.4.7 if not empty；
Step 3.4.4: by queue L1The middle highest detection block of confidence level is denoted as Wmax, calculate remaining detection block and WmaxDegree of overlapping,
Update the confidence level of detection blockWherein, biIt represents when advance
The detection block of row operation, iou are degree of overlapping；
Step 3.4.5: in queue L1Middle detection block of the confidence level lower than 0.1 after deleting update；
Step 3.4.6: by WmaxIt is placed in candidate queue L2And from queue L1Middle deletion, return step 3.4.3；
Step 3.4.7: queue L is returned2As final result.
8. a kind of separated based on dynamic background according to claim 3 stops detection algorithm, it is characterised in that: the step
3.5 the following steps are included:
Step 3.5.1: the prospect that will be unified for identical size proposes that frame inputs full articulamentum, output prospect propose frame classification and
The coordinate (x, y) in prospect proposal the frame upper left corner and the lower right corner, wherein y=Xx, y are the vector of n × 1, and n is classification number, X n
The matrix of × m, x are the vector of m × 1, and m is dimension when entering full articulamentum；
Step 3.5.2: coordinate is inputted into full convolutional network, is exported z=g (y), wherein g is to have exchanged forward and backward derivation letter
Several convolutional layers.
9. a kind of separated based on dynamic background according to claim 1 stops detection algorithm, it is characterised in that: the step 4
The following steps are included:
Step 4.1: the vehicle example after traversing all segmentations；
Step 4.2: detecting the detection block of vehicle and disobey the IoU for stopping region detection frame, region detection frame being stoppedIt disobeys and stopsWith IoUIt can stop；
Step 4.3: working as IoUIt disobeys and stops> b and IoUIt can stopWhen < a, determines that vehicle example is to disobey parking, indexed and queue L is addedw, no
Then, non-separated parking；It carries out in next step；
Step 4.4: if traversal is not finished, otherwise return step 1 terminates detection and returns to queue Lw。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910029090.XA CN109902676B (en) | 2019-01-12 | 2019-01-12 | Dynamic background-based violation detection algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910029090.XA CN109902676B (en) | 2019-01-12 | 2019-01-12 | Dynamic background-based violation detection algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109902676A true CN109902676A (en) | 2019-06-18 |
| CN109902676B CN109902676B (en) | 2021-04-09 |
Family
ID=66943606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910029090.XA Active CN109902676B (en) | 2019-01-12 | 2019-01-12 | Dynamic background-based violation detection algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109902676B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610202A (en) * | 2019-08-30 | 2019-12-24 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN111144475A (en) * | 2019-12-22 | 2020-05-12 | 上海眼控科技股份有限公司 | Method and device for determining car seat, electronic equipment and readable storage medium |
| CN111368687A (en) * | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| CN112132037A (en) * | 2020-09-23 | 2020-12-25 | 平安国际智慧城市科技股份有限公司 | Sidewalk detection method, device, equipment and medium based on artificial intelligence |
| CN113496158A (en) * | 2020-03-20 | 2021-10-12 | 中移(上海)信息通信科技有限公司 | Object detection model optimization method, device, equipment and storage medium |
| CN114297107A (en) * | 2021-12-29 | 2022-04-08 | 成都智明达电子股份有限公司 | Management method, equipment and medium for label Tag |
| CN114566063A (en) * | 2022-01-24 | 2022-05-31 | 深圳市捷顺科技实业股份有限公司 | Intelligent parking space guiding management method and device and storage medium |
| CN115082903A (en) * | 2022-08-24 | 2022-09-20 | 深圳市万物云科技有限公司 | Non-motor vehicle illegal parking identification method and device, computer equipment and storage medium |
| CN116433988A (en) * | 2023-06-01 | 2023-07-14 | 中国标准化研究院 | Multi-source heterogeneous image data classification treatment method |
| CN114297107B (en) * | 2021-12-29 | 2024-05-24 | 成都智明达电子股份有限公司 | Label Tag management method, device and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631899A (en) * | 2015-12-28 | 2016-06-01 | 哈尔滨工业大学 | Ultrasonic image motion object tracking method based on gray-scale texture feature |
| CN107609491A (en) * | 2017-08-23 | 2018-01-19 | 中国科学院声学研究所 | A kind of vehicle peccancy parking detection method based on convolutional neural networks |
| CN107730903A (en) * | 2017-06-13 | 2018-02-23 | 银江股份有限公司 | Parking offense and the car vision detection system that casts anchor based on depth convolutional neural networks |
| CN109033950A (en) * | 2018-06-12 | 2018-12-18 | 浙江工业大学 | Vehicle based on multiple features fusion cascade deep model, which is disobeyed, stops detection method |
| CN109102678A (en) * | 2018-08-30 | 2018-12-28 | 青岛联合创智科技有限公司 | A kind of drowned behavioral value method of fusion UWB indoor positioning and video object detection and tracking technique |
-
2019
- 2019-01-12 CN CN201910029090.XA patent/CN109902676B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631899A (en) * | 2015-12-28 | 2016-06-01 | 哈尔滨工业大学 | Ultrasonic image motion object tracking method based on gray-scale texture feature |
| CN107730903A (en) * | 2017-06-13 | 2018-02-23 | 银江股份有限公司 | Parking offense and the car vision detection system that casts anchor based on depth convolutional neural networks |
| CN107609491A (en) * | 2017-08-23 | 2018-01-19 | 中国科学院声学研究所 | A kind of vehicle peccancy parking detection method based on convolutional neural networks |
| CN109033950A (en) * | 2018-06-12 | 2018-12-18 | 浙江工业大学 | Vehicle based on multiple features fusion cascade deep model, which is disobeyed, stops detection method |
| CN109102678A (en) * | 2018-08-30 | 2018-12-28 | 青岛联合创智科技有限公司 | A kind of drowned behavioral value method of fusion UWB indoor positioning and video object detection and tracking technique |
Non-Patent Citations (2)
| Title |
|---|
| KAIMING HE: "Mask R-CNN", 《ICCV2017》 * |
| NAVANEETH BODLA等: "Improving Object Detection With One Line of Code", 《ARXIV:1704.04503V1》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610202A (en) * | 2019-08-30 | 2019-12-24 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN110610202B (en) * | 2019-08-30 | 2022-07-26 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN111144475A (en) * | 2019-12-22 | 2020-05-12 | 上海眼控科技股份有限公司 | Method and device for determining car seat, electronic equipment and readable storage medium |
| CN111368687A (en) * | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| CN113496158A (en) * | 2020-03-20 | 2021-10-12 | 中移(上海)信息通信科技有限公司 | Object detection model optimization method, device, equipment and storage medium |
| CN112132037B (en) * | 2020-09-23 | 2024-04-16 | 平安国际智慧城市科技股份有限公司 | Pavement detection method, device, equipment and medium based on artificial intelligence |
| CN112132037A (en) * | 2020-09-23 | 2020-12-25 | 平安国际智慧城市科技股份有限公司 | Sidewalk detection method, device, equipment and medium based on artificial intelligence |
| CN114297107A (en) * | 2021-12-29 | 2022-04-08 | 成都智明达电子股份有限公司 | Management method, equipment and medium for label Tag |
| CN114297107B (en) * | 2021-12-29 | 2024-05-24 | 成都智明达电子股份有限公司 | Label Tag management method, device and medium |
| CN114566063A (en) * | 2022-01-24 | 2022-05-31 | 深圳市捷顺科技实业股份有限公司 | Intelligent parking space guiding management method and device and storage medium |
| CN115082903A (en) * | 2022-08-24 | 2022-09-20 | 深圳市万物云科技有限公司 | Non-motor vehicle illegal parking identification method and device, computer equipment and storage medium |
| CN116433988B (en) * | 2023-06-01 | 2023-08-08 | 中国标准化研究院 | Multi-source heterogeneous image data classification treatment method |
| CN116433988A (en) * | 2023-06-01 | 2023-07-14 | 中国标准化研究院 | Multi-source heterogeneous image data classification treatment method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109902676B (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902676A (en) | A kind of separated based on dynamic background stops detection algorithm | |
| CN110175576A (en) | A kind of driving vehicle visible detection method of combination laser point cloud data | |
| KR102279376B1 (en) | Learning method, learning device for detecting lane using cnn and test method, test device using the same | |
| CN108171112A (en) | Vehicle identification and tracking based on convolutional neural networks | |
| CN108830171B (en) | Intelligent logistics warehouse guide line visual detection method based on deep learning | |
| CN107730906A (en) | Zebra stripes vehicle does not give precedence to the vision detection system of pedestrian behavior | |
| CN106845487A (en) | A kind of licence plate recognition method end to end | |
| CN110148196A (en) | A kind of image processing method, device and relevant device | |
| CN106663193A (en) | Systems and methods for curb detection and pedestrian hazard assessment | |
| CN107085696A (en) | A kind of vehicle location and type identifier method based on bayonet socket image | |
| CN110379168B (en) | Traffic vehicle information acquisition method based on Mask R-CNN | |
| Huang et al. | Spatial-temproal based lane detection using deep learning | |
| CN110689724B (en) | Automatic motor vehicle zebra crossing present pedestrian auditing method based on deep learning | |
| CN108875754B (en) | Vehicle re-identification method based on multi-depth feature fusion network | |
| Tümen et al. | Intersections and crosswalk detection using deep learning and image processing techniques | |
| CN111461221B (en) | Multi-source sensor fusion target detection method and system for automatic driving | |
| CN110427797B (en) | Three-dimensional vehicle detection method based on geometric condition limitation | |
| CN109886215A (en) | The cruise of low speed garden unmanned vehicle and emergency braking system based on machine vision | |
| Chao et al. | Multi-lane detection based on deep convolutional neural network | |
| CN106446785A (en) | Passable road detection method based on binocular vision | |
| CN110378210A (en) | A kind of vehicle and car plate detection based on lightweight YOLOv3 and long short focus merge distance measuring method | |
| Balaska et al. | Enhancing satellite semantic maps with ground-level imagery | |
| CN106228570A (en) | A kind of Truth data determines method and apparatus | |
| CN113516664A (en) | Visual SLAM method based on semantic segmentation dynamic points | |
| CN106980855A (en) | Traffic sign quickly recognizes alignment system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |