CN106096531A

CN106096531A - A kind of traffic image polymorphic type vehicle checking method based on degree of depth study

Info

Publication number: CN106096531A
Application number: CN201610397819.5A
Authority: CN
Inventors: 程鹏
Original assignee: Anhui Yunli Information Technology Co ltd
Current assignee: Anhui Guolian Information Technology Co ltd
Priority date: 2016-05-31
Filing date: 2016-05-31
Publication date: 2016-11-09
Anticipated expiration: 2036-05-31
Also published as: CN106096531B

Abstract

The invention discloses a kind of traffic image polymorphic type vehicle checking method based on degree of depth study, first the feature of neutral net is combined with algorithm of generating layered regions, the convolutional layer using neutral net is simultaneously achieved Area generation and two processes of regional determination, then the moving region using background model to carry out the discrete series image for special scenes is judged to that Area generation provides extra reference frame, and combine vehicle detection result background model has been carried out the renewal correction of point situation, in addition, also proposed network model's compression scheme and carry out the reduction of model parameter and the time of calculating, and propose the non-maxima suppression scheme of the new testing result optimization means replacement routine calculated based on grouping error, improve overall accuracy of detection.

Description

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study

Technical field

The present invention relates to technical field of vehicle detection, a kind of traffic image polymorphic type vehicle based on degree of depth study Detection method.

Background technology

Along with the development of society, the living standard of people is all greatly improved in " clothing, food, lodging and transportion--basic necessities of life " various aspects, embodies The automobile total quantity that the road infrastructure construction of various places is improved further with society on " OK " the most quickly increases.But road Road construction is a long-term process, needs accumulation for a long time to can be only achieved effect, does not the most often catch up with various places motor vehicles Growth rate, the solution of a difficult problem is to use the technological means of more science to carry out management road traffic, here it is intelligent Traffic system.Intelligent transportation system can add up the flow of vehicle on road, identify the travel route of vehicle, then by adjusting Traffic light time on route, special one-way road is set, the means such as passed through type of vehicle that limit road carry out road improvement Traffic status.And among this, vehicle testing techniques is the most important ingredient of intelligent transportation, it is follow-up relevant treatment Basis.While detection vehicle, the type of vehicle of the personnel intuitively reflected on route with freight traffic is examined with vehicle Survey combines, and forms the detecting system of a polymorphic type vehicle, can complete the management role of basic road traffic, for rear Continuous process provides the data message more enriched.

The process object of existing vehicle testing techniques mainly has two, i.e. video and image.For video, typically pass through Use background modeling algorithm to realize real-time moving vehicle detection, obtain the vehicle moving region in image, then utilize shade Information, car light information or vehicle window information obtain final vehicle sections image, complete vehicle detection process based on video.Phase Closing background modeling algorithm mainly have mixed Gauss model, ViBe algorithm or directly set background image, it is right that these schemes process As for continuous print sequence of video images, the subject matter existed is that model is easily subject to the various external condition such as illumination, weather Affect and cause accuracy, and continuously carrying out of system will bring substantial amounts of data and energy consumption.

Vehicle detection for image works, and relates generally to characteristic Design and selects two aspects with grader.The spy of vehicle Take over for use in the appearance information describing vehicle, use suitable characteristics can obtain vehicle and be different from the peculiar outward appearance of other types object Information.The most commonly used feature is mainly engineer, has Haar-like feature, SIFT feature, HOG feature, LBP Feature, Gabor characteristic and their improvement etc..Vehicle classification device then realizes sentencing with background area for vehicle region Disconnected, the follow-up judgement that can also realize vehicle type.Existing frequently-used grader mainly has SVM, AdaBoost, KNN etc. Deng.Also have class algorithm based on coupling to realize the detection of vehicle, mainly utilize mating of feature between image-region with template Degree judges, and actually mating from both classification is the different implementations that similarity judges, it is possible to regard as Similar processing method.The use of deep neural network in recent years promotes quickly sending out of the various application of computer vision field Exhibition, its application mode in vehicle detection is the most identical with the algorithm of target detection, i.e. by based on image bottom-up information Area generation judges to realize vehicle detection based on degree of depth study plus classification based on convolutional neural networks.

In a word, existing vehicle testing techniques means based on image or need use hand-designed complex characteristic carry For basis for estimation, or need Area growth based on image bottom-up information to provide data source, be not enough to reply extensive The process requirement of view data, so needing the accurate and quick vehicle of a kind of large-scale image data for many scenes Detection method.

Summary of the invention

The technical problem to be solved in the present invention is to provide a kind of traffic image polymorphic type vehicle detection based on degree of depth study Method.

The technical scheme is that

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study, includes training process and tested Journey:

Described training process has and includes following steps:

(1), information of vehicles mark: all original images that collection different images acquisition terminal obtains process, mark Publish picture as in the location coordinate information of vehicle, and add the simple judged result of type of vehicle and the general direction information of headstock, And all images and above-mentioned all markup informations are stored in data base；

(2), Background Modeling: for different image acquisition terminals, respectively collect training data formed consecutive frame it Between the image sequence of discontinuous change, then according to the difference of image entirety bright-dark degree, be divided into daytime and night two The situation of kind, uses gauss hybrid models to set up background model respectively；

(3), training Area generation initial model: use degree of deep learning algorithm that all training images are carried out image and divide Cut, obtain the partition data of all objects occurred in image, utilize the partition data the obtained LPO to traditional Area generation Algorithm carries out the adjustment in step (1) on data base, obtains Area generation initial model；

(4), training vehicle judges network: utilize the result of convolutional neural networks and Area generation initial model, with vehicle Position coordinates, vehicle type information and headstock angle information, as the output of network, carry out vehicle and judge the multitask connection of network Close study, obtain eurypalynous vehicle and judge network；

(5), training Area generation network: the convolutional layer of network, as feature, is carried out based on sliding window to utilize vehicle to judge The car that has of mouth judges the training with coordinate optimizing network without car two classification, obtains being applicable to the convolutional neural networks of Area generation, By itself and vehicle, i.e. Area generation network, judges that network integration obtains eurypalynous vehicle detection network；

(6), network structure compression: utilize the matrix decomposition means Area generation part to eurypalynous vehicle detection network Being optimized compression with the structure of regional determination part, the low-dimensional that convolutional layer specifically carries out convolution kernel is decomposed, is coupled complete Layer then uses svd algorithm to reduce number of parameters, the final complexity reducing model, reduces the amount of calculation in test process；

(7), training testing result Optimized model: by all of testing result is grouped, is formed and respectively correspond to truly The combination of testing result, is then calculated the relative deviation average based on weight of each group and real deviation value, utilizes defeated The neural network contact between the two that ingress quantity is 4, hidden layer node quantity is 6, output node quantity is 4 is made For testing result Optimized model；

Described test process has and includes following steps:

(8), background model application: determined the time of image acquisition by the judgement of image acquisition information and image intensities Section, then selects corresponding background model according to scene information, current test image is carried out Analysis on Prospect, determines appearance motion The region of object, as vehicle in present image, the connected domain being obtained Guan Bi by Morphological scale-space occurs that the motion of position detects As a result, external constraint information is provided for Area generation part；

(9), the test of vehicle detection network area generating portion: utilize background model to apply the motion detection result that obtains, Use the specific substantial amounts of boxed area of sliding window schema creation in moving region, then use the region that step (5) obtains Generate network and obtain two classification scores and the position optimization results that the vehicle presence or absence of these boxed area judges, select wherein The higher region of score, as including the candidate region of vehicle, is then used by non-maxima suppression method and reduces the candidate generated The quantity in region；

(10), judging section, vehicle detection network area test: the Area generation network utilizing step (5) to obtain exports The feature of convolutional layer, by the regional determination part of vehicle detection network to all candidate regions including vehicle at Reason, determines whether region is vehicle and obtains concrete vehicle location coordinate, type of vehicle result and headstock angle information；

(11), testing result optimization: for cross one another situation occurs between multiple testing results, use based on nerve The position deviation of network estimates that model processes, and obtains the judged result of optimum, improves the accuracy of detection method；

(12), background model online updating: utilize judged result that the correctness of the background model that step (2) obtains is carried out Judge, then it is optionally updated, improve the accuracy that background is described by model.

The described type of vehicle in step (1) is the headstock size according to vehicle, the vehicle number of axle and overall structure letter The difference of breath, is divided into car, bus, light card, Medium Truck, heavy truck, commercial vehicle, sport utility vehicle by vehicle With eight classifications of engineering truck；The general direction information of described headstock include left side, left front, just before, right before and five, right side Directional information；The described Area generation initial model in step (3), enters training image first by full convolutional network model Row segmentation, obtains the outline position information of all objects occurred in image, dividing as LPO algorithm object in the training process Cut mark, and after Area generation initial model training completes, use part training image as validation database, carry out The adjustment of LPO algorithm relevant parameter, reduces the resource occupation amount required for Area generation process and spends with the time.

In described step (4), training vehicle judges comprising the concrete steps that of network, utilizes existing convolutional neural networks frame Structure, each with the addition of a background classification in type of vehicle exports with two, headstock direction, i.e. arranges vehicle type information and sentence Disconnected output number is that the output number that type of vehicle number eight adds i.e. nine, angle information judges is set to vehicle angles number of types Slender acanthopanax one that is six, will treat as non-car region, its phase with the IOU in the real vehicles region Area generation result less than 0.3 during training The type of vehicle answered and headstock direction are both configured to respective background classification, and the vehicle database set up in step (1) is enterprising The combination learning of row multitask, obtains vehicle and judges network so that network can draw corresponding while obtaining vehicle location Type information, angle information；Wherein, during concrete training, notes following some: LPO algorithm is obtained to owning The Area generation result of training data inputs as the candidate region of network；Use in large-scale internet image data base The model that classification based training obtains carries out vehicle and judges the initialization of network model；By the study of Internet before reduction convolutional layer Speed is the fine setting study of 1/10th implementation models which floor is followed by, and the parameter of recanalization subsequent network layer, makes full use of The preliminary abstracting power that network has been acquired so that the ability to express of model is more powerful.

Described step (5) training Area generation network specifically includes following steps:

A, determine the relevant parameter of the sliding window model based on convolutional layer of use, determine sliding window have 64, 128,256 pixel three class size, tri-kinds of Aspect Ratios of 1:2,1:1,2:1, moving step length on the original image is 16 pixels；

B, training image for input, be mapped to the sliding window of design in convolutional layer, obtain final convolution special Levy, simultaneously by judge with the IOU of actual value to corresponding car and background label, convolutional layer can be designed in this process With full context layer, the characteristic pattern for image is processed, simulate the convolution feature extraction of this sliding window processing mode with Optimize；

C, project training network: use and judge, with vehicle, the convolutional layer structure that network is identical, add a similar and classification With classification layer and the recurrence layer of position optimization, perform judgement and the optimization of Area generation result；

D, utilize training network structure and the vehicle of design judge the convolution feature of network carry out the classification of car and background with The regression training of coordinate optimizing, obtains comprising the Area generation network of vehicle region position；

E, Area generation network and vehicle are judged that network merges, obtain vehicle detection network, then utilize vehicle The Area generation result of detection network, judges that to follow-up vehicle the full articulamentum of network, classification layer are carried out with returning layer segment Adjusting and optimizing so that it is in the output of regional determination partial adaptation Area generation network.

Network structure in described step (6) is compressed and is specifically included following steps:

A, svd algorithm compression is utilized to reduce the quantity of full context layer in network；For full context layer, actual neuron rings For y, it is specifically shown in formula (1):

Y=f (W*a) (1),

Wherein, f represents activation primitive；Assuming that the size of W is u × v, wherein u, v correspondence input respectively and output node Quantity, carries out SVD approximate factorization to it, obtains result W, is specifically shown in formula (2):

W≈U∑_tV^T (2),

Wherein, U, ∑_t, the size of V be respectively u × t, t × t, v × t, and ∑_tFor diagonal matrix, weight comparatively speaking Total quantity is changed to t × (u+v+1), and wherein, t is a Truncation Parameters, is used for controlling compression degree, and i.e. full context layer splits it The quantity of rear intermediate node, u, v are the most corresponding a full context layer input and the quantity of output node；By designing the value of t Less than the minima of u Yu v, following inequality (3) is had to set up:

T × (u+v+1) ＜ u × v (3),

So, decomposed by SVD and reduce overall number of parameters, when actually also reducing the process of full context layer Between；The low-dimensional that convolutional layer carries out convolution kernel is decomposed

B, utilizing Marx. the convolutional layer structure of network is compressed by the algorithm in Derby: assuming that the volume of current layer It is N number of that long-pending core number is that N, i.e. output node have, and the size of convolution kernel is d*d, and the characteristic pattern size obtained is H ' × W ', and input is logical Number of channels is C, then the amount of calculation of this convolutional layer is O (CNd²h′W′)；By convolution kernel { W_i∈R^d×d, i=[1...N] } represent, Can consider to be split as the synthesis result of two one-dimensional convolution kernels: the one-dimensional convolution kernel of the first kind uses { v_k∈R^d×1×C；K= [1...K] } represent, the characteristic pattern obtained is V, V (u, v) ∈ R^K, the one-dimensional convolution kernel of Equations of The Second Kind uses { h_n∈R^1×d×K；N= [1...N] } represent, then can obtainOverall amount of calculation is O (K (C+N) dH ' W′)；Two one-dimensional convolution kernels minimize use conjugate gradient decent by the reconstructed error of calculating convolution kernel and ask for, tool Body optimization aim；As followsFinally, as K (C+N) ＜ NCd, permissible Ensure K (C+N) dH ' W ' ＜ CNd²H′W^′, it practice, the convolutional layer realized optimizes multiple is approximately equal to the size of convolution kernel.

The described training testing result Optimized model in step (7) specifically includes herein below:

A, first carrying out the division of testing result, determine which testing result judges is same object；The side processed Method is the result every time selecting top score, then divides by calculating he IOU with residue detection square frame: assuming that N number of Testing result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2 ... N] } represent, selection has the most every time The square frame of big score is designated as b_k, calculate it and remain the IOU value of square frame and classify: showing that when IOU is more than 0.5 they predict same One object, forms a class square frame set B by these square frames_k1={ b_i|IoU(b_i, b_k) >=0.5, b_i∈ B}, side therein Frame is not participated in and is again divided, b_kAs one of them element；When IOU is less than 0.5 more than 0.3, it is believed that they share Subregion, obtains two class square frame set B_k2={ b_i| 0.3 ＜ IoU (b_i, b_k) ＜ 0.5, b_i∈ B}, square frame therein is joined Add and again divide；Repeat whole process until all of testing result is finally divided into K group, obtain with physical quantities with corresponding A class square frame and two class square frames be designated as { (B_k1, B_k2), k ∈ [1 2 ... K] }；

B, for an all of class square frame B_k1, calculate mean blockAnd all square frames is with mean block ground quaternary by mistake Difference groupAnd record reserved portion, then count Calculate the real difference value of the true square frame of object and mean block

C, carry out the standardization of error of coordinate according to all scores, obtain average difference values based on weightThen average difference values and real difference value are converted to based on mean block Relatively being expressed as of sizeWithUtilize one quaternary relation of neural networkRealize both Between the foundation of implication relation, as the Part I of testing result Optimized model；

D, data for two class labels, use above-mentioned same coordinate averageCarry out identical process, obtain phase AnswerThen set up similar another one quaternary relationAs testing result Optimized model second Part.

Background model in described step (8) is applied and is specifically included herein below:

A, by the data item information in data base, obtain the title of pending image, storage position and adopt accordingly Collection information；

B, according to the information of collection, determine the numbering of image acquisition terminal of correspondence, determine that use is white by overall shading value It or night, select suitable gauss hybrid models；

C, contrast according to the overall shading value of image Yu model average shading, carry out the adjustment of image intensities, will figure As the excursion of shading value is formed corresponding with the excursion of model, reduce the difference shadow for testing result of global illumination Ring；

D, use model carry out foreground detection to present image, then carry out Morphological scale-space, obtain motor region substantially Block, and it is divided into multiple connected domain；

E, the image coordinate of acquisition moving region boundary rectangle, determine its proportion in the picture, it is judged that arithmetic result Order of accuarcy, when the region more than 80% or less than 10% region be all judged as prospect time, illustrate motion detect Result is inaccurate, it is impossible to as the reference information of Area generation network test.

Generating portion test in vehicle detection network area in described step (9), uses the motion of Area generation network integration Testing result is likely comprised the candidate region of vehicle: first apply the result obtained according to background model, and obtaining may Moving vehicle region and the position of boundary rectangle frame, if motion detection result is inaccurate, i.e. moving region be less than view picture figure Picture 10% or more than entire image 70%, then entire image is set to a rectangle frame；Then with training process class Seemingly, in the inside of these rectangle frames with 16 pixels as step-length, the size having 64,128,256 pixels and 1:2,1:1,2:1 are set The sliding window of Aspect Ratio, image-region is mapped on convolutional layer conv5_3 as receptive field and obtains corresponding convolution Feature, carries out follow-up convolutional layer PR_conv, the process acquisition classification layer RP_cls_score of full context layer RP_FC, returns layer The output of RP_bbox_pred, obtains window area vehicle and the judged result of probability and the optimum results of coordinate occurs, finally Carry out non-maxima suppression and process the quantity reducing candidate region further, form the Area generation input of detection neutral net.

Described step (11) testing result optimization specifically includes herein below:

A and training similar process, first carry out the division of testing result, different testing results be combined, obtain Final testing result quantity.Assuming that N number of testing result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2...N] } Representing all of testing result, the method for actual treatment is to select the testing result of top score every time and calculate he and residue The IOU of detection square frame, before note, the square frame of maximum score is b_k: and take IOU part one class square frame set of composition more than 0.5 B_k1, b_kAs one of them element；Take IOU part to the two class square frame set B more than 0.3 less than 0.5_k2, wherein Square frame can participate in and again divide.Repeat whole process until all of testing result is finally divided into K group, obtain and thing Body quantity is designated as { (B with a corresponding class square frame and two class square frames_k1, B_k2), k ∈ [1 2...K] }；

B, for an all of class square frame B_k1, calculate mean blockAnd all square frames is with mean block ground quaternary by mistake Difference group { Δ B_k1j=(Δ x_k1jlt, Δ y_k1jlt, Δ x_k1jrb, Δ y_k1jrb), b_k1j∈B_k1, record is to reserved portion；

C, carry out the standardization of error of coordinate according to all scores, obtain average difference values based on weightThen average difference values is converted into relative table based on mean block size It is shown asThe Part I using testing result to optimize network carries out processing the grid deviation obtaining prediction

D, data for two class labels, use above-mentioned same coordinate averageCarry out identical process, be similar to Other grid deviation

E: carry out the merging of two parts deviation, uses the weights of 0.8 and 0.2 to carry out the summation of difference value respectivelyIt is then based on the deviation length being all worth to reality of a class square frame, then Carry out the output of true square frame position.

Described step (12) background model online updating specifically includes herein below:

A, according to vehicle detection as a result, it is possible to obtain the region that vehicle occurs in image, correspond to gauss hybrid models Testing result in, judged the accuracy of motion detection model by the IOU of vehicle region Yu moving region, when all motor region Territory thinks when all there is the IOU in vehicle and two kinds of regions more than 0.5 that model inspection result is more accurate, is now made without The renewal of background model；

B, when testing result occurs the region not comprising vehicle, it is believed that background model occurs in that erroneous judgement, this region Background describe bigger with actual variance, it is believed that the background model of corresponding region will should be judged as the district of prospect Territory judges for background, uses model to detect original sequence, obtains similar for current region testing result Parts of images forms new sequence as data source, re-establishes the background model of current corresponding region；

C, when region IOU less than 0.5 time, illustrate that the judgement for moving region is the most accurate, in fact it could happen that reason Predominantly the impact of shade, needs to be updated the parameter of model with non-car moving region, reduces model and may background be sentenced Break the probability for prospect；

D, when background model application section separate existing moving region less than image size 10% time, if vehicle detection is same Coming to nothing, be not updated model, otherwise explanation regional model is inaccurate, needs to utilize present image to detection model In corresponding to vehicle occur region be updated；

E, when background model application section separate existing moving region more than image size 70% time, illustrate present image and Background model difference is relatively big, and background model is little for the effect of present image, needs to utilize present image to all Fei Che districts The background in territory is updated.

Advantages of the present invention:

The present invention can process the position of vehicle, type and angle information simultaneously, it is provided that finer vehicle detection knot Really, take full advantage of the powerful abstracting power of neutral net and realize the description for target, obtained judging knot more accurately Really, it is simultaneously based on these and describes the generation achieving candidate region, improve the general degree of neutral net, decrease whole place The amount of calculation of reason flow process.It is simultaneously achieved the combination of moving object detection and the vehicle detection in still image, uses motion While information guiding vehicle detection, raising treatment effeciency, the result also by vehicle detection is carried out for motion detection model Revise, improve the accuracy of model.Additionally, or process replacement routine with Accumulated deviation prediction based on neutral net Non-maxima suppression, takes full advantage of all output results of detection network.

Accompanying drawing explanation

Fig. 1 is the flow chart of the training process of the present invention.

Fig. 2 is the flow chart of test process of the present invention.

Fig. 3 is the structural representation that vehicle of the present invention judges network.Wherein, dashed rectangle is used to be shown to be known knot Structure, VGG16conv1_1 ReLU5_3 is the structure of VGG16 network conv1_1 to ReLU5_3 part, RoI pooling layer, Internet in FC6 layer, FC7 layer are corresponding FCN algorithm, and last cls_score, ori_score, bbox_pred are respectively Predict the outcome for type of vehicle classification layer, headstock angle classification results, vehicle location.

Fig. 4 is the structural representation of Area generation network of the present invention.Wherein, dashed rectangle corresponding VGG16 network The structure of conv1_1 to ReLU5_3 part, RP-conv, RP-FC are respectively a convolutional layer and full context layer, and RP_cls_ Score represents that the classification layer differentiating vehicle and background, RP_bbox_pred are to obtain the recurrence layer that coordinate predicts the outcome.

Fig. 5 is the structural representation of the present invention eurypalynous vehicle detection network.Wherein, VGG16conv1_1 ReLU5_3 Partly, RoI pooling with Fig. 3 identical, PR-conv, RP-FC, RP_cls_score are corresponding with RP_bbox_pred Fig. 3's As a result, RP_data layer then carries out Screening Treatment to Area generation result, remaining FC6 ' layer, FC7 ' layer, cls_score ' layer, Ori_score ' layer, bbox_pred ' Rotating fields are identical with Fig. 3, but are the result after adjusting for the parameter in network.

Detailed description of the invention

Training process has and includes following steps:

(1), information of vehicles mark: all original images that collection different images acquisition terminal obtains process, mark Publish picture as in the location coordinate information of vehicle, and add the simple judged result of type of vehicle and the general direction information of headstock, And all images and above-mentioned all markup informations are stored in data base；Wherein, type of vehicle is that the headstock according to vehicle is big Little, the vehicle number of axle and the difference of overall structure information, be divided into vehicle car, bus, light card, Medium Truck, heavy block Car, commercial vehicle, sport utility vehicle and eight classifications of engineering truck, wherein, light card refers mainly to pick up and has small-sized Passenger-cargo dual-purpose minibus, it is owing to its axletree number is not more than two that Medium Truck is different from heavy truck；The general direction letter of headstock Breath include left side, left front, just before, right before and five, right side directional information, its angular range substantially have respectively-90 ° to- 60 ° ,-60 ° to-15 ° ,-15 ° to 15 °, 15 ° to 60 °, 60 ° to 90 °；

(3), training Area generation initial model: use degree of deep learning algorithm that all training images are carried out image and divide Cut, obtain the partition data of all objects occurred in image, utilize the partition data the obtained LPO to traditional Area generation Algorithm carries out the adjustment in step (1) on data base, obtains Area generation initial model；Specifically first by full convolution net Training image is split by network model, obtains the outline position information of all objects occurred in image, exists as LPO algorithm The segmentation mark of object during training, and after Area generation initial model training completes, use part training image to make For validation database, carry out the adjustment of LPO algorithm relevant parameter, reduce the resource occupation amount required for Area generation process and time Between spend；

(4), training vehicle judges network: utilize the result of convolutional neural networks and Area generation initial model, with vehicle Position coordinates, vehicle type information and headstock angle information, as the output of network, carry out vehicle and judge the multitask connection of network Close study, obtain eurypalynous vehicle and judge network；Comprise the concrete steps that: set up structure detection neutral net as shown in Figure 2, its Middle dashed rectangle part represents and utilizes existing convolutional neural networks framework, each in type of vehicle exports with two, headstock direction From with the addition of a background classification, i.e. arrange the output number that vehicle type information judges be type of vehicle number eight add one i.e. nine, Angle information judge output number be set to vehicle angles number of types slender acanthopanax one that is six, during training by with real vehicles region The IOU Area generation result less than 0.3 is as non-car region, and its corresponding type of vehicle is both configured to respective with headstock direction Background classification, carries out the combination learning of multitask in the vehicle database that step (1) is set up, and obtains vehicle and judges network, Allow the network to draw corresponding type information, angle information while obtaining vehicle location；Wherein, specifically training The most what time Cheng Zhong, note: the Area generation result to all training datas that the LPO algorithm that just traditional area generates obtains Candidate region as network inputs；The model using the classification based training in large-scale internet image data base to obtain is carried out Vehicle judges the initialization of network model；By the learning rate of Internet before reduction convolutional layer ReLU5_3 for which floor is followed by 1/10th implementation models fine setting study, the parameter of recanalization subsequent network layer, make full use of what network had been acquired Preliminary abstracting power so that the ability to express of model is more powerful.

(5), training Area generation network: the convolutional layer conv1_1-ReLU5_3 layer of network is as spy to utilize vehicle to judge Levy, carry out based on sliding window having car to judge and the training of coordinate optimizing network without car two classification, obtain being applicable to region raw The convolutional neural networks become, i.e. Area generation network, by its with vehicle judgement network integration obtain as shown in Figure 4 eurypalynous Vehicle detection network；RP_conv layer carries out the quick process of convolution feature in the diagram, directly obtains corresponding to 64*64 size sense By wild convolution character representation, RP_FC layer then carries out the efficient combination of each feature passage；

Training Area generation network specifically includes following steps:

Network structure is compressed and is specifically included following steps:

Y=f (W*a) (1),

W≈U∑_tV^T (2),

Wherein, U, ∑_t, the size of V be respectively u × t, t × t, v × t, and Σ_tFor diagonal matrix, weight comparatively speaking Total quantity is changed to t × (u+v+1), and wherein, t is a Truncation Parameters, is used for controlling compression degree, and i.e. full context layer splits it The quantity of rear intermediate node, u, v are the most corresponding a full context layer input and the quantity of output node；By designing the value of t Less than the minima of u Yu v, following inequality (3) is had to set up:

T × (u+v+1) ＜ u × v (3),

B, utilize Jaderberg (name, Marx. in Derby) algorithm the convolutional layer structure of network is pressed Contracting: assuming that the convolution kernel number of current layer to be that N, i.e. output node have N number of, the size of convolution kernel is d*d, and the characteristic pattern obtained is big Little for H ' × W ', input channel number is C, then the amount of calculation of this convolutional layer is O (CNd²H′W′)；By convolution kernel { W_i∈R^d ^×d, i=[1...N] } represent, it may be considered that it is split as the synthesis result of two one-dimensional convolution kernels: the one-dimensional convolution of the first kind Core uses { v_k∈R^d×1×C；K=[1...K] } represent, the characteristic pattern obtained is V, V (u, v) ∈ R^K, the one-dimensional convolution kernel of Equations of The Second Kind Use { h_n∈R^1×d×K；N=[1...N] } represent, then can obtainOverall calculating Amount is O (K (C+N) dH ' W ')；Two one-dimensional convolution kernels minimize under use conjugate gradient by calculating the reconstructed error of convolution kernel Fall method is asked for, concrete optimization aim；As followsFinally, as K (C+ N) during ＜ NCd, it is ensured that K (C+N) dH ' W ' ＜ CNd²H′W^′, it practice, the convolutional layer realized optimizes multiple is approximately equal to volume The size of long-pending core；

Training testing result Optimized model specifically includes herein below:

A, first carrying out the division of testing result, determine which testing result judges is same object；The side processed Method is the result every time selecting top score, then divides by calculating he IOU with residue detection square frame: assuming that N number of Testing result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2...N] } represent, selection has the most every time The square frame of big score is designated as b_k, calculate it and remain the IOU value of square frame and classify: showing that when IOU is more than 0.5 they predict same One object, forms a class square frame set B by these square frames_k1={ b_i|IoU(b_i, b_k) >=0.5, b_i∈ B}, side therein Frame is not participated in and is again divided, b_kAs one of them element；When IOU is less than 0.5 more than 0.3, it is believed that they share Subregion, obtains two class square frame set B_k2={ b_i| 0.3 ＜ IoU (b_i, b_k) ＜ 0.5, b_i∈ B}, square frame therein is joined Add and again divide；Repeat whole process until all of testing result is finally divided into K group, obtain with physical quantities with corresponding A class square frame and two class square frames be designated as { (B_k1, B_k2), k ∈ [1 2 ... K] }；

D, data for two class labels, use above-mentioned same coordinate averageCarry out identical process, obtain corresponding 'sThen set up similar another one quaternary relationAs testing result Optimized model second Point；

Test process has and includes following steps:

(8), background model application: determined the time of image acquisition by the judgement of image acquisition information and image intensities Section, then selects corresponding background model according to scene information, current test image is carried out Analysis on Prospect, determines appearance motion The region of object, as vehicle in present image, the connected domain being obtained Guan Bi by Morphological scale-space occurs that the detection of position is tied Really, external constraint information is provided for Area generation part；

Background model is applied and is specifically included herein below:

E, the image coordinate of acquisition moving region boundary rectangle, determine its proportion in the picture, it is judged that arithmetic result Order of accuarcy, when the region more than 80% or less than 10% region be all judged as prospect time, illustrate motion detect Result is inaccurate, it is impossible to as the reference information of Area generation network test；

(9), vehicle detection network area generating portion test: use Area generation network integration motion detection result to obtain Likely comprise the candidate region of vehicle: first apply the result obtained according to background model, obtain possible moving vehicle district Territory and the position of boundary rectangle frame thereof, if motion detection result is inaccurate, i.e. moving region less than entire image 10% or More than the 70% of entire image, then entire image is set to a rectangle frame；Then similar, at these rectangles with training process The inside of frame, with 16 pixels as step-length, arranges the size having 64,128, the 256 pixels Aspect Ratio with 1:2,1:1,2:1 Sliding window, is mapped to image-region on convolutional layer conv5_3 as receptive field and obtains corresponding convolution feature, carry out follow-up Convolutional layer PR_conv, the process of full context layer RP_FC obtain classification layer RP_cls_score, return layer RP_bbox_pred's Output, obtains window area vehicle and the judged result of probability and the optimum results of coordinate occurs, finally carry out non-maximum and press down System processes the quantity reducing candidate region further, forms the Area generation input of detection neutral net；

(10), judging section, vehicle detection network area test: the Area generation network utilizing step (5) to obtain exports The feature of convolutional layer conv5_3, by the regional determination part of vehicle detection network to all candidate regions including vehicle FC6 ' layer, FC7 ' layer, cls_score ', ori_score ' process with bbox_pred ', determine region be whether vehicle also Obtain concrete vehicle location coordinate, type of vehicle result and headstock angle information；

Testing result optimization specifically includes herein below:

A and training similar process, first carry out the division of testing result, different testing results be combined, obtain Final testing result quantity.Assuming that N number of testing result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2...N] } Representing all of testing result, the method for actual treatment is to select the testing result of top score every time and calculate he and residue The IOU of detection square frame, before note, the square frame of maximum score is b_k: and take IOU part one class square frame set of composition more than 0.5 B_k1, b_kAs one of them element；Take IOU part to the two class square frame set B more than 0.3 less than 0.5_k2, wherein Square frame can participate in and again divide.Repeat whole process until all of testing result is finally divided into K group, obtain and thing Body quantity is designated as { (B with a corresponding class square frame and two class square frames_k1, B_k2), k ∈ [1 2 ... K] }；

D, data for two class labels, use above-mentioned same coordinate averageCarry out identical process, obtain class As other grid deviation

E: carry out the merging of two parts deviation, uses the weights of 0.8 and 0.2 to carry out the summation of difference value respectivelyIt is then based on the deviation length being all worth to reality of a class square frame, then Carry out the output of true square frame position；

(12), background model online updating: utilize judged result that the correctness of the background model that step (2) obtains is carried out Judge, then it is optionally updated, improve the accuracy that background is described by model；

Background model online updating specifically includes herein below:

Claims

1. a traffic image polymorphic type vehicle checking method based on degree of depth study, it is characterised in that: include training process And test process:

Described training process has and includes following steps:

(1), information of vehicles mark: all original images that collection different images acquisition terminal obtains process, and mark is published picture The location coordinate information of vehicle in Xiang, and add the simple judged result of type of vehicle and the general direction information of headstock, and will All images and above-mentioned all markup informations are stored in data base；

(2), Background Modeling: for different image acquisition terminals, collect training data respectively and formed between consecutive frame non- Continually varying image sequence, then according to the difference of image entirety bright-dark degree, is divided into two kinds of feelings of daytime and night Condition, uses gauss hybrid models to set up background model respectively；

(3), training Area generation initial model: use degree of deep learning algorithm that all training images carry out the segmentation of image, obtain The partition data of all objects occurred in image, utilizes the partition data the obtained LPO algorithm to traditional Area generation Carry out the adjustment on data base in step (1), obtain Area generation initial model；

(4), training vehicle judges network: utilize the result of convolutional neural networks and Area generation initial model, with vehicle location Coordinate, vehicle type information and headstock angle information, as the output of network, carry out vehicle and judge that the multitask of network combines Practise, obtain eurypalynous vehicle and judge network；

(5), training Area generation network: the convolutional layer of network, as feature, carries out based on sliding window to utilize vehicle to judge There is car to judge the training with coordinate optimizing network without car two classification, obtain being applicable to the convolutional neural networks of Area generation, i.e. district Territory generates network, and itself and vehicle being judged, network integration obtains eurypalynous vehicle detection network；

(6), network structure compression: utilize matrix decomposition means to the Area generation part of eurypalynous vehicle detection network and district The structure of judging section, territory is optimized compression, and the low-dimensional that convolutional layer specifically carries out convolution kernel is decomposed, to full context layer then Use svd algorithm to reduce number of parameters, the final complexity reducing model, reduce the amount of calculation in test process；

(7), training testing result Optimized model: by all of testing result is grouped, is formed and respectively correspond to truly detect The combination of result, is then calculated the relative deviation average based on weight of each group and real deviation value, utilizes input joint Point quantity is 4, hidden layer node quantity is 6, output node quantity is that the neural network contact between the two of 4 is as inspection Survey result optimizing model；

Described test process has and includes following steps:

(8), background model application: determined the time period of image acquisition by the judgement of image acquisition information and image intensities, Then select corresponding background model according to scene information, current test image is carried out Analysis on Prospect, determines that moving object occurs The region of body, obtains the connected domain of Guan Bi by Morphological scale-space and occurs the motion detection knot of position as vehicle in present image Really, external constraint information is provided for Area generation part；

(9), vehicle detection network area generating portion test: utilize background model to apply the motion detection result obtained, in fortune Dynamic region uses the specific substantial amounts of boxed area of sliding window schema creation, then uses the Area generation that step (5) obtains Network obtains two classification scores and the position optimization results that the vehicle presence or absence of these boxed area judges, selects wherein score Higher region, as including the candidate region of vehicle, is then used by non-maxima suppression method and reduces the candidate region generated Quantity；

(10), judging section, vehicle detection network area test: the convolution that the Area generation network utilizing step (5) to obtain exports All candidate regions including vehicle are processed, really by the feature of layer by the regional determination part of vehicle detection network Determine whether region is vehicle and obtains concrete vehicle location coordinate, type of vehicle result and headstock angle information；

(11), testing result optimization: for cross one another situation occurs between multiple testing results, use based on neutral net Position deviation estimate model process, obtain optimum judged result, improve detection method accuracy；

(12), background model online updating: utilize judged result that the correctness of the background model that step (2) obtains is sentenced Disconnected, then it is optionally updated, improve the accuracy that background is described by model.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: the described type of vehicle in step (1) is the headstock size according to vehicle, the vehicle number of axle and overall structure information Difference, is divided into car, bus, light card, Medium Truck, heavy truck, commercial vehicle, sport utility vehicle and work by vehicle Eight classifications of journey car；The general direction information of described headstock include left side, left front, just before, right before and direction, five, right side Information；The described Area generation initial model in step (3), is carried out training image point first by full convolutional network model Cut, obtain the outline position information of all objects occurred in image, as the segmentation mark of LPO algorithm object in the training process Note, and after Area generation initial model training completes, use part training image as validation database, carry out LPO calculation The adjustment of method relevant parameter, reduces the resource occupation amount required for Area generation process and spends with the time.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: in described step (4), training vehicle judges comprising the concrete steps that of network, utilizes existing convolutional neural networks framework, In type of vehicle exports with two, headstock direction, each with the addition of a background classification, i.e. arrange what vehicle type information judged Output number is that the output number that type of vehicle number eight adds i.e. nine, angle information judges is set to vehicle angles number of types slender acanthopanax One that is six, will treat as non-car region with the IOU in the real vehicles region Area generation result less than 0.3 during training, and it is corresponding Type of vehicle and headstock direction are both configured to respective background classification, carry out many in the vehicle database that step (1) is set up The combination learning of task, obtains vehicle and judges network so that network can draw corresponding class while obtaining vehicle location Type information, angle information；Wherein, during concrete training, notes following the most what time: LPO algorithm is obtained to all training The Area generation result of data inputs as the candidate region of network；Use the classification in large-scale internet image data base The model that training obtains carries out vehicle and judges the initialization of network model；By the learning rate of Internet before reduction convolutional layer Learn for being followed by the fine setting of which floor 1/10th implementation models, the parameter of recanalization subsequent network layer, make full use of network The preliminary abstracting power acquired so that the ability to express of model is more powerful.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: described step (5) training Area generation network specifically includes following steps:

A, determine the relevant parameter of the sliding window model based on convolutional layer of use, determine that sliding window has 64,128,256 Pixel three class size, tri-kinds of Aspect Ratios of 1:2,1:1,2:1, moving step length on the original image is 16 pixels；

B, for input training image, the sliding window of design is mapped in convolutional layer, obtains final convolution feature, with Time by judge with the IOU of actual value to corresponding car and background label, convolutional layer can be designed in this process with complete Characteristic pattern for image is processed by context layer, simulates the convolution feature extraction of this sliding window processing mode with excellent Change；

C, project training network: use and judge, with vehicle, the convolutional layer structure that network is identical, add a similar and classification and position Put classification layer and the recurrence layer of optimization, perform judgement and the optimization of Area generation result；

D, the training network structure utilizing design and vehicle judge that the convolution feature of network carries out car and the classification of background and coordinate The regression training optimized, obtains comprising the Area generation network of vehicle region position；

E, Area generation network and vehicle are judged that network merges, obtain vehicle detection network, then utilize vehicle detection The Area generation result of network, judges that to follow-up vehicle the full articulamentum of network, classification layer are adjusted with returning layer segment Optimize so that it is in the output of regional determination partial adaptation Area generation network.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: the network structure in described step (6) is compressed and specifically included following steps:

A, svd algorithm compression is utilized to reduce the quantity of full context layer in network；For full context layer, it is y that actual neuron rings, It is specifically shown in formula (1):

Y=f (W*a) (1),

Wherein, f represents activation primitive；Assuming that the size of W is u × v, wherein u, v correspondence input and quantity of output node respectively, It is carried out SVD approximate factorization, obtains result W, be specifically shown in formula (2):

W≈U∑_tV^T (2),

Wherein, U, ∑_t, the size of V be respectively u × t, t × t, v × t, and ∑_tFor diagonal matrix, the comparatively speaking sum of weight Quantitative change turns to t × (u+v+1), and wherein, t is a Truncation Parameters, is used for controlling compression degree, i.e. full context layer split after in The quantity of intermediate node, u, v are the most corresponding a full context layer input and the quantity of output node；By the value of design t less than u With the minima of v, following inequality (3) is had to set up:

T × (u+v+1) ＜ u × v (3),

So, decomposed by SVD and reduce overall number of parameters, actually also reduce the process time of full context layer；Right Convolutional layer carries out the low-dimensional of convolution kernel and decomposes

B, utilizing Marx. the convolutional layer structure of network is compressed by the algorithm in Derby: assuming that the convolution kernel of current layer It is N number of that number is that N, i.e. output node have, and the size of convolution kernel is d*d, and the characteristic pattern size obtained is H ' × W ', input channel number For C, then the amount of calculation of this convolutional layer is O (CNd²H′W′)；By convolution kernel { W_i∈R^d×d, i=[1...N] } represent, permissible Consider to be split as the synthesis result of two one-dimensional convolution kernels: the one-dimensional convolution kernel of the first kind uses { v_k∈R^d×1×C；K= [1...K] } represent, the characteristic pattern obtained is V, V (u, v) ∈ R^K, the one-dimensional convolution kernel of Equations of The Second Kind uses { h_n∈R^1×d×K；N= [1...N] } represent, then can obtainOverall amount of calculation is O (K (C+N) dH ' W′)；Two one-dimensional convolution kernels minimize use conjugate gradient decent by the reconstructed error of calculating convolution kernel and ask for, tool Body optimization aim；As followsFinally, as K (C+N) ＜ NCd, permissible Ensure K (C+N) dH ' W ' ' ＜ CNd²H ' W ', it practice, the convolutional layer realized optimizes multiple is approximately equal to the size of convolution kernel.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: the described training testing result Optimized model in step (7) specifically includes herein below:

A, first carrying out the division of testing result, determine which testing result judges is same object；The method processed is Selecting the result of top score, the IOU then detecting square frame by calculating him with residue divides every time: assuming that N number of detection Result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2...N] } represent, select that there is the most maximum obtaining every time The square frame divided is designated as b_k, calculate the IOU value of it and residue square frame and classify: showing that they predict when IOU is more than 0.5 same Object, forms a class square frame set B by these square frames_k1={ b_i|IoU(b_i, b_k) >=0.5, b_i∈ B}, square frame therein is not Participate in and again divide, b_kAs one of them element；When IOU is less than 0.5 more than 0.3, it is believed that they have shared part Region, obtains two class square frame set B_k2={ b_i| 0.3 ＜ IoU (b_i, b_k) ＜ 0.5, b_i∈ B}, square frame therein is participated in again Secondary division；Repeat whole process until all of testing result is finally divided into K group, obtain and physical quantities and corresponding Class square frame and two class square frames are designated as { (B_k1, B_k2), k ∈ [1 2...K] }；

B, for an all of class square frame B_k1, calculate mean blockAnd all square frames and mean block ground quaternary margin of error GroupAnd record reserved portion, then calculate thing The true square frame of body and the real difference value of mean block

D, data for two class labels, use above-mentioned same coordinate averageCarry out identical process, obtain correspondingThen set up similar another one quaternary relationAs testing result Optimized model second Point.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: the background model in described step (8) is applied and specifically included herein below:

A, by the data item information in data base, obtain the title of pending image, storage position and gather letter accordingly Breath；

B, according to the information of collection, determine the numbering of image acquisition terminal of correspondence, determine that use daytime is also by overall shading value It is night, selects suitable gauss hybrid models；

C, contrast according to the overall shading value of image Yu model average shading, carry out the adjustment of image intensities, and image is bright The excursion of darkness is formed corresponding with the excursion of model, reduces the difference impact for testing result of global illumination；

D, use model carry out foreground detection to present image, then carry out Morphological scale-space, obtain motion block substantially, and It is divided into multiple connected domain；

E, the image coordinate of acquisition moving region boundary rectangle, determine its proportion in the picture, it is judged that the standard of arithmetic result Really degree, when the region more than 80% or the region less than 10% are all judged as prospect, illustrates the result of motion detection Inaccurate, it is impossible to as the reference information of Area generation network test.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: generating portion test in vehicle detection network area in described step (9), uses the motion detection of Area generation network integration Result is likely comprised the candidate region of vehicle: first applies the result obtained according to background model, obtains possible fortune Dynamic vehicle region and the position of boundary rectangle frame thereof, if motion detection result is inaccurate, i.e. moving region is less than entire image 10% or more than entire image 70%, then entire image is set to a rectangle frame；Then similar with training process, The inside of these rectangle frames, with 16 pixels as step-length, arranges and has the size of 64,128,256 pixels and the length of 1:2,1:1,2:1 The sliding window of wide ratio, is mapped to image-region on convolutional layer conv5_3 as receptive field and obtains corresponding convolution feature, Carry out follow-up convolutional layer PR_conv, the process of full context layer RP_FC obtains classification layer RP_cls_score, return layer RP_ The output of bbox_pred, obtains window area vehicle and the judged result of probability and the optimum results of coordinate occurs, finally carry out Non-maxima suppression processes the quantity reducing candidate region further, forms the Area generation input of detection neutral net.

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: described step (11) testing result optimization specifically includes herein below:

A and training similar process, first carry out the division of testing result, different testing results be combined, and obtains final Testing result quantity.Assuming that N number of testing result uses B={b_i=(x_ilt, y_ilt, x_irb, y_irb), i ∈ [1 2...N] } carry out table Showing all of testing result, the method for actual treatment is to select the testing result of top score every time and calculate he and residue detection The IOU of square frame, before note, the square frame of maximum score is b_k: and take IOU part one class square frame set B of composition more than 0.5_k1, b_k As one of them element；Take IOU part to the two class square frame set B more than 0.3 less than 0.5_k2, square frame therein Can participate in and again divide.Repeat whole process until all of testing result is finally divided into K group, obtain and physical quantities It is designated as { (B with a corresponding class square frame and two class square frames_k1, B_k2), k ∈ [1 2...K] }；

B, for an all of class square frame B_k1, calculate mean blockAnd all square frames and mean block ground quaternary margin of error Group { Δ B_k1j=(Δ x_k1jlt, Δ y_k1jlt, Δ x_k1jrb, Δ y_k1jrb), b_k1j∈B_k1, record is to reserved portion；

D, data for two class labels, use above-mentioned same coordinate averageCarrying out identical process, obtain being similar to is another Outer grid deviation

A kind of traffic image polymorphic type vehicle checking method based on degree of depth study the most according to claim 1, its feature It is: described step (12) background model online updating specifically includes herein below:

A, according to vehicle detection as a result, it is possible to obtain the region that vehicle occurs in image, correspond to the inspection of gauss hybrid models Survey in result, judged the accuracy of motion detection model by the IOU of vehicle region Yu moving region, when all moving regions all Think when the IOU in appearance vehicle and two kinds of regions is more than 0.5 that model inspection result is more accurate, be now made without background The renewal of model；

B, when testing result occurs the region not comprising vehicle, it is believed that background model occurs in that erroneous judgement, the back of the body in this region Scene describing is bigger with actual variance, it is believed that sentenced in the region that should be judged as prospect in the background model of corresponding region Disconnected for background, use model that original sequence is detected, obtain the part similar for current region testing result Image forms new sequence as data source, re-establishes the background model of current corresponding region；

C, when region IOU less than 0.5 time, illustrate that the judgement for moving region is the most accurate, in fact it could happen that reason main For the impact of shade, needing to be updated the parameter of model with non-car moving region, reducing model may be judged as background The probability of prospect；

D, when background model application section separate existing moving region less than image size 10% time, if vehicle detection also without As a result, not being updated model, otherwise explanation regional model is inaccurate, needs to utilize present image to right in detection model Should be updated in the region that vehicle occurs；

E, when background model application section separate existing moving region more than image size 70% time, present image and background are described Model difference is relatively big, and background model is little for the effect of present image, needs to utilize present image to all non-car regions Background is updated.