CN106295605A - Traffic lights detection and recognition methods - Google Patents

Abstract

Embodiment of the disclosure about a kind of traffic lights detection and recognition methods, multiple channel characteristics including the sample calculating training set, the multiple channel characteristics of pondization is for training detector, and the feature of described feature pyramid different scale is detected to obtain target area by the detector that the feature pyramid of calculating input image and use are trained by sliding window.

Description

Traffic lights detection and recognition methods

Technical field

The disclosure belongs to computer vision and object detection technical field, especially relates to a kind of traffic lights detection and identifies Method.

Background technology

The most automatically detect that the position of the traffic lights in front and state are that senior auxiliary is driven and nobody drives Sail middle a kind of important technology.Under normal circumstances due to complicated traffic scene, the illumination of acute variation, and the resolution of camera Rate so that the detection of traffic lights becomes relatively difficult.A three major types is had currently for the detection method of traffic lights:

The first kind is method based on image procossing.The operations such as the method passes through Threshold segmentation, morphological transformation are to image Process, obtain object area interested in picture, then by specific priori, such as the connectivity of region, length and width Ratio, shape, relative position etc., process these regions, screen layer by layer, finally obtain is exactly the region at traffic lights place, passes through Setpoint color threshold value or utilize special color space to judge the color of traffic lights.R.de Charette et al. is at document “R.de Charette and F.Nashashibi,Real time visual traffic lights recognition based on Spot Light Detection and adaptive traffic lights templates,IEEE Intelligent Vehicles Symposium, pp.358-363,2009 " propose a kind of method that traffic lights detect, should Method is converted by morphological image, Threshold segmentation, obtains object candidate region.Candidate regions is screened followed by outward appearance ratio Territory, obtains the state of traffic lights finally by template matching.Its weak point is to adapt to changeable scene, and threshold value is excessively Sensitivity, inadequate robust.

Equations of The Second Kind is method based on Orientation on map, is measured by GPS accurately and the traffic lights information of artificial mark, To more accurately traffic lights priori.Close to traffic lights when, geometric transformation is utilized to obtain the candidate region of object, then Classify in candidate region.As V.John et al. document " V.John, K.Yoneda, Z.Liu, and S.Mita, Saliency Map Generation by the Convolutional Neural Network for Real-Time Traffic Light Detection Using Template Matching.IEEE Trans.Computational Imaging, vol.1, no.3, pp.159-173, Sept.2015 " the middle significance map proposing to be generated off-line by GPS, profit By in-vehicle camera parameter close to traffic lights when, obtain, by trigonometry, the region that traffic lights occur, then use convolution Neutral net and template matching detect traffic lights classification.Its deficiency is excessively to rely on sensor device, under same effect, High cost.

3rd class is method based on machine learning, as Shi et al. document " Z.Shi, Z.Zhou, and C.Zhang, Real-Time Traffic Light Detection With Adaptive Background Suppression Filter.IEEE Trans.Intelligent Transportation Systems,vol.17,no.3,pp.690-700, Oct.2015 " in the method that proposes, by the sample in training set is learnt, it is possible to adaptive carry out filtering background, Thus obtain target area interested, the most again the result obtained is classified.Can effectively keep away based on machine learning Manpower-free arranges multiple threshold value, the model obtained by study, more has universality.This type of method is increasingly becoming target inspection The main flow algorithm in survey field.

Summary of the invention

Embodiment of the disclosure and disclose a kind of traffic lights detection and recognition methods, many including the sample calculating training set Individual channel characteristics, the multiple channel characteristics of pondization is for training detector, the feature pyramid of calculating input image, and uses The feature of feature pyramid different scale is detected to obtain target area by the detector trained by sliding window.

In certain embodiments, non-maxima suppression method is used with the threshold value selected, obtained target area to be carried out Screening is to obtain final goal region.

In certain embodiments, use cap conversion and rim detection, with the threshold value selected, final goal region is carried out essence Trueization.

In certain embodiments, multiple channel characteristics Training Support Vector Machines multi-categorizers of training set sample are utilized.

In certain embodiments, the final goal region of the support vector machine multi-categorizer prediction precision trained is used The classification of middle traffic lights.

In certain embodiments, training set includes the picture as positive sample and the picture as negative sample.

In certain embodiments, multiple channel characteristics include Color Channel feature and gradient orientation histogram feature.

In certain embodiments, the multiple channel characteristics of pondization includes the channel characteristics after carrying out region segmentation is carried out pond Change.

In certain embodiments, training detector include using decision tree forest to train detector, and by repeatedly The detector that each stage is obtained by iteration cascades.

In certain embodiments, under the feature pyramid of calculating input image includes calculating different scale, input picture is many Individual channel characteristics.

Embodiment of the disclosure that disclosed traffic lights detection can solve of the prior art at least one with recognition methods A little above-mentioned deficiencies, solve the traffic lights test problems under complex scene.Embodiment of the disclosure and led to by the integration of study object Road feature, and by pond, integration feature is carried out abstract, utilize the decision tree forest of cascade quickly to detect object, simultaneously Utilizing traffic lights distinctive spatial distribution priori to retrain detection target, this can effectively reduce flase drop, and then improves The degree of accuracy of detection.Use disclosed method, it is possible under different illumination conditions and weather condition, preferably detected effect Really, the impact of deformation can be effectively coped with simultaneously.

Accompanying drawing explanation

Present disclose provides accompanying drawing so that being further appreciated by of disclosure, accompanying drawing constitutes the part of the application, But it is only used for illustrating the non-limiting example of some inventions embodying inventive concept rather than for making any limit System.

Fig. 1 is the flow chart of the traffic lights detection according to some embodiments of the disclosure and recognition methods.

Fig. 2 is the traffic lights detection according to some embodiments of the disclosure and the diagram identifying experimental result.

Fig. 3 is the traffic lights detection according to some embodiments of the disclosure and the diagram identifying experimental result.

Detailed description of the invention

Will hereinafter use the essence that those skilled in the art pass on them to work to others skilled in the art to be led to The term often used describes the inventive concept of the disclosure.But, these inventive concepts can be presented as many different forms, because of And should not be considered limited to embodiments described herein.These embodiments are provided so that present disclosure is more detailed and complete Whole, and it scope included completely is passed on to those skilled in the art.It must also be noted that these embodiments do not have to be mutually exclusive.Come May be assumed that into from assembly, step or the element of an embodiment and can exist or use in another embodiment.Without departing from these public affairs In the case of the scope of the embodiment opened, can substitute shown with diversified alternative and/or equivalent implementations and retouch The specific embodiment stated.The application is intended to cover any amendment or the modification of embodiments discussed herein.

Some aspects in described aspect the most substantially can be only used to put into practice Alternative.The most for purposes of illustration, describe specific numeral, material and configuration in an embodiment, but, field Technical staff in the case of there is no these specific detail, it is also possible to put into practice alternative embodiment.In other cases, may Well-known feature is omitted or simplified, in order to do not make illustrative embodiment be difficult to understand.

Additionally, hereinafter contribute to understanding illustrative embodiment, various operations are described successively in order to multiple discrete Operation；But, described order is not construed as meaning that these operations are necessarily dependent upon this order and perform.But not Must operate to perform these with the order presented.

" in certain embodiments " hereinafter, the phrase such as " in one embodiment " may or may not refer to identical reality Execute example.Term " includes ", " having " and " comprising " is synonym, unless otherwise specified in context.Phrase " A and/or B " mean (A), (B) or (A and B).Phrase " A/B " means (A), (B) or (A and B), is similar to phrase " A and/or B ".Short Language " at least one in A, B and C " means (A), (B), (C), (A and B), (A and C), (B and C) or (A, B and C).Phrase " (A) B " means that (B) or (A and B), i.e. A are optional.

Fig. 1 shows the flow chart of the traffic lights detection according to some embodiments of the disclosure and recognition methods.The disclosure is real Execute the sample conduct of the traffic lights detection in example and the traffic lights identified under the pre-prepd different scenes of use and illumination condition Positive sample, simultaneously using a number of frame without traffic lights as background picture.Constituting for traffic lights detection and identification During training set (i.e. data set), the traffic lights picture as positive sample is inputted with fixing size, and from the back of the body without traffic lights Scape picture carries out randomization according to the size of inputted positive sample, obtains negative sample.At the traffic lights figure as positive sample It the most for the moment, can be zoomed in and out to be adjusted to fixed dimension by sheet original size.In certain embodiments, such as above-mentioned extraction just Sample constitutes the training set for follow-up machine-learning process with negative sample, as shown in step S101, S102, S103 and S104. In step S105, S106, calculate multiple channel characteristics of the training set including positive sample and negative sample.Multiple channel characteristics can To include Color Channel, gradient magnitude and gradient orientation histogram feature etc..

In certain embodiments, the sample of training set can be transformed into hsv color space, and led to by following formula calculating three The feature in road:

$colorfeature(i,j)=\left\{\begin{array}{c}H(i,j)\\ S(i,j)\\ V(i,j)\end{array}\right.---\left(1\right)$

The space coordinates of wherein i, j representative picture, the value of the most corresponding three the different passages of H, S, V.H represents color, Value is 0-254.Such as, red corresponding 0, green corresponding 120, blue corresponding 240；S represents the saturation i.e. brightness of color；V Represent tone.

In certain embodiments, the gradient orientation histogram feature of the sample of training set can be calculated, each in sample image The gradient direction value of pixel can be calculated by following formula:

G_x(x, y)=H (x+1, y)-H (x-1, y) (2)

G_y(x, y)=H (x, y+1)-H (x, y-1)

In above formula, (x y) represents the pixel value in gray space, G to H_x(x, y), G_y(x y) is illustrated respectively in (x, y) place's water The gradient of gentle vertical direction.Thus can obtain gradient magnitude and direction is:

$G(x,y)=\sqrt{G_x{(x,y)}^2+G_y{(x,y)}^2}---\left(3\right)$

$\∂(x,y)={\tan }^{-1}\left(\frac{G_y(x,y)}{G_x(x,y)}\right).$

Then according to the cell size specified, the gradient of pixel each in unit can be carried out in different intervals by direction Ballot.Each interval angular range is 360/N, and wherein N is the quantity of gradient direction.The most i.e. can get the gradient of whole unit Direction histogram, is together in series the rectangular histogram of whole image the most again, obtains the gradient orientation histogram feature that is shown below:

$\mathrm{grad}\mathrm{feature}(i,j)=\∂(i,j,n)---\left(4\right)$

Wherein i, j are the coordinate of picture, n=1,2 ... N, represent different interval number corresponding to gradient direction.

In certain embodiments, the feature of whole image can be expressed as Color Channel feature and gradient orientation histogram is special The set levied:

$Feature(i,j)=\left\{\begin{array}{c}colorfeature(i,j)\\ gradfeature(i,j)\end{array}\right.---\left(5\right)$

Calculate multiple channel characteristics of training set sample by above formula after, obtained channel characteristics can be carried out Region segmentation, determines the size size as pond in each piece of region.Then regional is carried out maximum pond (max Pooling) or meansigma methods pond (average pooling), the channel characteristics of Chi Huahou is obtained, as shown in step S107.? In some embodiments, the channel characteristics before and after pond is processed, the dimension of such as uniform characteristics, and process is tied Fruit combination obtains final channel characteristics for training detector.

In step S108, decision tree forest is used to train detector.In certain embodiments, can pass through AdaBoost algorithm, uses decision tree to train detector.The detector obtained each stage by successive ignition carries out level Connection.During detector is trained, traffic lights spatial distribution probability constraints is added for Adaboost output layer object function to obtain Object functionWhereinIt it is space Distribution probability, x and y is the space coordinates of traffic lights, h_tX () is decision tree classifier at this, α_tFor the power that each grader is corresponding Weight.In step S109, it is thus achieved that housebroken detector.Then image to be analyzed can be inputted, and calculating input image Characteristics of image under multiple dimensioned.

In step s 110 using image to be tested as input, and in step S111, calculate the spy of different scale subsequently Levy, for building the feature pyramid of different scale.In certain embodiments, following formula is used to calculate different scale hypograph many Channel characteristics:

$F_s=R(F,s)\×s^{-{\mathrm{\λ}}_{\mathrm{\Ω}}}---\left(6\right)$

Wherein F_sRepresenting yardstick s characteristic of correspondence, R represents that F=Ω (I) represents image to image use yardstick s resampling The feature of respective channel, Ω is corresponding to different feature passages.And corresponding to the parameter lambda of different characteristic passage_ΩBy simultaneous with Lower formula calculates:

${\mathrm{\μ}}_s=\frac{1}{N}\underset{i=1}{\overset{N}{\Σ}}{f}_{\mathrm{\Ω}}\left(I_s^i\right)/f_{\mathrm{\Ω}}\left(I^i\right)---\left(7\right)$

Wherein μ_sFor statistical data collection global feature along with the average of change of scale.

$f_{\mathrm{\Ω}}\left(I_{s1}\right)/f_{\mathrm{\Ω}}\left(I_{s2}\right)={(s_1/s_2)}^{-{\mathrm{\λ}}_{\mathrm{\Ω}}}+\mathrm{\ϵ}---\left(8\right)$

${\mathrm{\μ}}_s=s^{-{\mathrm{\λ}}_{\mathrm{\Ω}}}+E\[\mathrm{\ϵ}\]---\left(9\right)$

Wherein E [ε] represents the expected value of error, f_Ω(I_s) it is the weighted sum of all passages.

$f_{\mathrm{\Ω}}\left(I\right)={\Σ}_{i,j,k}{\mathrm{\ω}}_{i,j,k}F(i,j,k)---\left(10\right)$

Wherein ω feature is the weight of respective channel, and k represents the sequence of passage.

Simultaneous solution obtains λ_ΩAfter, below equation can be used to obtain the feature pyramid of image:

$F_s\≈R(F_{s^{\′}},s/s^{\′})\×{(s/s^{\′})}^{-{\mathrm{\λ}}_{\mathrm{\Ω}}}---\left(11\right)$

Wherein F_s′It is the feature calculated by standard method, other yardstick s characteristic of correspondence F_sBy above-mentioned formula Carry out approximation to obtain.All yardstick S characteristic of correspondence set constitutive characteristic pyramids.

After the feature of the different scale obtaining image to be detected included by feature pyramid in step S111, by making With the detector of training in step S109, use sliding window to carry out detecting at each yardstick in step S112 thus walked Object candidate area shown in rapid S113.In step S114, in certain embodiments, the candidate region for obtaining uses Non-maxima suppression method filters out final goal region with predetermined threshold value.Can be for detection in non-maxima suppression The bounding box obtained calculates its Duplication: overlap=intersection (bbs).If overlap is more than threshold tau, then only Keep score the highest bbs_i.Wherein the value of τ is set to 0.5, and the score of bbs is sued for peace by the threshold value that the node of decision tree is corresponding Arrive.It should be noted that the value that those skilled in the art it is contemplated that threshold value can be appointed according to image and the practical situation of application scenarios Meaning selects.

Alternatively, cap can be used to become the final goal region of gained as shown in step S115 in certain embodiments Change and rim detection, by selecting suitable threshold value further precision final goal region.It should be noted that those skilled in the art It is contemplated that the value of threshold value arbitrarily can select according to image and the practical situation of application scenarios.Cap converts Blackhat=Close (I, filter)-I may be used for region the darkest in prominent image, obtained by non-maxima suppression Final goal region carry out cap conversion and can be used to the background of removing around traffic lights, it is thus achieved that more accurate traffic lights district Territory.Closed operation during wherein Close () is mathematical morphology, cap conversion is the closed operation difference with original image of image.In step In rapid S116, obtain the final goal region of precision.

Alternatively, multiple dimensioned feature pyramid feature can be utilized in step S117 to use support vector machine (SVM) Train a multi-categorizer, and is classified in the detection region finally given, so that it is determined that the classification of traffic lights.Real at some Execute in example, it is possible to use positive negative sample in above-mentioned training set and the multi-channel feature calculated before train many points an of SVM Class device.May correspond to the final goal region of precision, when utilizing detection, calculated channel characteristics is with the SVM mould trained Type is predicted thus obtains the classification of traffic lights.

The disclosure, by the utilization to multi-channel feature, fully excavates the information of traffic lights in different scene so that it is obtain Feature representation more accurately, can effectively tackle, by pondization operation, the impact that deformation brings.Use the decision tree of cascade Forest detector, it is possible to realize the quick detection to target.By adding spatial distribution probability constraints, optimized detector target letter Number, effectively removes flase drop.Use the abundant search graph of sliding window as space, place, position the region that traffic lights occur, and pass through Morphological scale-space the most accurately detects region, thus promotes the effect of object detection.The model obtained by study can be more Expand to well different scenes, can preferably tackle the impact of the other factorses such as different illumination variation simultaneously.

At central processing unit it isI5-3470 3.2GHz CPU, in save as 16G, OS be WINDOWS 10 operation system In the environment of system, use MATLAB software that the method for disclosure embodiment has been carried out experimental verification.The data used in experiment For real road scene video sequence.First from 34 video segments marked in advance, bag is selected in proof procedure Containing the image of traffic lights, and the detector after training pattern is trained as described in above step.Then according to above-mentioned steps 4 videos are tested to detect traffic lights by S110-S117.By the conduct of accurate rate and recall rate, i.e. precision ratio and recall ratio Weighing the index of Detection results, Detection results is as shown in tables 1 and 2.

Red light	Green light	Background	Negative sample
				1994	2126	725	8000

Form 1

Video sequence number	Frame number	Accuracy rate	Recall rate
				1	360	0.9929	0.8213
2	420	0.9854	0.8782
				3	482	0.9716	0.9033
4	310	0.9486	0.9245

Form 2

In training set, total totally 4120 the positive samples including red light and green light, are carried out above from 725 background pictures Described randomization, obtains 8000 samples negative sample as training.After testing, the average essence of the method for embodiment hereof Really rate is about 97.46%, and average recall rate is about 88.18%, and this shows the whole detection better performances of this method.Fig. 2 and Fig. 3 Respectively illustrate the traffic lights detection according to some embodiments of the disclosure and the diagram identifying experimental result.The result of Fig. 2 and Fig. 3 Display under rainy day and high light conditions the method for disclosure embodiment all can accurately detect traffic lights and judge red light and Green light.The method of disclosure embodiment also has enough robustness with reply weather and the impact of illumination variation.

Part Methods step herein and flow process may need to be performed by computer, thus with hardware, software, firmware and Its any combination of mode is implemented, and can include the executable instruction of computer.The executable instruction of this computer can To store in the form of a computer program product on a machine-readable medium or to carry out in the way of remote server download There is provided, and read by one or more processors of general purpose computer, special-purpose computer and/or other programmable data processing meanss Take and perform the function/action to indicate in implementation method step and flow process.Machine readable media includes but not limited to floppy disk, light Dish, compact disk, magneto-optic disk, read only memory ROM, random access memory ram, electronically erasable programmable rom (EPROM), electrically erasable The transmitting signal of programming ROM (EEPROM), storage card, flash memory and/or electricity, light, sound and other forms (such as carrier wave, red External signal, digital signal etc.).

It is furthermore noted that, term "and/or" herein can represent " with ", "or", distance, " one ", " some but not Whole ", " both neither " and/or " both are all ", but in this regard and unrestricted.Although herein it has been shown and described that The specific embodiment of the disclosure, but apparent to those skilled in the art can be in the situation without departing from scope Under carry out numerous change, change and modifications.It addition, in above-mentioned detailed description of the invention, it can be seen that various features are in single enforcement Example is combined together to simplify disclosure.This publicity pattern should not be construed as the embodiment needs that reflection is claimed Clearly more described than each claim have more features.On the contrary, as reflected in claim, the master of the disclosure What topic relied on is the less feature of feature more all than single disclosed embodiment.Therefore, each claim of claims Item itself remains the most complete embodiment.To sum up, it would be recognized by those skilled in the art that at the model without departing from the disclosure Enclose with spirit in the case of, can be changed and modified in broader each side.Appended claims is in the range of it Cover all this type of falling in disclosure true scope and spirit to change, change and modifications.

Claims (10)

1. traffic lights detection and a recognition methods, including:

Calculate multiple channel characteristics of the sample of training set；

The plurality of channel characteristics of pondization is for training detector；

The feature pyramid of calculating input image；And

The detector trained is used to detect to obtain to the feature of described feature pyramid different scale by sliding window To target area.

2. the method for claim 1, also includes using non-maxima suppression method with the threshold value of selection to obtained Target area carries out screening to obtain final goal region.

3. method as claimed in claim 2, also include using cap conversion and rim detection with the threshold value that selects to described Whole target area carries out precision.

4. method as claimed in claim 3, also includes the plurality of channel characteristics training utilizing described training set sample Hold vector machine multi-categorizer.

5. method as claimed in claim 4, also includes using the support vector machine multi-categorizer prediction trained described accurately The classification of traffic lights in the final goal region changed.

6. the method as according to any one of claim 1-5, wherein said training set includes the picture as positive sample and work Picture for negative sample.

7. the method as according to any one of claim 1-5, wherein said multiple channel characteristics include Color Channel feature and Gradient orientation histogram feature.

8. the method as according to any one of claim 1-5, wherein the plurality of channel characteristics of pondization includes carrying out region Channel characteristics after segmentation carries out pond.

9. the method as according to any one of claim 1-5, wherein trains described detector to include using decision tree forest to come Training detector, and by successive ignition, the detector that each stage obtains is cascaded.

10. the method as according to any one of claim 1-5, wherein the feature pyramid of calculating input image includes calculating not With multiple channel characteristics of input picture under yardstick.