CN104021378B - Traffic lights real-time identification method based on space time correlation Yu priori - Google Patents

Abstract

Traffic lights real-time identification method based on space time correlation and priori belongs to the transport information detection field of intelligent transportation industry.The present invention positions area-of-interest on the original image first with priori, and the region unrelated with traffic lights is filtered out by empirical value.Then, signal lamp red green region is extracted and on this basis using shape facility filtering.The subregion after filtering is read in afterwards, the HOG features of subregion are extracted successively, recycle grader to be trained signal lamp sample.Finally, current demand signal lamp is identified according to the discriminant function of grader.Such as front green light, can travel；If front red light, sends stop sign.If the two is all present, determine whether to travel according to space time correlation information and place track.The present invention meets the detection recognition point of traffic lights, can real-time and accurately detect traffic lights information, applies in the middle of intelligent vehicle, aids in its correct safety traffic.

Description

Traffic lights real-time identification method based on space time correlation Yu priori

Technical field

The present invention is the real-time Dynamic Signal lamp identification side that a kind of utilization video image is based on space time correlation and priori Method, belongs to the transport information detection field of intelligent transportation industry.

Background technology

With society and economic fast development, intelligent vehicle is increasingly paid close attention to by people.Reason is following：Intelligence Can vehicle can substitute driver driving, reduce traffic accident generation, or even make colour blindness anomalous trichromatism patient drive a car turn into can Energy.For the traveling of vehicle, at crossing, accurately identification traffic lights color is necessary in real time.Therefore, for solving nobody The real-time dynamic traffic light detection identification of driving, also lifting and progressive space.

Solve this problem it may first have to the Position Approximate of traffic lights is obtained from complicated background picture, Then further position and recognize, and judge the specifying information of traffic lights.At present, the recognition methods for signal lamp is mainly The various features information of basis signal lamp.The method detection has certain accuracy rate, but real-time is not strong with broad applicability, It is not particularly suited for intelligent vehicle.

The content of the invention

In the middle of intelligent vehicle, crossroads traffic light information is accurately identified it is an object of the invention to by real time execution of the present invention, Improve the security of its traveling, it is proposed that a kind of traffic lights real-time identification method based on space time correlation Yu priori.

To achieve these goals, this invention takes following technical scheme：

The camera position of collection video fixed first, installed in the rearview mirror centre position of intelligent automobile, apart from ground 1.2 meters, camera wide-angle is more than 120 degree, and resolution ratio is more than 640*480.

First, the area-of-interest (ROI) of image is determined based on priori.Exist with reference to camera installation site and signal lamp Position in image, determines to obtain the ROI of image using the space time correlation information.In the middle of the process that signal lamp draws near, The position occurred in picture is bottom-up, when position reaches picture height 1/2, start can accurate marker lamp, therefore Determine that top half is the ROI for obtaining image.

2nd, the red green region of ROI is extracted.Video camera capture images type is RGB, but due to RGB color light Larger according to influenceing, with unstability, and the color of signal lamp is easier to split under YCbCr space, therefore will capture picture Color space is converted to YCbCr by RGB.Red and green area in ROI is extracted after converted respectively.Come just with this color region Step framing signal lamp.

3rd, red, the green area that extract are filtered using shape facility, to exclude other red, green non-letters The interference of signal lamp object.Shape facility includes area features, boundary rectangle feature and density feature etc., with the reality of signal lamp Shape facility value is further accurately positioned signal lamp as threshold value.

4th, the HOG features in red green region are extracted.The edge of signal lamp backboard is an obvious feature, and HOG is substantially It is the statistics of gradient information, and gradient is primarily present in the place at edge, so the feature that HOG features are applied to signal lamp is carried Take, as the input item of grader after the completion of this feature extraction, judgement is recognized by grader (present invention is SVM classifier) Whether detection zone is signal lamp.

5th, SVM classifier training signal lamp sample, generates signal lamp grader.Video conduct under the actual road conditions of collection Experimental data, the intercept signal lamp picture from video, including red, green state circular lamp, arrow lamp is used as training sample, instruction Practice 2 classifiers of generation, the signal lamp of 4 types is capable of identify that per classifiers.(it is circular, turn left, straight trip is turned right)

6th, with reference to priori, space time correlation and grader marker lamp.Recognized first with grader discriminant function The classification of signal lamp, afterwards with reference to priori, the further accurate identification information of space time correlation, with the identification knot of debug Really.

A kind of traffic lights real-time identification method based on space time correlation Yu priori, it is characterised in that including with Lower step：

(1) camera position of collection video fixed first, installed in the rearview mirror centre position of intelligent automobile, distance ground Face 1-1.2 meters, camera wide-angle is more than 120 degree, and resolution ratio is more than 640*480；By the video camera on intelligent vehicle, with 20- The frame per second of 50 frames/second obtains the image image of intelligent vehicle road ahead in real time；With reference to priori, camera is installed on intelligence The surface of energy vehicle, the position that signal lamp occurs in picture is bottom-up, when position reaches picture height 1/2, starts Being capable of accurate marker lamp；Therefore set area-of-interest as the top half of picture；

(2) color of signal lamp is extracted；The picture/mb-type that camera acquisition is arrived is RGB, and picture color space is turned from RGB YCbCr is changed to, conversion formula is as follows：

The extraction of signal lamp red and green is carried out under Cb passages, the color region of signal lamp is tentatively found；R_minFor red The minimum value 108, R of chromatic threshold value_maxIt is the maximum 160.G of red threshold_minIt is the minimum value 10, G of green threshold_maxIt is green Maximum 60.f (x, y) of threshold value represents the pixel value at point (x, y) place；

After color region finds, the field color to finding strengthens, and other extraneous areas colors weaken；

(3) characteristic filter；Upper figure color region is carried out into binaryzation, is filtered according to following characteristics successively；

Filter condition 1：Area；All profiles in traversal binary map, ask for the area of all profiles, and area is in certain limit Interior region retains；

It is assumed that image has N_rIndividual candidate region, is designated as R_i, i=1 ... N_r；A_maxThe maximum of contour area is represented, value is 8000, A_minThe minimum value of contour area is represented, value is 60；(R_i(A) i-th area of profile) is represented；

Filter condition 2：Shape；The boundary rectangle of each profile is asked for, wide, the height of boundary rectangle is obtained, filter condition is outer Meet the ratio of width to height rate R of rectangle_wh, the region that ratio meets following condition retains；

Filter condition 3：Density；Seek the density of each profile in binary map；For signal lamp, in color region block All pixels point is same color, and arranges tight between pixel, and the density tried to achieve is big；F (x, y) represents binary map midpoint The pixel value at (x, y) place, A is contour area；Region of the density more than 0.6 retains；

(4) HOG features are extracted successively to the red green region for remaining；

(5) video under actual road conditions is gathered, signal lamp picture is therefrom partitioned into as sample, these is trained using SVM Signal lamp sample, generates signal lamp grader；Training positive sample includes following kind of picture：Red circular, green are circular, red Color left-hand rotation arrow, red straight trip arrow, red right-hand rotation arrow, green left-hand rotation arrow, green straight trip arrow, green right-hand rotation arrow； Negative sample is the non-signal lamp picture that random shearing goes out from the video image for obtaining；The grader for training includes red, green Two groups of color, every group includes 4 graders again, is respectively circle, left-hand rotation, straight trip, the grader of right turn signal lamp；

(6) identification of signal lamp；To the above-mentioned extracted region HOG features remained by characteristic filter, feature is designated as X, the grader group of corresponding color is selected according to the color in the region, 4 graders of group HOG features x is substituted into afterwards where To carry out Classification and Identification successively, corresponding classification results are obtained；Classification results include the color and type of signal lamp:It is circular, left Turn, keep straight on, turn right；

The centre coordinate value in each signal lamp region is obtained in identification process, judges that the size of centre coordinate abscissa is closed System can obtain the positional information of each signal lamp, and the types results of combining classification device identification obtain the complete information of signal lamp；

It is identified using space time correlation information afterwards；

Association in time：

(1) using inter frame image it is continuous in time series the characteristics of be identified；By inter frame image in time series Upper continuous feature, it is ensured that wrong identification result does not send；

(2) it is identified using the colour switching sequential nature of same signal lamp；

Space correlation：

(1) position relationship using signal lamp in space is identified；

(2) it is identified using signal lamp position relationship parallel in the horizontal direction in itself.

Compared with existing detection method, the invention has the advantages that：(1) invention be applied to automatic driving car it On, speed, the process time per frame has stronger practical value at 100ms or so (resolution ratio 1000*1000).(2) originally Invention is capable of identify that polytype traffic lights, can be prevented effectively from the interference of other objects such as tail-light.(3) present invention is different from Existing method recognizes traffic lights, combines space time correlation and priori, employs machine learning method, improves accuracy rate And real-time.As shown in Figure 6, judge that Fig. 6 is identification error result by space time correlation information, identification information does not send.It is attached Fig. 7, it is shown that per frame process time and recognition result information.

Brief description of the drawings

Fig. 1 signal lamp identification process figures of the invention；

The artwork of Fig. 2 video cameras capture；

Fig. 3 color extraction result figures；

Fig. 4 characteristic filter result figures；

Fig. 5 traffic lights recognition result figures；

Fig. 6 space time correlation result figures；

Fig. 7 is per frame process time result figure.

Specific embodiment

The invention will be further described below in conjunction with the accompanying drawings：

First, the camera position of collection video fixed first, installed in the rearview mirror centre position of intelligent automobile, distance Ground 1-1.2 meters, camera wide-angle is more than 120 degree, and resolution ratio is more than 640*480.By the video camera on intelligent vehicle, with The frame per second of 20-50 frames/second obtains the image image of intelligent vehicle road ahead in real time.With reference to priori, camera is installed on The surface of intelligent vehicle, the position that signal lamp occurs in picture is bottom-up, when position reaches picture height 1/2, opens Beginning being capable of accurate marker lamp.Therefore it is the top half of picture to set area-of-interest (ROI)；

Secondly, the color of signal lamp is extracted.The picture/mb-type that camera acquisition is arrived is RGB, by picture color space from RGB YCbCr is transformed into, conversion formula is as follows.

As can be seen that carrying out the extraction of signal lamp red and green under cb passages after color space conversion, tentatively look for To the color region of signal lamp.R_minIt is the minimum value 108, R of red threshold_maxIt is the maximum 160, G of red threshold_minFor green The minimum value 10, G of chromatic threshold value_maxIt is the maximum 60 of green threshold, f (x, y) represents the pixel value at point (x, y) place.

After color region finds, the field color to finding strengthens, and other extraneous areas colors weaken, such as accompanying drawing Shown in 3.

3rd, characteristic filter.Upper figure color region is carried out into binaryzation, is filtered according to following characteristics successively.

Filter condition 1：Area；All profiles in traversal binary map, ask for the area of all profiles, and area is in certain limit Interior region retains.Such as red flag, its area is unsatisfactory for threshold range very much greatly, can be filtered by the condition Fall.

It is assumed that image has N_rIndividual candidate region, is designated as R_i, i=1 ... N_r。A_maxThe maximum of contour area is represented, value is 8000, A_minThe minimum value of contour area is represented, value is 60.(R_i(A) i-th area of profile) is represented.

Filter condition 2：Shape；The boundary rectangle of each profile is asked for, wide, the height of boundary rectangle is obtained, filter condition is outer Meet the ratio of width to height rate R of rectangle_wh, the region that ratio meets following condition retains.

Filter condition 3：Density；Seek the density of each profile in binary map.For signal lamp, in color region block All pixels point is same color, and arranges tight between pixel, and the density tried to achieve is big.A_iIt is i-th area of profile, Sum_iIt is the summation of all pixels point number in i-th profile.ρ represents the density of profile, and region of the density more than 0.6 retains.

4th, HOG features are extracted in the red green region to remaining successively.HOG features preferably characterize object Gradient direction feature, and the edge of signal lamp is obvious, it is adaptable to use this feature.

5th, the video under actual road conditions is gathered, signal lamp picture is therefrom partitioned into as sample, train this using SVM A little signal lamp samples, generate signal lamp grader.Training positive sample includes following kind of picture：Red circular, green be circular, Red left-hand rotation arrow, red straight trip arrow, red right-hand rotation arrow, green left-hand rotation arrow, green straight trip arrow, green right-hand rotation arrow Head.Negative sample is the non-signal lamp picture that random shearing goes out from the video image for obtaining.The collection of positive negative sample will be included not Situations such as same weather, different time sections, there are enough generalization abilities with the grader for ensureing training.The grader for training includes Red, green two groups, every group includes 4 graders again, is respectively circle, left-hand rotation, straight trip, the grader of right turn signal lamp.

Finally, the identification of signal lamp.To the above-mentioned extracted region HOG features remained by characteristic filter, feature note It is x, the grader group of corresponding color is selected according to the color in the region, 4 classification of group HOG features x is substituted into afterwards where Device (each grader has been trained, known to weight a and biasing b) carries out the Classification and Identification (recognition function of grader successively It is ax+b>0, current signature meets the recognition function of which grader, and the type of signal lamp is exactly the type of the grader), i.e., Can obtain corresponding classification results.Classification results include the color and type (circular, left-hand rotation, straight trip, right-hand rotation) of signal lamp.

If for example, present image be one right-hand rotation greensignal light, then extracted green light HOG features it Afterwards, the grader group of green classification is selected.(group has green circular, green left-hand rotation, green straight trip, green to turn right 4 and classify Device), x is substituted into green circle grader first, the recognition function of the grader is unsatisfactory for, therefore be not circular lamp, continue next Step；X is substituted into green left-hand rotation grader, is unsatisfactory for, next step；X is substituted into green straight trip grader, is unsatisfactory for, next step； X substitutes into green right-hand rotation grader, meets, and it is green circular signal lamp to obtain result.

Simultaneously as one group of number of signal lamp one to three, therefore will also be to each recognition result affix position Information.The centre coordinate value in each signal lamp region is obtained in identification process, the magnitude relationship of centre coordinate abscissa is judged The positional information of each signal lamp is can obtain, the types results of combining classification device identification obtain the complete information of signal lamp.With There are 2 signal lamps as a example by Fig. 2 in accompanying drawing, in Fig. 2, left side is red straight trip, and right side is turned right for green.By above-mentioned grader Identification can identify that, in the presence of 2 lamps, red straight trip is turned right with green, is next exactly to obtain this 2 centre coordinates in region, The size for comparing abscissa understands that red area keeps left, and green area is kept right, therefore final result is 2 lamps：The red straight trip in left side, Right side green is turned right.

Accurately recognized using space time correlation information afterwards.

Association in time：

(1) using inter frame image it is continuous in time series the characteristics of accurately recognized.By this feature, it is ensured that mistake Recognition result does not send.By taking a red eye as an example, recognition result the 1st arrives N (20<=N<=40) frame for red, if N+1 frames are to N+i frames (5<=i<=10) image recognition result is for green or does not detect, and N+i+1 frames are to N+M (20<M<40) frame recognition result is red, then recognition result of the invention is red.For from temporal associativity, only After red light is continued for some time, green light could occur.

(2) accurately recognized using the colour switching sequential nature of same signal lamp.In terms of temporal associativity, together One signal lamp is with green light -- sequential transformations of amber light -- red light -- green light.If recognition result the 1st arrives N (80<=N<=200) Frame is green light, N+1 to N+M (1<=M<=200) frame be red light, then identification is made a mistake, and a few frame results are abandoned.

Space correlation：

(1) position relationship using signal lamp in space is accurately recognized.So that one group of signal lamp has 3 lamps as an example. The orders that generally 3 lamps are indicated turn right or turn left for-straight trip-of turning left, and-straight trip-straight trip or straight trip-straight trip-are straight OK, but if the abnormal conditions such as right-hand rotation-straight trip-straight trip occurs in recognition result, then the frame result is abandoned.Because being closed from space For in connection property, right-turn lane can not possibly be located at the leftmost side, due to the appearance of the sequence of positions in spatial relationship, according to empty Between relevance principle can recognize that more accurate object information so that ensure intelligent vehicle it is accurate, in real time, safety by crossing.

(2) accurately recognized using signal lamp position relationship parallel in the horizontal direction in itself.Have 3 with one group of signal lamp As a example by individual lamp.3 positions of lamp are necessarily parallel in the horizontal direction, that is to say, that the ordinate position of 3 lamps should differ Less.Therefore from spatial relationship for, if comprising multiple lamps in recognition result, certain lamp has with the vertical position coordinates of other lamps Larger gap, then must wrong result occur, can be excluded with reference to other information.

Wherein, L_iIt is i-th center ordinate of signal lamp, D is threshold value 100.

Claims (1)

1. a kind of traffic lights real-time identification method based on space time correlation Yu priori, it is characterised in that including following Step：

(1) camera position of collection video fixed first, installed in the rearview mirror centre position of intelligent automobile, apart from ground 1- 1.2 meters, camera wide-angle is more than 120 degree, and resolution ratio is more than 640*480；By the video camera on intelligent vehicle, with 20-50 frames/ The frame per second of second obtains the image image of intelligent vehicle road ahead in real time；With reference to priori, camera is installed on intelligent vehicle Surface, the position that signal lamp occurs in picture is bottom-up, and when position reaches picture height 1/2, starting can be accurate True marker lamp；Therefore set area-of-interest as the top half of picture；

(2) color of signal lamp is extracted；The picture/mb-type that camera acquisition is arrived is RGB, and picture color space is transformed into from RGB YCbCr, conversion formula is as follows：

$\left\{\begin{array}{c}Y=0.257\&times;R+0.564\&times;G+0.098\&times;B+16\\ Cb=-0.148\&times;R-0.291\&times;G+0.439\&times;B+128\\ Cr=0.439\&times;R-0.368\&times;G-0.071\&times;B+128\end{array}\right.---\left(1\right)$

The extraction of signal lamp red and green is carried out under Cb passages, the color region of signal lamp is tentatively found；R_minIt is red threshold The minimum value 108, R of value_maxIt is the maximum 160, G of red threshold_minIt is the minimum value 10, G of green threshold_maxIt is green threshold Maximum 60, f (x, y) represent point (x, y) place pixel value；

$\begin{array}{c}\left\{f(x,y)\right|R_{\min }\&le;f_{cb}\&le;R_{\max }\},Color=\mathrm{Re}d\\ \left\{f(x,y)\right|G_{\min }\&le;f_{cb}\&le;G_{\max }\},Color=Green\end{array}---\left(2\right)$

After color region finds, the field color to finding strengthens, and other extraneous areas colors weaken；

$\stackrel{\&OverBar;}{C_{RGB}(R,G,B)}=\left\{\begin{array}{c}{(255,0,0)}^T,Color=\mathrm{Re}d\\ {(0,255,0)}^T,Color=Green\\ {(128,128,128)}^T,Color=Else\end{array}\right.---\left(3\right)$

(3) characteristic filter；Above-mentioned picture color region is carried out into binaryzation, is filtered according to following characteristics successively；

Filter condition 1：Area；All profiles in traversal binary map, ask for the area of all profiles, and area is within the specific limits Region retains；

It is assumed that image has N_rIndividual candidate region, is designated as R_i, i=1 ... N_r；A_maxThe maximum of contour area is represented, value is 8000, A_minThe minimum value of contour area is represented, value is 60；R_i(A) i-th area of profile is represented；

$Bool\left(R_i\right(A\left)\right)=\left\{\begin{array}{c}1,A_{max}\&le;A\&le;A_{min}\\ 0\end{array}\right.---\left(4\right)$

Filter condition 2：Shape；The boundary rectangle of each profile is asked for, wide, the height of boundary rectangle is obtained, filter condition is external square The ratio of width to height rate R of shape_wh, the region that ratio meets following condition retains；

$Bool\left(R_i\right(R_{wh}\left)\right)=\left\{\begin{array}{c}1,0.8\&le;R_{wh}\&le;1.5\\ 0\end{array}\right.---\left(5\right)$

Filter condition 3：Density；Seek the density of each profile in binary map；It is all in color region block for signal lamp Pixel is same color, and arranges tight between pixel, and the density tried to achieve is big；F (x, y) expression binary maps midpoint (x, Y) pixel value at place, A is contour area；Region of the density more than 0.6 retains；

$\mathrm{\&rho;}=\frac{A}{\underset{i=1}{\overset{m}{\&Sigma;}}\underset{j=1}{\overset{n}{\&Sigma;}}f(x,y)}---\left(6\right)$

(4) HOG features are extracted successively to the red green region for remaining；

(5) video under actual road conditions is gathered, signal lamp picture is therefrom partitioned into as sample, these signals is trained using SVM Lamp sample, generates signal lamp grader；Training positive sample includes following kind of picture：Red circular, green are circular, red left Turn arrow, red straight trip arrow, red right-hand rotation arrow, green left-hand rotation arrow, green straight trip arrow, green right-hand rotation arrow；Negative sample Originally it is the non-signal lamp picture that goes out of random shearing from the video image for obtaining；The grader for training includes red, green two Group, every group includes 4 graders again, is respectively circle, left-hand rotation, straight trip, the grader of right turn signal lamp；

(6) identification of signal lamp；To the above-mentioned extracted region HOG features remained by characteristic filter, feature is designated as x, according to According to the region color select corresponding color grader group, afterwards HOG features x substitute into where group 4 graders come according to It is secondary to carry out Classification and Identification, obtain corresponding classification results；Classification results include the color and type of signal lamp:It is circular, turn left, it is straight Row, right-hand rotation；

The centre coordinate value in each signal lamp region is obtained in identification process, the magnitude relationship of centre coordinate abscissa is judged i.e. The positional information of each signal lamp is can obtain, the types results of combining classification device identification obtain the complete information of signal lamp；

It is identified using space time correlation information afterwards；

Association in time：

(1) using inter frame image it is continuous in time series the characteristics of accurately recognized, it is ensured that wrong identification result does not send, Recognition result the 1st to N frames be the first color, if N+1 frames to N+i two field pictures recognition result be the second color or without examine Measure, and N+i+1 frames are first color to N+M frames recognition result, then last recognition result is first color, its In, the scope of N is 20<=N<The scope of=40, i is 5<=i<The scope of=10, M is 20<M<40；

(2) accurately recognized using the colour switching sequential nature of same signal lamp, the frame that discarding makes a mistake；

Space correlation：

(1) sequence of positions using one group of signal lamp in the identical or different direction of instruction in space accurately recognized, is abandoned The order of signal lamp locus described in recognition result is improper frame；

(2) it is parallel in the horizontal direction using one group of signal lamp and accurately known in the position relationship that ordinate position is more or less the same , the vertical position coordinates of certain lamp and other lamps does not have larger gap, then must wrong result occur, can be arranged with reference to other information Remove, judgment mode is：

Wherein, L_iIt is i-th center ordinate of signal lamp, D is threshold value 100.