JP2009043067A - Traffic light recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely identify and extract a luminous body of a traffic light from an object such as a sign on the basis of an image photographed with a photographing means. <P>SOLUTION: Since a first extraction means M1 extracts a first object with brightness not lower than a first threshold from an image in front of a vehicle photographed with a television camera 11, although an object such as a sign with lower brightness is excluded, and only the first object with higher brightness including the luminous body of the traffic light is extracted, there is some possibility that a part with lower brightness is eliminated and only a part with higher brightness remains, in the case of the luminous body of the traffic light. A re-extraction area setting means M2 sets a re-extraction area of a prescribed relatively narrow size around the first object, and a second extraction means M3 extracts a second object with brightness lower than the first threshold and not lower than a second threshold in the re-extraction area in which the object with lower brightness such as the sign is not present, so that the whole of the luminous body of the traffic light can be extracted while suppressing a computation load lower. A traffic light determining means M4 thereby accurately determines whether the second object is the luminous body of the traffic light. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a traffic signal recognition device that captures and recognizes a traffic signal with an imaging means mounted on a vehicle.

The traffic light is detected based on the position of the intersection detected by the navigation system and the image in front of the host vehicle captured by the TV camera, and the color of the light-emitting body on which the traffic light is lit is determined to guide the vehicle by voice. This is known from Patent Document 1 below.
Japanese Patent Laid-Open No. 10-177699

  By the way, in the above-mentioned conventional one, the light emitter of the traffic light is distinguished from the difference in hue of red, yellow, and blue (green) and the shape and diameter of the contour. Since the brightness is determined with a certain threshold value, if the threshold value is too low, there is a problem that a low-brightness signboard or the like may be mistakenly recognized as a light emitter of the traffic light. Only a part of the illuminant that is shining strongly can be detected, and there is a possibility that detection failure of the illuminant of the traffic light may occur.

  The present invention has been made in view of the above circumstances, and an object thereof is to reliably identify and extract a light emitter of a traffic light from an object such as a signboard based on an image captured by an imaging unit.

  In order to achieve the above object, according to the first aspect of the present invention, in a traffic signal recognition device that captures and recognizes a traffic signal by an imaging means mounted on a vehicle, from an image in front of the vehicle captured by the imaging means. First extraction means for extracting a first object having a luminance equal to or higher than a first threshold; re-extraction area setting means for setting a re-extraction area of a predetermined size around the first object; and brightness in the re-extraction area Comprising: a second extraction unit that extracts a second object that is equal to or greater than a second threshold value that is less than the first threshold value; and a traffic light determination unit that determines whether the second object is a light emitter of a traffic light. A traffic light recognition device is proposed.

  According to the second aspect of the present invention, in addition to the configuration of the first aspect, the second threshold value setting means for setting the second threshold value is provided, and the second threshold value setting means includes the second object. A traffic signal recognizing device is proposed in which the second threshold value is set based on the maximum luminance.

  Note that the television camera 11 of the embodiment corresponds to the imaging means of the present invention.

  According to the configuration of the first aspect, since the first extraction unit extracts the first object having the luminance equal to or higher than the first threshold from the image in front of the vehicle imaged by the imaging unit, an object such as a signboard with low luminance is excluded. Only the first object with high luminance including the light emitter of the traffic light is extracted. However, when the first object is the light emitter of the traffic light, there is a possibility that the low luminance part is missing and only the high luminance part remains. . The re-extraction area setting means sets a re-extraction area of a predetermined size around the first object, and the second extraction means has a luminance in a relatively narrow re-extraction area where there is no low-luminance object such as a signboard. Since the second object that is equal to or greater than the second threshold value that is less than the first threshold value is extracted, the entire light emitter of the traffic light can be extracted while suppressing the calculation load and preventing erroneous detection of a signboard or the like. Therefore, the traffic light determining means can accurately determine whether or not the second object is a light emitter of the traffic light.

  According to the second aspect of the present invention, since the second threshold value setting means for setting the second threshold value sets the second threshold value based on the maximum luminance of the first object, the light emitter of the traffic light is extracted with the optimum threshold value. can do.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  1 to 7 show an embodiment of the present invention. FIG. 1 is a block diagram of an electronic control unit of a traffic signal recognition device. FIG. 2 is a diagram showing an image in front of the own vehicle taken by a TV camera. FIG. 4 is a flowchart of a main routine, FIG. 5 is a flowchart of a subroutine of step S1 in FIG. 4, FIG. 6 is a flowchart of a subroutine of step S2 in FIG. Is a map for determining the light emitter of the traffic light from the position and radius of the second object.

  As shown in FIG. 1, the electronic control unit U of the traffic signal recognition apparatus of the present embodiment includes a first extraction unit M1, a re-extraction area setting unit M2, a second extraction unit M3, a traffic signal determination unit M4, Second threshold setting means M5. The first extraction unit M1 is connected to a television camera 11 as an imaging unit mounted on the vehicle.

  The electronic control unit U stores a first threshold value of luminance in advance, and the first extraction means M1 has the first luminance value from the front image (see FIG. 2) in the traveling direction of the host vehicle captured by the TV camera 11. A first object having a threshold value or more is extracted. As shown in FIG. 2, in the image captured by the television camera 11, signboards 13, 14 and the like are shown in addition to the traffic light 12, and the signboards 13, 14 may be erroneously recognized as the traffic light 12. It is necessary to be surely excluded from the object. There is a large difference in luminance between the traffic light 12 and the signboards 13 and 14, the luminance of the light emitting body of the traffic light 12 that emits light by a lamp or LED is high, and the brightness of the signboards 13 and 14 that do not emit light themselves is low. Therefore, by setting the first threshold value to be higher than the luminance of the signboards 13 and 14 and lower than the luminance of the light emitter of the traffic light 12, there is a possibility of the light emitter of the traffic light 12 excluding the signboards 13 and 14. A certain first object can be extracted.

  FIG. 3A shows a case where the threshold value is too low, and an area that protrudes beyond the contour of the light emitter of the traffic light 12 is extracted. With such a too low threshold, not only the size and shape of the illuminant cannot be recognized correctly, but also the signs 13 and 14 are extracted, so that the traffic lights 12 and the signs 13 and 14 cannot be identified.

  FIG. 3B shows the case where the first threshold value is adopted as the threshold value, and not the entire light emitter of the traffic light 12 but only the high luminance portion at the center is extracted so that the size and shape of the light emitter are correct. Although it cannot be recognized, since the signboards 13 and 14 are not extracted, the traffic light 12 and the signboards 13 and 14 can be identified.

  The re-extraction area setting means M2 sets a predetermined range (for example, 40 pixels × 40 pixels) as a re-extraction area (see FIG. 2) around the portion where the maximum luminance of the light emitter is detected, and the second extraction means M3 Performs re-extraction in the re-extraction region using a new second threshold value. The second threshold value setting means M5 for setting the second threshold value used at this time extracts the second object with a second threshold value lower than the first threshold value. Since the region where re-extraction is performed is a limited range in the vicinity of the first object (for example, the light emitter of the traffic light 12), even if extraction is performed with a low second threshold, the low-intensity signboards 13, 14 and the like are There is no risk of being extracted as two objects.

  When extraction is performed using the second threshold value, as shown in FIG. 3B, the first object (for example, the light emitter of the traffic light 12) extracted with the first threshold value having a high value is larger than the actual size. In contrast, as shown in FIG. 3C, the light emitter of the traffic light 12 can be extracted with a size close to the actual size. And the traffic light determination means M4 determines whether the 2nd object extracted by the 2nd extraction means M3 is a light-emitting body of the traffic light 12, based on the dimension, a position, a color, etc.

  Next, the above operation will be described in more detail based on the flowcharts of FIGS.

  In step S1 of the main routine of FIG. 4, a lighting area (high luminance area) including the light emitter area of the traffic light 12 is extracted from the image of the television camera 11, and the traffic light 12 is detected from the lighting area in a subsequent step S2.

  In step S11 of the flowchart of FIG. 5 showing the lighting region extraction routine, an image in front of the vehicle is picked up by the TV camera 11, and each pixel of the image is represented by RGB (three primary colors of red, green, and blue light) in step S12. Color representation method) to HSV [color representation method by chromaticity (hue), saturation, brightness (brightness)]. In subsequent step S13, a pixel having a first threshold value or more is detected as a first object from each pixel of the image. The first threshold value is set for each of chromaticity, saturation, and luminance. For example, the chromaticity is 0 to 150, 331 to 360 (in terms of 0 to 360) (or 0 to 100%). 0 to 42%, 55 to 100%), saturation is 270 to 360 (75 to 100%), and luminance is 180 to 360 (50 to 100%).

  In the subsequent step S14, the maximum values of the luminance and saturation of the pixels equal to or higher than the first threshold value are calculated, and in step S15, the second threshold value including the luminance and saturation is set to 50% of the maximum value. That is, the brightness of the second threshold is set to 50% of the maximum brightness, and the saturation of the second threshold is set to 50% of the maximum saturation.

  In the subsequent step S16, a re-extraction area of 20 pixels × 20 pixels is set centering on the pixel having the maximum luminance, and in the re-extraction area, a pixel having the second threshold value or more is extracted as a second object in the re-extraction area.

  The area formed by the pixels having the second threshold value or more in step S21 of the flowchart of FIG. 6 showing the traffic light detection routine is a circle that is the shape of the light emitter of the traffic light 12, and the pixels having the second threshold value or more are formed in step S22. If the height and diameter of the center of the area are within a predetermined range, it is determined that this is the light emitter of the traffic light 12.

  FIG. 7 is a map showing the criteria for judgment in step S22. The vertical axis indicates the position of the pixel having the maximum luminance on the screen, and the horizontal axis indicates the radius of the region formed by the pixel having the second threshold value or more. If the traffic light 12 as the second object is close to the own vehicle, the position is on the upper side of the screen and the radius is larger. Conversely, if the traffic light 12 is far from the own vehicle, the position is on the lower side of the screen. In addition, since the radius is small, the second object existing in the shaded area in FIG. 7 can be determined as the light emitter of the traffic light 12.

  When the second object is thus determined to be the light emitter of the traffic light 12, the process proceeds to step S23, where the chromaticity is 0 to 30, 331 to 360 (0 to 8%, 92 to 100%). If it is within the range, it is determined in step S24 that the red signal is lit. If the chromaticity is in the range of 31 to 90 (9 to 25%) in step S25, it is determined that the yellow signal is lit in step S26. If the chromaticity is in the range of 91 to 150 (26 to 42%) in step S27, it is determined that the blue (green) signal is lit in step S28.

  As described above, by extracting the first object having a luminance equal to or higher than the first threshold from the image in front of the vehicle imaged by the TV camera 11, objects such as the signboards 13 and 14 having a low luminance are excluded, and the traffic light 12 Only the first object with high luminance including the illuminant is extracted, but the traffic light 12 may not be correctly recognized as the illuminant of the traffic light 12 because the low luminance part of the illuminant is missing and only the high luminance part remains. There is. Therefore, a re-extraction area of a predetermined size is set around the first object, and the luminance of the re-extraction area where a low-luminance object such as the signboard 13 or 14 does not exist is a second threshold that is equal to or higher than a second threshold that is less than the first threshold. Since two objects are extracted, the entire light emitter of the traffic light 12 can be extracted by excluding objects such as the signboards 13 and 14 while keeping the calculation load low. Thereby, it can be accurately determined whether or not the second object is the light emitter of the traffic light 12. Moreover, since the second threshold value for extracting the second object is set as a half value of the maximum luminance of the first object, the light emitter of the traffic light 12 can be extracted with the optimum second threshold value.

  Although the embodiments of the present invention have been described above, various design changes can be made without departing from the scope of the present invention.

  For example, although the first threshold value is set based on chromaticity, saturation, and luminance in the embodiment, it may be set based only on luminance.

  In the embodiment, the second threshold is set based on the luminance and the saturation, but only the luminance is set. Or you may set based on a brightness | luminance, saturation, and chromaticity.

  In the embodiment, the horizontal traffic light 12 is exemplified, but the present invention can be applied to a vertical traffic light.

Block diagram of electronic control unit of traffic light recognition device The figure which shows the picture ahead of the own car which is photographed with the television camera Diagram showing light emitters of traffic lights extracted with different thresholds Main routine flowchart Flowchart of the subroutine of step S1 in FIG. Flowchart of the subroutine of step S2 in FIG. Map for determining light emitter of traffic light from position and radius of second object

Explanation of symbols

11 TV camera (imaging means)
12 traffic light M1 first extraction means M2 re-extraction area setting means M3 second extraction means M4 traffic light determination means M5 second threshold value setting means

Claims (2)

In a traffic signal recognition device that captures and recognizes a traffic signal (12) by an imaging means (11) mounted on a vehicle,
First extraction means (M1) for extracting a first object having a luminance equal to or higher than a first threshold from an image ahead of the vehicle imaged by the imaging means (11);
Re-extraction area setting means (M2) for setting a re-extraction area of a predetermined size around the first object;
A second extracting means (M3) for extracting a second object having a luminance equal to or higher than a second threshold value less than the first threshold value in the re-extraction region;
Traffic light determination means (M4) for determining whether or not the second object is a light emitter of the traffic light (12);
A traffic light recognizing device comprising:   A second threshold value setting unit (M5) for setting the second threshold value, wherein the second threshold value setting unit (M5) sets the second threshold value based on a maximum luminance of the first object; The traffic light recognition apparatus according to claim 1.

Images