JP2007072948A - Traffic signal lighting state identification method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To identify with a single camera or the like whether or not there is a traffic signal related to traffic of its own vehicle and a lighting state even with respect to light a feature point of which is hard to obtain by a small calculation amount. <P>SOLUTION: The traffic signal lighting state identification apparatus is for photographing near a front of a mobile body, such as an automobile, and identifying either presence of the traffic signal included in the image or the lighting state or both and is provided with: an ellipse decision part 12 for deciding whether or not the light included in the image is elliptic; a normal vector calculation part 13 for obtaining a normal vector from a major axis and minor axis of the elliptic shape; and a light decision part 14 for deciding according to the normal vector of the light whether or not the light is related to the traffic of its own vehicle and identifying from the image a light of the traffic signal for its own vehicle to follow. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a traffic light lighting state identification method and apparatus for identifying either or both of the presence and absence of a traffic light and a lighting state included in an image obtained by photographing the vicinity of the front of a moving body such as an automobile.

2. Description of the Related Art Conventionally, a technique has been proposed in which a camera is installed on a moving body such as an automobile and traffic lights, traffic signs, and the like are recognized based on images captured by the camera. For example, Patent Document 1 identifies traffic lights, traffic signs, and the like from images taken by two cameras using a technique such as stereo matching, and the method is based on three points on the identified traffic lights, traffic signs, etc. A technique is disclosed in which a line vector is obtained, and it is determined from the normal vector whether the vehicle is a traffic signal or a traffic sign to be followed.
JP 2004-272694 A

  In the technique disclosed in Patent Document 1, three-dimensional coordinates of three points on an object surface are obtained from images taken by two cameras using a technique such as stereo matching, and then three-dimensional three-dimensional. It is necessary to obtain the normal vector of the object from the coordinates.

  However, the calculation of the three-dimensional coordinates by stereo matching has a problem that the amount of calculation is large because it is necessary to scan two images while corresponding to each other's color information.

  For objects that are difficult to identify feature points, such as signal lights that are lit at night, the corresponding feature points required for stereo matching from two images taken with two cameras are accurately detected. There was a problem that it was difficult to specify well.

  Furthermore, there has been a problem that two cameras are required in order to obtain the normal vector of the lamp by stereo matching.

  Therefore, the present invention solves the above-described problems, and identifies the presence or absence of a traffic light related to the traffic of the own vehicle and the lighting state with a small amount of calculation, even with a light that is difficult to obtain a feature point, using a single camera or the like. It is an object of the present invention to provide a method and apparatus for identifying a traffic light lighting state that can be used.

The traffic light lighting state identification method of the present invention is a traffic light lighting state identification method for identifying either or both of the presence and absence of a traffic light and a lighting state included in an image obtained by capturing the vicinity of the front of a moving object including an automobile,
Determining whether the traffic light included in the image is elliptical;
Obtaining a normal vector of the lamp from the major axis and minor axis of the elliptical lamp when the lamp is elliptical; and
Determining whether or not the lamp is related to the traffic of the own vehicle based on a predetermined reference value from the normal vector of the lamp.

The traffic light lighting state identification device according to the present invention is a traffic light lighting state identification device that captures the vicinity of the front of a moving object including an automobile and identifies either or both of the presence of a traffic signal and the lighting state included in the image. ,
Means for determining whether or not the traffic light included in the image is elliptical;
Means for obtaining a normal vector of the lamp from the major axis and minor axis of the elliptical lamp when the lamp is elliptical;
Means for determining whether or not the lamp is related to traffic of the own vehicle based on a predetermined reference value from the normal vector of the lamp.

  According to the traffic light lighting state identification method and apparatus of the present invention, the presence or absence of a traffic light related to the traffic of the own vehicle and the lighting state are identified by a single camera or the like even with a small amount of calculation and a light whose feature point is difficult to obtain. It becomes possible.

  Hereinafter, an embodiment of a traffic light lighting state identifying method and apparatus according to the present invention will be described with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the same element and the overlapping description is abbreviate | omitted.

  First, the functional configuration of the traffic light lighting state identification device according to an embodiment of the present invention will be described with reference to FIG. As shown in FIG. 1, the traffic light lighting state identification device includes a lamp detection unit 11, an ellipse determination unit 12, a normal vector calculation unit 13, and a lamp determination unit 14. Hereinafter, each component will be described.

  The lamp detection unit 11 detects lamp candidates based on image color, luminance information, and the like. At this time, if the image includes an object similar in color and brightness to the lamp, it can be detected as a lamp in the same manner as an actual lamp.

  The ellipse determination unit 12 determines whether the lamp is elliptical with respect to the lamp detected by the lamp detection unit 11. Here, the example of the process which determines whether a lamp is elliptical using FIG. 3 is shown. First, an image is binarized into a lighting part 12a and a non-lighting part 12b by providing threshold values for color and luminance information (FIG. 3A). Next, the width a in the X direction, the width b in the Y direction, and the center coordinate O of the lighting part 12a are obtained (FIG. 3B), and the X diameter a / 2 and the Y diameter b / 2 are centered on O. An ellipse 12c is drawn (FIG. 3C). Finally, the ratio of the area of the pixel (black part in FIG. 3D) belonging to the exclusive OR of the pixel included in the lighting part 12a and the pixel included in the ellipse 12c to the area of the pixel included in the ellipse 12c is as follows. If it is below a specific threshold, it is determined that the lamp is an ellipse, otherwise the lamp is not an ellipse.

The normal vector calculation unit 13 obtains the difference between the normal vector of the light and the traveling direction vector of the own vehicle from the X and Y diameters of the ellipse. Here, an example of a process for obtaining the normal vector P of the lamp from the X and Y diameters of the ellipse will be described with reference to FIG. In FIG. 4, the difference θ between the normal vector P of the lamp and the traveling direction of the host vehicle is obtained by cos ⁻¹ (a / b). Here, although the influence of the lamp height on the normal vector is ignored, as described in the process example of the lamp determination unit 14 below, in this embodiment, the vehicle and the lamp are There is no problem because the lights whose distance on the road surface is below a certain value are excluded from identification.

  The lamp determination unit 14 determines whether the lamp is related to the passage of the vehicle from the lamp normal vector obtained by the normal vector calculation unit 13 and the lamp diameter. Below, the example of the process which performs a light determination is shown.

  First, lights having a diameter of a light input to the light determination unit 14 that are equal to or greater than a specific threshold are excluded from the light candidates that the host vehicle should follow. This is because, when the distance between the light and the vehicle is too close, the effect of the light height component on the difference θ between the normal vector and the direction of travel of the vehicle becomes large, and the inclination of the light and the vehicle is on the road surface. This is because the difference between the angle projected on and θ becomes large. However, since it is necessary to determine the lighting state of the traffic light before the stop line at the intersection, it is only necessary to be able to determine the lighting in a situation where the distance between the traffic light and the company is greater than the road width. For example, when the road width is 10 meters, the height of the signal light is 4 meters, and the height of the camera installation position is 1 meter, the influence of the height direction on the difference θ is tan-1 (0.3), that is, about 16.6 degrees, which is a sufficiently small value.

  If the distance between the traffic signal and the vehicle can be generated from information such as GPS and car navigation system, the distance between the traffic signal and the vehicle is less than a certain distance from the candidate lights to be followed by the vehicle, It may be replaced with the process.

  Next, a light whose difference θ between the normal vector of the light input to the light determination unit 14 and the traveling direction vector of the own vehicle is equal to or greater than a specific threshold is excluded from the candidates for the light that the own vehicle should follow. .

  Finally, the lamp having the largest diameter in the image is selected as the lamp to be followed by the vehicle among the lights not excluded from the lamp candidates to be followed by the company.

  Next, operation | movement of the traffic light lighting state identification method in one Embodiment of this invention is demonstrated using FIG.

  First, an image near the front of the host vehicle is input from a camera or the like installed in the host vehicle (step S1).

  Next, based on information such as color and brightness, a light that is lit is detected from the image (step S2).

  At this time, a plurality of lights can be detected from the image.

  Next, it is determined whether or not the shape of the lamp is elliptical (step S3). Here, when it is determined that the lamp is not elliptical, the lamp is regarded as a false lamp erroneously detected from the color and luminance information.

  Next, a normal vector of the lamp is calculated from the major axis and minor axis of the elliptical lamp (step S4).

  Next, based on the normal vector of the light, it is determined whether the light is related to the traffic of the own vehicle (step S5). If so, the lighting state of the lamp is output (step S6). If the lamp is not elliptical, if it is not related to the traffic of the own vehicle, or if a lighting state is output, the process returns to step S3 for all the lamps, and step S3 and subsequent steps are repeated (loop 1).

  As described above, according to the above-described embodiment, it is possible to identify the light of the traffic signal related to the own vehicle from the image captured by one camera.

  Note that, by combining the above-described embodiment with a conventional lamp identification technique using stereo matching, a more highly available lamp identification technique can be obtained. For example, when two cameras mounted on the vehicle are operating correctly, the light is identified by stereo matching, and when one of the two cameras cannot obtain an image, Light identification according to the above embodiment can be performed from the image of the camera.

  Further, by combining the above-described embodiment with a conventional lighting identification method using stereo matching, a more accurate lighting identification method can be obtained. For example, the lighting identification according to the above embodiment is performed only on an object for which a normal vector cannot be obtained by stereo matching, such as a long-distance lighting, and stereo after the lighting approaches. Light identification by matching can be performed.

  In addition, the thing described by the term of the own vehicle in this invention is not necessarily restricted to a motor vehicle, What is necessary is just a moving means according to the lighting condition of a traffic light.

  In addition, the object described in the term “camera” in the present invention may be any means for obtaining an image, such as an image sensor.

  The traffic light lighting state identifying method and apparatus according to the present invention identify the presence or absence of a traffic light related to the traffic of the own vehicle and the lighting state with a small amount of calculation, even for a light whose feature points are difficult to obtain, with a single camera or the like. It is useful as an apparatus and method for controlling a moving body according to a traffic signal such as an automobile or a train. It can also be applied as a warning generation means for automobile drivers.

It is a block diagram which shows the flow of the process by one Embodiment of the traffic light lighting state identification device which concerns on this invention. It is a flowchart of the traffic light lighting state identification method which concerns on this invention. It is explanatory drawing of the process of an ellipse determination part. It is explanatory drawing for processing the detection of a normal line.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Light detection part 12 ... Ellipse determination part 13 ... Normal vector calculation part 14 ... Light determination part

Claims (2)

A signal lighting state identification method for identifying either or both of the presence and absence of a traffic signal and a lighting state included in an image obtained by photographing the vicinity of a moving object including an automobile,
Determining whether the traffic light included in the image is elliptical;
Obtaining a normal vector of the lamp from a major axis and a minor axis of the elliptical lamp when the lamp is elliptical; and
Determining whether or not the lamp is related to traffic of the vehicle based on a predetermined reference value from the normal vector of the lamp. A traffic light lighting state identification device for photographing the vicinity of the front of a moving body including an automobile and identifying either or both of the presence or absence of the traffic light included in the image and the lighting state,
Means for determining whether the light of the traffic light included in the image is elliptical;
Means for obtaining a normal vector of the lamp from the major axis and minor axis of the elliptical lamp when the lamp is elliptical;
A traffic light lighting state identification device comprising: means for determining whether or not the lamp is related to traffic of the own vehicle based on a predetermined reference value from the normal vector of the lamp.

Images