JP2020061052A - Traffic light determination device - Google Patents

Abstract

To provide a traffic light determination device capable of suppressing an erroneous detection of a traffic light.SOLUTION: A traffic light determination device comprises: a surrounding environment information determination unit that determines whether a surrounding environment is a bright environment or a dark environment; an image acquisition unit that acquires a first image that is an image when the surrounding environment is one of the bright environment and the dark environment, and a second image that is an image when it is the other environment; a specification unit that specifies a signal light candidate from the first image and the second image; a position calculation unit that calculates a position on a map of the signal light candidate specified by the specification unit; and a determination unit for determining that the signal light candidate is a signal light when a position on the map calculated by the position calculation unit of the signal light candidate specified from the first image is coincident with a position on the map calculated by the position calculation unit of the signal light candidate specified from the second image by the specification unit.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a traffic light determination device.

  Conventionally, as described in Patent Document 1, it is a technique for setting the position of a traffic light on a map, and when a vehicle enters an intersection, the color and shape of a light emitting object is detected from a captured image to detect the traffic light. There is known a technique of registering the determined information in a map database.

Japanese Patent Publication No. 2013-518298

  By the way, on the image taken to detect the traffic light, there is a problem that stationary objects of a color similar to the signal light color (red, yellow, blue) are reflected in the vicinity of the traffic light, and they are erroneously detected as the traffic light. Exists. Here, "a stationary object having a color similar to the signal light color" means, for example, in a bright environment (bright environment), the sky (blue of the blue sky or red of the sunset. It has a shape similar to the fire surface and may be erroneously detected.), Window glass that reflects the sky (blue sky or sunset) due to reflection, structures such as walls with a color similar to the signal light color, or the structure For example, a picture drawn on an object. In a dark environment (dark environment), neon lamps, street lights, etc. that emit light at night correspond. The above-mentioned conventional technology has room for improvement in suppressing such false detections.

  Then, this invention makes it a subject to provide the traffic light determination apparatus which can suppress the erroneous detection of a traffic light.

  The traffic light determination device according to the present invention is a surrounding environment information determining unit that determines whether the surrounding environment is a bright environment or a dark environment, and the surrounding environment is one of the bright environment and the dark environment. An image acquisition unit that acquires a first image that is an image of, and a second image that is an image in the case of the other, a specifying unit that specifies a traffic light candidate from the first image and the second image, and A position calculation unit that calculates a position on the map of the traffic light candidate specified by the specifying unit, a position on the map calculated by the position calculation unit of the traffic light candidate specified from the first image, and the specifying unit is the first A traffic light determination device comprising: a determination unit that determines that the traffic signal candidate is a traffic light when the position on the map calculated by the position calculation unit of the traffic light candidate identified from the two images matches.

  In this traffic light determination device, the fact that the traffic light emits light both in a bright environment and in a dark environment is used, and when the same traffic signal candidate is specified from the first image and the second image, the traffic signal candidate is identified as the traffic signal candidate. It is determined that Therefore, even if a stationary object that may be erroneously detected as a traffic signal is reflected in either the first image or the second image and is identified as a traffic signal, the stationary object is identified in the other image. Since it is not performed, the possibility of erroneously detecting the stationary object as a traffic light is reduced, and the detection accuracy of the traffic light can be improved.

  ADVANTAGE OF THE INVENTION According to this invention, the traffic light determination apparatus which can suppress the erroneous detection of a traffic light can be provided.

It is a block diagram which shows the structure of the traffic light determination apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows the signal shape used for the signal candidate position registration part to estimate the position of a signal. It is a flow chart which shows up to traffic signal candidate position registration among processings which a traffic signal judging device of Drawing 1 performs. It is a flow chart which shows up to a traffic signal fixed position registration among processings which a traffic signal judging device of Drawing 1 performs. It is a figure which shows an example of a 1st image. It is a figure which shows a mode that the traffic light candidate position was registered into the map database. It is a figure which shows an example of a 2nd image. It is a figure which shows a mode that the traffic light fixed position was registered into the map database.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the same or corresponding elements will be denoted by the same reference symbols, without redundant description.

[First Embodiment]
FIG. 1 is a block diagram showing the configuration of a traffic light determination device 100 according to the first embodiment. As shown in FIG. 1, the traffic light determination device 100 is mounted on a vehicle V such as an automobile. The traffic light determination device 100 includes an external sensor 1, a GPS [Global Positioning System] reception unit 2, an internal sensor 3, a map database 4, and an ECU [Electronic Control Unit] 10.

  The external sensor 1 is a detection device that detects an external situation that is peripheral information of the vehicle V. The external sensor 1 includes at least one of a camera, a radar [Radar], and a lidar [LIDAR: Laser Imaging Detection And Ranging]. The camera is an image pickup device that takes an image of the external situation of the vehicle V.

  The camera is provided, for example, on the back side of the windshield of the vehicle V. The camera transmits to the ECU 10 imaging information regarding the external situation of the vehicle V. The camera may be a monocular camera or a stereo camera.

  The radar detects an obstacle outside the vehicle V using radio waves (for example, millimeter waves). The radar detects an obstacle by transmitting a radio wave around the vehicle V and receiving a radio wave reflected by the obstacle. The radar transmits the detected obstacle information to the ECU 10.

  The rider uses light to detect an obstacle outside the vehicle V. The rider transmits light around the vehicle V, receives the light reflected by the obstacle, measures the distance to the reflection point, and detects the obstacle. The rider transmits the detected obstacle information to the ECU 10. Cameras, riders and radars do not necessarily have to be duplicated.

  The GPS receiving unit 2 measures the position of the vehicle V (for example, the latitude and longitude of the vehicle V) by receiving signals from three or more GPS satellites. The GPS receiver 2 transmits the measured position information of the vehicle V to the ECU 10. It should be noted that instead of the GPS receiving section 2, other means capable of specifying the latitude and longitude of the vehicle V may be used. Further, it is preferable to have a function of measuring the azimuth of the vehicle V in order to collate the measurement result of the sensor with the map information described later.

  The internal sensor 3 is a detection device that detects the traveling state of the vehicle V. The internal sensor 3 includes at least one of a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. The vehicle speed sensor is a detector that detects the speed of the vehicle V. As the vehicle speed sensor, for example, a wheel speed sensor that is provided for the wheels of the vehicle V or a drive shaft that rotates integrally with the wheels and that detects the rotational speed of the wheels is used. The vehicle speed sensor transmits the detected vehicle speed information (wheel speed information) to the ECU 10.

  The acceleration sensor is a detector that detects the acceleration of the vehicle V. The acceleration sensor includes, for example, a longitudinal acceleration sensor that detects the longitudinal acceleration of the vehicle V and a lateral acceleration sensor that detects the lateral acceleration of the vehicle V. The acceleration sensor transmits acceleration information of the vehicle V to the ECU 10, for example. The yaw rate sensor is a detector that detects a yaw rate (rotational angular velocity) around the vertical axis of the center of gravity of the vehicle V. As the yaw rate sensor, for example, a gyro sensor can be used. The yaw rate sensor transmits the detected yaw rate information of the vehicle V to the ECU 10.

The map database 4 is a database including map information. The map database is formed, for example, in an HDD [Hard Disk Drive] mounted on the vehicle. The map information includes, for example, road position information, road shape information (for example, curve, type of straight line portion, curvature of curve, etc.), and intersection and branch point position information. Further, it is preferable to include the output signal of the external sensor 1 in the map information in order to use the position information of the shielding structure such as a building or a wall and the SLAM (Simultaneous Localization And Mapping) technology. The map database may be stored in a computer of a facility such as an information processing center that can communicate with the vehicle V.
Further, the map database 4 may include legal speed information that is information about the legal speed of the road on which the vehicle travels and the road intersecting with the road.

  As shown in FIG. 1, the ECU 10 controls the automatic traveling of the vehicle V. The ECU 10 is an electronic control unit including a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like. In the ECU 10, various programs are executed by loading a program stored in the ROM into the RAM and executing it by the CPU. The ECU 10 may be composed of a plurality of electronic control units.

  The ECU 10 has a vehicle position recognition unit 11, a traffic light identification unit 12, an external environment information acquisition unit 13, a traffic signal candidate position registration unit 14, an external environment matching determination unit 15, a traffic signal matching determination unit 16, and a traffic signal fixed position registration unit 17. are doing. In the following description, for simplicity of description, a case where a camera is used as an example of the external sensor 1 and a captured image of the camera is used as the detection result of the external sensor 1 will be described. Even when a rider or a radar is used as the external sensor 1, the modes described below are similarly established.

  The vehicle position recognizing unit 11 is based on the position information of the vehicle V received by the GPS receiving unit 2 and the map information of the map database 4, and the position of the vehicle V on the map (hereinafter referred to as “vehicle position”) and azimuth information. Recognize. When the vehicle position of the vehicle V can be measured by a sensor installed outside the road or the like, the vehicle position recognition unit 11 may acquire the vehicle position from this sensor through communication.

  The traffic signal identifying unit 12 identifies a traffic signal existing in front of the host vehicle from a captured image obtained by capturing the front of the host vehicle. The traffic signal identification unit 12 identifies a traffic signal candidate included in the captured image from the captured information regarding the captured image in front of the vehicle captured by the camera of the external sensor 1. The traffic light identification unit 12 may identify the traffic light from one captured image, or may integrate the identification results from a plurality of captured images to perform the identification. Further, the traffic light identification unit 12 may scan the entire captured image to search for the traffic light, or may scan only the area of the captured image that is recognized to be at a predetermined height (for example, 5 m) or more and signal the traffic light. May be searched. Further, the traffic signal identifying unit 12 may identify the traffic signal constantly or at predetermined time intervals regardless of the positional relationship between the own vehicle and the traffic signal S, or based on the recognition result of the vehicle position recognizing unit 11, the intersection etc. The traffic signal may be specified when the predetermined area is reached.

  The traffic light identification unit 12 uses, for example, the color information of the captured image (for example, the color information may be obtained by using the brightness) and / or the shape of the image (for example, the Hough transform is used) to generate a circular illuminant. The lighting surface of the traffic light may be specified by specifying. Further, as an aspect in which the traffic light is recognized by template matching using teacher data of an image including the shape of the lighting surfaces of the traffic light and the shape of the traffic light housing (pole, etc.) using the identification result Good.

  The external environment information acquisition unit 13 specifies whether the external environment of the vehicle V is a bright environment or a dark environment. The determination as to whether the external environment state is a bright environment or a dark environment is performed based on the camera identification result of the external sensor 1 or time information. When the camera identification result of the external sensor 1 is used, when the average brightness in the captured image is equal to or higher than the predetermined brightness, it is possible to determine that the environment is bright. On the contrary, when the average brightness in the captured image is less than the predetermined brightness, it is possible to determine that the environment is a dark environment. On the other hand, when the time information is used, it is determined that the environment is bright when the time is within a predetermined time zone (6 am to 6 pm as an example), and the environment is dark when the time is outside the predetermined time zone. It is possible to adopt a mode in which

  At this time, the predetermined brightness and the predetermined time period are values that make it less likely that a stationary object, which causes false detection in one of the bright and dark environments, will be falsely detected in the other environment. Can be determined experimentally and experimentally. For example, a window glass partially reflecting a blue sky or a sunset causes an erroneous detection in a bright environment, but does not easily cause an erroneous detection in a dark environment. This is because there is a high possibility that the sun is not appearing in the dark environment, that is, there is a low possibility that the window glass reflects sunlight. Therefore, the predetermined brightness and the predetermined time zone described above are empirically set so that the brightness or the time zone when the possibility of the false detection becomes lower than the threshold value can be set as the dark environment.

  Further, as an aspect in which an in-vehicle illuminance sensor (not shown) is used, it is determined that the environment is bright when the illuminance of the external environment is equal to or higher than a predetermined threshold, and that the environment is dark when the illuminance is less than the predetermined threshold. Good.

  When the traffic light specifying unit 12 specifies the traffic light in either the first image or the second image, the traffic light candidate position registration unit 14 uses the position of the traffic light (coordinate position on the map) as the traffic light candidate position in the map database. 4 (hereinafter, sometimes referred to as “temporary registration”). In converting the position on the image into the coordinate position on the map (calculating the coordinate position on the map from the position on the image), a known position is recognized based on the vehicle position and the direction information recognized by the vehicle position recognition unit 11. By the method, the distance and direction between the vehicle and the object on the image are calculated, and the coordinate position is determined. The "position of the traffic light" at this time can be a position of the housing of the traffic light, for example, a position at the center in the width direction of the housing of the traffic light. When the traffic light identifying unit 12 identifies a circular light emitting object corresponding to the traffic light surface as a traffic light, it may be the coordinate position on the map of the light emitting object. It may also be the position of the pole of the traffic light. However, in a dark environment, it is possible that only the lighting surface of the traffic light can be specified, and the pole and housing of the traffic light cannot be specified. In that case, the position of the pole or the housing estimated from the identified traffic light surface using the traffic light template as shown in FIG. 2 may be set as the “traffic light position”. For example, when the traffic light identification unit 12 identifies the red traffic light fire surface SR, it can be estimated that the pole P is present in a portion that has moved to the left by the predetermined distance LR, and the position of the pole P can be set as the “traffic light position”. In addition, it is estimated that the end of the housing H is present in the portion that has moved to the right by the predetermined distance LH from the red traffic light surface SR, and the width of the housing H (here, the sum of the predetermined distance LR and the predetermined distance LH) is estimated. The position of the center of the width may be the “position of the traffic light”. The above method is similarly applied by using the predetermined distance LY when the traffic light identification unit 12 identifies the yellow traffic light fire surface SY and by using the predetermined distance LG when identifying the green traffic light fire surface SG. You can

  The external environment matching determination unit 15 determines whether or not the external environment at the time of acquiring the first image (whether it is a bright environment or dark environment) and the external environment at the time of acquiring the second image match. .

  The traffic light matching determination unit 16 determines whether or not the traffic light specified by the traffic light specifying unit 12 in the second image matches the traffic light specified in the first image. Specifically, based on the position of the own vehicle recognized by the vehicle position recognition unit 11 and the signal candidate position registered in the map database 4 by the signal candidate position registration unit 14, the map of the signal specified in the second image. A mode may be used in which it is determined whether or not the upper position matches the traffic signal candidate position registered in the map database 4. However, in the present specification, "matching" does not mean, for example, that the values of the coordinates of the two completely match, but "matching" when the two are within a predetermined distance. And

  When the traffic light matching determination unit 15 determines that the traffic light identified by the second image matches the traffic light identified by the first image, the traffic light determined position registration unit 17 sets the position of the traffic light as a traffic light determined position on the map. Register in database 4. Specifically, the signal candidate position registered by the signal candidate position registration unit 14 is updated to the signal confirmed position.

  Next, the processing executed by the traffic light determination device 100 will be specifically described with reference to the flowcharts of FIGS. 3 and 4. FIG. 3 is a flow chart until the traffic signal candidate position is registered, and FIG. 4 is a flow chart until the traffic signal fixed position is registered. In the following description, a mode in which the first image is acquired in a dark environment and the second image is acquired in a bright environment will be described in detail. On the contrary, the first image is acquired in a bright environment and the second image is dark. A mode in which the information is acquired in the environment may be used.

  First, the process of registering the signal candidate position will be described with reference to FIG. This control may be always performed, or may be performed when the vehicle position recognition unit 11 detects that the vehicle V approaches or enters an intersection.

  In S11, the external environment information acquisition unit 13 determines whether the external environment of the vehicle V is a bright environment or a dark environment. As described above, the determination method may be based on the brightness in the image detected by the camera of the external sensor 1 or based on the time information. Next, in S12, the external sensor 1 acquires the first image A.

  Next, in S13, the traffic light identification unit 12 identifies the traffic light candidate objects (SA1, SA2) shown in the first image A (see FIG. 5). Here, it is assumed that the signal candidate object SA1 is actually a lighting surface of the signal, and the signal candidate object SA2 is actually a neon lamp that lights up at night, for example. As described above, since the traffic light identifying unit 12 can be configured to specify the light emitting object, the traffic light candidate objects (SA1, SA2) may be specified as a circular light emitting object as shown in FIG. At this stage, the traffic light candidate SA2 is not a traffic light, but the traffic light identifying unit 12 erroneously identifies the traffic light candidate SA2 as a traffic light. Next, in S14, the traffic light candidate position registration unit 14 registers the traffic light candidate position corresponding to the position on the first image of the traffic light identified by the traffic light identification unit 12 in the map database 4 (see FIG. 6). At the same time, the determination result, which is acquired by the external environment information acquisition unit 13 in S11, whether the environment is bright or dark is registered in the map database 4 in association with the signal candidate position. Specifically, the signal candidate positions corresponding to the signal candidate object SA1 and the signal candidate object SA2 are registered as C1 and C2, respectively.

  Next, the process from capturing the second image to registering the traffic signal fixed position will be described with reference to FIG. First, in S21, the external environment information acquisition unit 13 determines whether the external environment of the vehicle V is a bright environment or a dark environment. Next, in S22, the external sensor 1 captures the second image B (see FIG. 7). Next, in S23, the external environment matching determination unit 15 determines whether the external environment (whether the light environment or the dark environment) at the time of acquiring the second image B, which is performed by the external environment information acquisition unit 13 in S21, It is determined whether or not it is different from the external environment when one image A is acquired. If it is determined in S23 that they are not different, the process returns to S21. If it is determined in S23 that they are different, the process proceeds to S24. In S24, the traffic signal identification unit 12 identifies the traffic signal candidate objects (SB1, SB2) shown in the second image B (see FIG. 7). Here, actually, the signal candidate object SB1 is a lighting surface of the signal, and the signal candidate object SB2 is actually, for example, a window glass reflecting a sunset. At this stage, although the traffic signal candidate object SB2 is not a traffic signal, the traffic signal specifying unit 12 erroneously specifies the traffic signal candidate object SB2 as a traffic signal. Here, the determination result obtained by the external environment information acquisition unit 13 in S21, which is the bright environment or the dark environment, is associated with the position on the map of the signal candidate object (SB1, SB2) and is linked to the map. Register in database 4.

  In S25, the traffic light matching determination unit 16 determines whether or not the position on the map of the traffic light identified by the traffic light identification unit 12 in the second image B matches the traffic light candidate position registered by the traffic light candidate position registration unit 14. to decide. Specifically, signal candidate positions corresponding to the signal candidate object SB1 and the signal candidate object SB2 are respectively D1 and D2, and these are signal candidate positions C1 and C2 temporarily registered by the signal candidate position registration unit 14 in S14. It is determined whether they match (see FIG. 8). In FIG. 8, it is determined that the signal candidate positions C1 and D1 match. On the other hand, it is determined that the signal candidate position C2 does not match with either D1 or D2, and the signal candidate position D2 does not match with either C1 or C2. If it is determined in S25 that they match, the process proceeds to S26. If it is determined in S25 that they do not match, the process proceeds to S27.

  In S26, the traffic signal final position registration unit 16 updates the temporarily registered traffic signal candidate position and registers it in the map database 4 as the traffic signal final position. Specifically, in FIG. 8, the signal candidate position C1 is updated to the signal confirmed position. On the other hand, in S27, it is determined that the signal candidate position registered by the signal candidate position registration unit 14 is the one in which the false detection target is erroneously registered and is not the signal, and the signal candidate position is deleted from the map database 4. . Specifically, in FIG. 8, it is determined that the signal candidate positions C2 and D2 are erroneously registered as erroneously detected objects, and are deleted from the map database 4.

  As described above, in the traffic light determination device 100 of the present embodiment, the fact that the traffic light emits light in both bright and dark environments is used, and when a traffic light candidate that matches the first image and the second image is specified, The identified traffic signal candidate is determined to be a traffic signal. Therefore, even if a stationary object that may be erroneously detected as a traffic signal is reflected in either the first image or the second image, the stationary object is not specified in the other image, so the stationary object is not identified. The risk of erroneous detection as a traffic signal is reduced, and the detection accuracy of the traffic signal can be improved.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments and can be implemented in various forms.

  In the above-described embodiment, the traffic light identification unit 12 does not need to use the same identification method when the external environment is a bright environment and when the external environment is a dark environment. In this case, an appropriate identification and recognition method may be applied according to the external environment, such as pattern recognition of a traffic light shape.

  In the above embodiment, the external environment information acquisition unit 13 may use weather information acquired using a navigation system or the like (not shown). For example, the weather information may be a bright environment when the weather information is fine or clear, and a dark environment when the weather information is cloudy or rainy. Alternatively, the cloud amount information at the current position may be used as the weather information, and a bright environment may be set when the cloud amount is less than the predetermined threshold value and a dark environment may be set when the cloud amount is more than the predetermined threshold value.

  In the above-described embodiment, when the traffic light identification unit 12 identifies the traffic light from the second image B, the traffic light may be searched for the traffic light by scanning the entire captured image. It is also possible to scan only the area set based on the traffic signal candidate target obtained from the above to search for the traffic signal. In other words, when identifying the traffic light from the second image B, a mode of confirming whether or not the traffic light candidate object and the traffic light candidate position obtained from the first image A are recognized as the traffic light fixed position. can do.

  In the flowchart (FIGS. 3 and 4) of the above embodiment, the acquisition of the external environment information by the external environment information acquisition unit 13 (S11 and S21) is the acquisition of the first image A or the second image B (S12 and S22). Although it is performed in advance, the order is not limited to this and may be performed after the acquisition of the first image A or the second image B. That is, the external environment information acquisition unit 13 determines the external environment based on the average brightness of the first image A or the second image B (described above) after the acquisition of the first image A or the second image B. It may be determined whether it is the environment or the dark environment.

  In the above embodiment, the external environment information acquisition unit 13 may use weather information acquired using a navigation system or the like (not shown). For example, the weather information may be a bright environment when the weather information is fine or clear, and a dark environment when the weather information is cloudy or rainy. Alternatively, the cloud amount information at the current position may be used as the weather information, and a bright environment may be set when the cloud amount is less than the predetermined threshold value and a dark environment may be set when the cloud amount is more than the predetermined threshold value.

  In the above embodiment, the map database 4 in the traffic light determination device 100 may be built in the vehicle V, or may be provided in a data center or the like outside the vehicle V. In that case, the vehicle V is provided with a communication device for communicating with the data center, and the output of the ECU 10 is transmitted to the map database 4 via the communication device. When the map database 4 is provided in a data center outside the vehicle V, the accuracy of the map database 4 can be improved by updating the map database 4 not only by using the vehicle V but also by using the signal identification result of another vehicle. In particular, the accuracy of the traffic signal position can be improved.

  In the above-described embodiment, some of the functions of the ECU 10, that is, the vehicle position recognition unit 11, the traffic light identification unit 12, the external environment information acquisition unit 13, the traffic signal candidate position registration unit 14, the external environment matching determination unit 15, and the traffic light matching determination. Part of the unit 16 and the traffic light fixed position registration unit 17 may be executed by a computer in a facility such as an information processing center that can communicate with the vehicle V.

Reference numeral 13 ... External environment information acquisition unit (peripheral environment information determination unit), 1 ... External sensor (image acquisition unit), 12 ... Traffic signal identification unit (identification unit), 14 ... Traffic signal candidate position registration unit (position calculation unit), 16 ... Traffic light coincidence determination unit (determination unit).

Claims (1)

A surrounding environment information determination unit that determines whether the surrounding environment is a light environment or a dark environment,
An image acquisition unit that acquires a first image that is an image when the surrounding environment is one of the bright environment and the dark environment, and a second image that is an image when the other environment,
A specifying unit that specifies a signal candidate from the first image and the second image,
A position calculation unit that calculates a position on the map of the traffic light candidate identified by the identification unit;
A position on the map calculated by the position calculation unit of the traffic light candidate specified from the first image, and a position on the map calculated by the position calculation unit of the traffic light candidate specified from the second image by the specifying unit, And a determination unit that determines that the candidate traffic light is a traffic light, the traffic light determination device.

Images