JP2019096137A - Signal apparatus recognition device - Google Patents

Abstract

To provide a signal apparatus recognition device with which it is possible to appropriately recognize the result of signal apparatus lighting state recognition by a driver that is inputted, and determine the lighting state of a signal apparatus.SOLUTION: A signal apparatus recognition device 100 recognizes the lighting state of a signal apparatus ahead of a vehicle on the basis of the captured image of a camera 2 of the vehicle. The signal apparatus recognition device comprises: a confirmation necessity determination unit 12 for determining whether or not the driver of the vehicle needs to confirm the lighting state of the signal apparatus on the basis of the recognition result of signal apparatus lighting state; a notification unit 13 for notifying the driver to the effect that the lighting state of the signal apparatus be confirmed, when it is determined by the confirmation necessity determination unit that the lighting state of the signal apparatus be confirmed by the driver; a driver operation recognition unit 14 for recognizing the brake and accelerator operations of the driver in response to the notification; and a lighting state determination unit 15 for determining the lighting state of the signal apparatus on the basis of recognition result of the driver operation recognition unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a traffic signal recognition apparatus.

  Conventionally, U.S. Pat. No. 876,1991 is known as a technical literature related to a traffic signal recognition apparatus. This document describes performing automatic driving of the vehicle based on the recognition result of the traffic light state. Further, Japanese Patent Application Laid-Open No. 2017-041233 describes that when an unclear object is identified in the observation result of the environment of the vehicle, the driver is asked for a response by voice or gesture.

U.S. Patent No. 8761991 Unexamined-Japanese-Patent No. 2017-041233

  In recognition of a traffic light by a camera of a vehicle, there are cases where the lighting state of the traffic light can not be recognized with sufficient accuracy due to occlusion (shielding) or the like due to a leading vehicle or a branch of a tree. As one solution in such a case, it is conceivable to have the driver confirm the lighting state of the traffic light and to input the confirmation result. However, in the above-mentioned Japanese Patent Application Laid-Open No. 2017-041233, no attention is paid to the signal recognition, and there is room for improvement in the input of the confirmation result by the driver.

  Therefore, in the present technical field, it is desired to provide a traffic signal recognition apparatus capable of appropriately recognizing the input of the confirmation result of the lighting state of the traffic light by the driver and determining the lighting state of the traffic light.

  In order to solve the above problems, one aspect of the present invention is a traffic signal recognition apparatus that recognizes the lighting state of a traffic signal in front of a vehicle based on a captured image of a camera of the vehicle. A confirmation necessity determination unit that determines whether it is necessary for the driver of the vehicle to confirm the lighting condition of the traffic light, and the confirmation necessity determination unit determine that the lighting condition of the traffic light is necessary for the driver Based on the recognition result of the driver operation recognition unit that recognizes the driver's brake operation and the accelerator operation in response to the notification. And a lighting state determination unit that determines the lighting state of the traffic light.

  As described above, according to the traffic signal recognition apparatus according to one aspect of the present invention, the lighting status of the traffic signal can be determined by appropriately recognizing the input of the confirmation result of the lighting status of the traffic signal by the driver.

It is a block diagram showing a signal recognition device concerning one embodiment. It is a flowchart which shows the signal apparatus judgment processing at the time of recognizing the lighting condition of a signal apparatus as a green signal. It is a flowchart which shows the signal apparatus judgment processing at the time of recognizing the lighting condition of a signal apparatus as a green signal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The traffic signal recognition apparatus 100 shown in FIG. 1 is a device that is mounted on a vehicle such as a passenger car and recognizes the lighting state of a traffic signal in front of the vehicle. The lighting state of the traffic light includes a green light (passing permission state), a yellow light (transition state), and a red signal (passing prohibited state).

[Configuration of traffic signal recognition apparatus]
Hereinafter, the configuration of the traffic signal recognition apparatus 100 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the traffic signal recognition apparatus 100 includes a traffic signal recognition ECU (Electronic Control Unit) 10. The traffic signal recognition ECU 10 is an electronic control unit having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a timer, and the like. In the traffic signal recognition ECU 10, for example, a program stored in the ROM is loaded into the RAM, and the CPU loaded the program into the RAM to realize various functions. The traffic signal recognition ECU 10 may be configured of a plurality of electronic units.

  The traffic signal recognition ECU 10 is connected to the GPS reception unit 1, the camera 2, the radar sensor 3, the map database 4, the operation detection unit 5, the HMI (Human Machine Interface) 6, and the autonomous driving ECU 50.

  The GPS receiving unit 1 measures the position of the vehicle on the map (for example, the latitude and longitude of the vehicle) by receiving signals from three or more GPS satellites. The GPS receiver 1 transmits the measured position information of the vehicle to the signal recognition ECU 10.

  The camera 2 is an imaging device for imaging the front of a vehicle. The camera 2 is provided on the back of the windshield of the vehicle. The camera 2 transmits a captured image in front of the vehicle to the signal recognition ECU 10. The camera 2 may be a monocular camera or a stereo camera.

  The radar sensor 3 is a detection device that detects an obstacle around the vehicle using radio waves (for example, millimeter waves) or light. The radar sensor 3 includes, for example, a millimeter wave radar or a rider (LIDAR: Light Detection and Ranging). The radar sensor 3 transmits radio waves or light around the vehicle, and detects an obstacle by receiving the radio waves or light reflected by the obstacle. The radar sensor transmits the detected obstacle information to the signal recognition ECU 10. The obstacles include moving obstacles such as pedestrians, bicycles, and other vehicles, as well as fixed obstacles such as guardrails and buildings.

  The map database 4 is a database that stores map information. The map database 4 is formed, for example, in an HDD [Hard Disk Drive] mounted on a vehicle. The map information includes road position information, road shape information (for example, curves, types of straight portions, curvature of curves, etc.), position information of intersections and junctions, position information of traffic lights, and the like.

  The map database 4 may store, in association with the map information, a traffic light shielded area set in advance as an area where the occlusion of the traffic light occurs. In addition, the map database 4 may store a traffic signal recognition degradation area, which is preset as a traffic signal area causing recognition degradation due to backlighting or low brightness, in association with the map information. The map database 4 may store information on difficult to recognize whether or not the lighting state of the traffic light itself is difficult to recognize, not the area, in association with the positional information on the traffic light. The map database 4 may be stored in a server that can communicate with the vehicle.

  The operation detection unit 5 detects a driving operation of the vehicle by the driver. The operation detection unit 5 includes, for example, an accelerator pedal sensor that detects the depression amount of the accelerator pedal of the vehicle by the driver, and a brake pedal sensor that detects the depression amount of the brake pedal of the vehicle by the driver. The operation detection unit 5 transmits the detection result of the driving operation by the driver to the signal recognition ECU 10.

  The HMI 6 is an interface for inputting and outputting information between the traffic signal recognition apparatus 100 and the driver. The HMI 6 includes a display, a speaker, and the like. The HMI 6 performs an image output of a display and an audio output from a speaker in response to a control signal from the traffic signal recognition ECU 10. The display may include a head-up display. The HMI 6 may have an operation unit such as a touch panel for the driver to input an operation, and a voice recognition device that recognizes the voice of the driver. The HMI 6 may be connected to a vibration actuator that vibrates the steering wheel of the vehicle.

  The autonomous driving ECU 50 is an electronic control unit that is mounted on the vehicle and performs the autonomous driving of the vehicle. The automatic driving is a vehicle control that causes the vehicle to automatically travel without the driver performing a driving operation. The autonomous driving ECU 50 may be configured of a plurality of electronic units. A part of the functions of the autonomous driving ECU 50 may be executed by a server that can communicate with the vehicle.

  The autonomous driving ECU 50 is preset based on, for example, the position on the map of the vehicle measured by the GPS reception unit 1, the map information of the map database 4, the surrounding environment of the vehicle, and the vehicle state (vehicle speed, yaw rate, etc.) Generate a travel plan along the target route. The target route may be set manually by an occupant of the vehicle, or automatically set by a known navigation system or an autonomous driving ECU 50.

  The autonomous driving ECU 50 executes the autonomous driving in accordance with the travel plan. The autonomous driving ECU 50 executes the autonomous driving by transmitting a control signal to an actuator (engine actuator, steering actuator, brake actuator, etc.) of the vehicle. The autonomous driving ECU 50 can generate a travel plan and execute the autonomous driving by a known method.

  Next, the functional configuration of the traffic signal recognition ECU 10 will be described. The traffic signal recognition ECU 10 includes a traffic signal recognition unit 11, a confirmation necessity determination unit 12, a notification unit 13, a driver operation recognition unit 14, a lighting state determination unit 15, and a learning unit 16. Note that part of the functions of the signal recognition ECU 10 described below may be executed by a server that can communicate with the vehicle.

  The traffic signal recognition unit 11 recognizes the lighting state of the traffic signal in front of the vehicle based on the captured image of the camera 2 of the vehicle. The traffic signal recognition unit 11 can recognize whether the lighting state of the traffic light is a blue light, a yellow light, or a red light, for example, by a known image recognition process. The traffic signal recognition unit 11 may recognize the lighting state of the traffic signal by an algorithm combining the captured image of the camera 2 and the detection result of the radar sensor 3.

  The confirmation necessity determination unit 12 determines whether it is necessary for the driver of the vehicle to confirm the lighting condition of the traffic light, based on the recognition result of the lighting condition of the traffic light recognized by the traffic light recognition unit 11. If it is not possible to recognize the lighting condition of the traffic light with sufficient accuracy (certainty factor) due to occlusion (shielding) of the traffic light due to a leading vehicle or a branch of a tree, back light, rain, etc. It is determined that confirmation of the lighting state is necessary.

  Specifically, based on the recognition result of the lighting state of the traffic light recognized by the traffic light recognition unit 11, the confirmation necessity determination unit 12 recognizes the occlusion of the traffic light by the preceding vehicle or the like. The confirmation necessity determination unit 12 determines that the driver needs to check the lighting state of the traffic light, for example, when the shielded portion of the traffic light is equal to or more than a certain percentage. The confirmation necessity determination unit 12 may further use the detection result (obstacle information) of the radar sensor 3 to accurately recognize the occlusion of the traffic light.

  In addition, based on the position on the map of the vehicle measured by the GPS reception unit 1 and the map information (traffic light shielding area) of the map database 4, the confirmation necessity determination unit 12 recognizes that the vehicle is located in the traffic light shielding area. In this case, it may be determined that the driver needs to check the lighting state of the traffic light. SLAM (Simultaneous Localization and Mapping) may be used to recognize the position of the vehicle on the map.

  Confirmation necessity determination unit 12 is preset based on the position of the vehicle measured by GPS reception unit 1 on the map, the map information of map database 4 (the signal recognition degradation area), and the timer of signal recognition ECU 10. When it is recognized that the vehicle is located in the traffic light recognition lower area during the back light generation time zone, it may be determined that the driver needs to confirm the lighting state of the traffic light. The confirmation necessity determination unit 12 may determine that it is necessary for the driver to confirm the lighting state of the traffic signal, when the occurrence of backlight is recognized from the information of the luminance in the captured image of the camera 2.

  The confirmation necessity determination unit 12 is based on the position of the vehicle measured by the GPS reception unit 1 on the map and the map information (recognition difficult signal information) of the map database 4 and the traffic signal in front of the vehicle is a recognition difficult signal When it is recognized that there is, it may be determined that the driver needs to check the lighting state of the traffic light.

  If the confirmation necessity determination unit 12 recognizes that the wiper is operating based on the detection result of the wiper detection unit that detects the operation of the wiper of the windshield of the vehicle, the driver is notified that It may be determined that confirmation of the lighting state is necessary. In addition, even if the confirmation necessity determination unit 12 acquires weather information by wireless communication and recognizes that it is bad weather such as rainy weather, it is determined that it is necessary for the driver to confirm the lighting condition of the traffic light. Good.

  The confirmation necessity determination unit 12 may determine whether it is necessary for the driver to confirm the lighting state of the traffic signal by comprehensively using the various types of information described above. The confirmation necessity determination unit 12 may perform the above determination using an algorithm such as deep learning, SVM (Support Vector Machine), or a decision tree.

  The notification unit 13 notifies the driver of the confirmation of the lighting state of the traffic light when the confirmation necessity determination unit 12 determines that the lighting of the lighting device of the traffic light needs to be confirmed. The “notice of confirmation of the lighting state of the traffic light” is a notification for confirming to the driver the lighting state of the traffic light in front of the vehicle and requesting an operation according to the confirmation result. The notification unit 13 asks the driver for a brake operation or an accelerator operation as an operation according to the confirmation result. The brake operation and the accelerator operation will be described later.

  The notification unit 13 notifies the driver of at least one of the image display by the display and the audio output by the speaker via the HMI 6. In addition, the notification unit 13 may further perform notification by vibration (tactile sense). The notification unit 13 can vibrate the steering wheel of the vehicle by the vibration actuator of the HMI 6. As an example, the notification unit 13 can request confirmation of the lighting state of the traffic light by saying that “the certainty of recognition of the traffic light is low” by audio output from the speaker. Thereafter, the notification unit 13 asks the driver for an operation according to the confirmation result of the lighting state of the traffic light. In addition, when the driver already understands the operation according to the confirmation result, it is not essential to perform the notification for the operation in addition to the notification for the confirmation.

  When the notification by the notification unit 13 is performed, the driver operation recognition unit 14 recognizes the driver's brake operation and accelerator operation on the notification based on the detection result of the operation detection unit 5. The brake operation is, for example, an operation of depressing the brake pedal to the brake operation threshold or more. The brake operation threshold is a preset threshold. The brake operation threshold may be set to a value smaller than a brake start threshold at which a vehicle brake is started by depression of the brake pedal.

  The accelerator operation is, for example, an operation of depressing the accelerator pedal to the accelerator operation threshold or more. The accelerator operation threshold is a preset threshold. The accelerator operation threshold may be smaller than an accelerator start threshold at which the accelerator of the vehicle is started by depressing the accelerator pedal.

  When the confirmation necessity determination unit 12 determines that the driver needs to confirm the lighting condition of the traffic light, the lighting condition determination unit 15 determines that the traffic signal ahead of the vehicle is detected based on the recognition result of the driver operation recognition unit 14. Determine the lighting status of.

  Specifically, when the lighting condition judging unit 11 recognizes the lighting condition of the traffic light as a green light, the lighting condition judging unit 15 judges by the confirmation necessity determination unit 12 that the driver needs confirmation of the lighting condition of the traffic light When the driver's operation recognition unit 14 recognizes the driver's brake operation, the lighting state of the traffic light is determined to be a red light (the lighting state of the traffic light is corrected to a red light). When the driver's accelerator operation is recognized, the lighting state determination unit 15 determines that the lighting state of the traffic light is a green light (maintains the recognition result of the lighting state of the traffic light).

  The lighting state determination unit 15 may determine that the lighting state of the traffic light is a green light when neither the driver's brake operation nor the accelerator operation is recognized even if a predetermined time has elapsed since the notification. Alternatively, the lighting state determination unit 15 may correct the lighting state of the traffic light to a red light from the viewpoint of risk avoidance.

  When the lighting condition judging unit 11 recognizes the lighting condition of the traffic light as a red light, the lighting condition judging unit 15 judges that the driver needs confirmation of the lighting condition of the traffic light by the confirmation necessity determination unit 12 and the driver When the driver's accelerator operation is recognized by the operation recognition unit 14, the lighting state of the traffic light is determined to be a green light (the lighting state of the traffic light is corrected to a green light). When the driver's brake operation is recognized, the lighting state determination unit 15 determines that the lighting state of the traffic light is a red signal (maintains the recognition result of the lighting state of the traffic light).

  The lighting state determination unit 15 sets the lighting state of the traffic light as a red light without correcting the recognition result when neither the driver's brake operation nor the accelerator operation is recognized even if a predetermined time has elapsed since the notification. You may judge.

  In addition, the lighting condition determination unit 15 determines that the confirmation of the lighting condition of the traffic light is necessary for the driver by the confirmation necessity determination unit 12 when the lighting condition of the traffic light is recognized as a yellow light by the traffic light recognition unit 11 Sometimes, the same determination may be made as when the lighting state of the traffic light is recognized as a red light.

  The lighting state determination unit 15 transmits the determination result of the lighting state of the traffic signal to the automatic driving ECU 50. The autonomous driving ECU 50 generates or corrects a travel plan of the vehicle based on the determination result of the lighting state of the traffic light. When it is determined that the lighting state of the traffic light is a red light, the autonomous driving ECU 50 generates a travel plan of the vehicle so as to stop at the stop line. When it is determined that the lighting state of the traffic light is a green light, the autonomous driving ECU 50 generates a travel plan of the vehicle so as to pass the stop line.

  When the recognition result of the lighting state of the traffic light is different from the judgment result of the lighting state of the traffic light by the lighting state judgment unit 15, the learning unit 16 associates the recognition result with the judgment result and collects learning data. Do. The learning unit 16 collects the recognition result and the judgment result as learning data when the recognition deterioration of the traffic signal occurs. The learning unit 16 can use the determination result of the lighting state of the traffic signal by the operation of the driver as the correct value.

  When the recognition result and the determination result are different, the learning unit 16 may learn a traffic signal in front of the vehicle as a traffic signal whose recognition of the lighting state is difficult. Alternatively, the driver's judgment result may be learned as the correct answer. For example, when the driver depresses the accelerator although the recognition result is a red light, the performance of the learning unit can be improved by learning the situation at that time as a green light. Also, image data when recognition is difficult may be stored in the database. When the recognition result and the determination result are different, the learning unit 16 may learn the position on the map of the vehicle when the imaging by the camera 2 is performed as a position where it is difficult to recognize the lighting state of the traffic light. The learning unit 16 stores the learning result in the map database 4 or an arbitrary database.

[Traffic signal determination processing of traffic signal recognition device]
Next, the signal judgment processing by the signal recognition device 100 according to the present embodiment will be described. Here, a case where the lighting state of the traffic light is recognized as a green light and a case where a lighting state of the traffic light is recognized as a red light will be described.

<Traffic signal determination processing when the lighting condition of the traffic light is recognized as a green light>
FIG. 2 is a flowchart showing a signal judging process when the lighting state of the signal is recognized as a green light. The process of the flowchart shown in FIG. 2 is executed when the lighting condition of the traffic light is recognized as a green light by the traffic light recognition unit 11.

  As shown in FIG. 2, the signal recognition ECU 10 determines whether the driver of the vehicle needs to check the lighting state of the traffic light by the confirmation necessity determination unit 12 as S10. The confirmation necessity determination unit 12 performs the above determination based on the recognition result of the lighting state of the traffic light recognized by the traffic light recognition unit 11. When it is not determined that the driver needs to check the lighting state of the traffic light (S10: NO), the traffic light recognition ECU 10 ends the current processing. When it is determined that the driver needs to check the lighting state of the traffic light (S10: YES), the traffic light recognition ECU 10 proceeds to S12.

  In S12, the traffic signal recognition ECU 10 notifies the driver of the confirmation of the lighting state of the traffic signal by the notification unit 13. The notification unit 13 notifies the driver of at least one of the image display by the display and the audio output by the speaker via the HMI 6.

  In S14, the signal recognition ECU 10 determines whether the driver operation recognition unit 14 has recognized the driver's brake operation. The driver operation recognition unit 14 recognizes the brake operation from the detection result of the brake pedal sensor provided on the brake pedal of the vehicle.

  When it is not determined that the driver's brake operation has been recognized (S14: NO), the signal recognition ECU 10 proceeds to S16. If it is not determined that the driver's brake operation has been recognized, the driver's accelerator operation is used instead of the case where the driver's brake operation can not be recognized even if a predetermined time has elapsed since the notification, and instead of the brake operation. The case of recognition is included. When it is determined that the driver's brake operation is recognized (S14: YES), the signal recognition ECU 10 proceeds to S18.

  In S16, the traffic light recognition ECU 10 determines the lighting state of the traffic light as a green light by the lighting state determination unit 15 (maintains the recognition result of the lighting state of the traffic light). Thereafter, the signal recognition ECU 10 ends the current process.

  In S18, the signal recognition ECU 10 determines that the lighting state of the traffic light is a red signal by the lighting state determination unit 15 (corrects the lighting state of the traffic light to a red signal).

  In S20, the signal recognition ECU 10 performs learning by associating the recognition result of the lighting state of the signal and the determination result by the learning unit 16. Thereafter, the signal recognition ECU 10 ends the current process.

  Note that the process of learning in S20 is not essential. Further, in S14, when the driver's brake operation can not be recognized even if a predetermined time has elapsed since the notification, the lighting state of the traffic signal may be corrected to a red light from the viewpoint of risk avoidance.

<Traffic signal determination processing when the lighting state of the traffic light is recognized as a red light>
FIG. 3 is a flowchart showing a signal judging process when the lighting state of the signal is recognized as a red signal. The process of the flowchart shown in FIG. 3 is executed when the lighting condition of the traffic light is recognized as a red light by the traffic light recognition unit 11.

  As shown in FIG. 3, the signal recognition ECU 10 determines whether or not the driver of the vehicle needs to check the lighting state of the traffic light by the confirmation / non-necessity determination unit 12 as S30. When it is not determined that the driver needs to check the lighting state of the traffic light (S30: NO), the traffic light recognition ECU 10 ends the current processing. When it is determined that the driver needs to check the lighting state of the traffic light (S30: YES), the traffic light recognition ECU 10 proceeds to S32.

  In S32, the traffic signal recognition ECU 10 notifies the driver of confirmation of the lighting state of the traffic signal by the notification unit 13. The notification unit 13 notifies the driver of at least one of the image display by the display and the audio output by the speaker via the HMI 6.

  In S34, the traffic signal recognition ECU 10 determines whether the driver operation recognition unit 14 has recognized the driver's accelerator operation. When it is not determined that the driver's accelerator operation is recognized (S34: NO), the signal recognition ECU 10 proceeds to S36. If it is not determined that the driver's accelerator operation has been recognized, the driver's brake operation is used instead of the accelerator operation when the driver's accelerator operation can not be recognized even if a predetermined time has elapsed since the notification. The case of recognition is included. When it is determined that the driver's accelerator operation is recognized (S34: YES), the signal recognition ECU 10 proceeds to S38.

  In S36, the traffic light recognition ECU 10 determines the lighting state of the traffic light as a red signal by the lighting state determination unit 15 (maintains the recognition result of the lighting state of the traffic light). Thereafter, the signal recognition ECU 10 ends the current process.

  In S38, the traffic light recognition ECU 10 determines the lighting state of the traffic light as a green light by the lighting state judging unit 15 (corrects the lighting state of the traffic light to a green light).

  In S40, the traffic signal recognition ECU 10 performs learning by associating the recognition result of the lighting state of the traffic signal with the judgment result by the learning unit 16. Thereafter, the signal recognition ECU 10 ends the current process. Note that the process of learning in S40 is not essential.

[Operation effect of the traffic signal recognition device]
According to the traffic signal recognition apparatus 100 according to the present embodiment described above, when the lighting state of the traffic signal can not be recognized with sufficient accuracy (certainty factor) from the captured image of the camera, By notifying the confirmation and recognizing the brake operation or the accelerator operation by the driver, the lighting state of the traffic signal can be determined. Moreover, the traffic signal recognition apparatus 100 does not perform an extra input operation for the driver to cause an erroneous input such as voice input or a special button to be pressed, but the lighting state of the traffic light is used. Since the confirmation result by the driver is inputted by the brake operation of the vehicle or the accelerator operation, it is possible to make an intuitive input operation that matches the driver's physical sense, appropriately recognize the driver's input and The lighting state can be determined.

  Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. The present invention can be carried out in various forms including various modifications and improvements based on the knowledge of those skilled in the art, including the embodiments described above.

  For example, the traffic signal recognition apparatus 100 may be part of an automatic driving system. In this case, the traffic signal recognition ECU 10 can constitute part of the autonomous driving ECU 50.

  On the other hand, the traffic signal recognition apparatus 100 can also be used in vehicles that do not have an automatic driving function. The signal recognition ECU 10 of the signal recognition apparatus 100 does not necessarily have to be connected to the automatic driving ECU 50, and may be connected to a driving support ECU that performs driving support of the vehicle. In this case, the traffic signal recognition ECU 10 transmits the determination result of the lighting state of the traffic light to the driving support ECU to determine the lighting state of the traffic light in driving support such as issuing a warning when the vehicle enters an intersection of red light. The results can be used.

  The traffic signal recognition ECU 10 does not necessarily have to use the radar sensor 3 to recognize the occlusion of the traffic signal. In this case, the signal recognition ECU 10 does not have to be connected to the radar sensor 3. Further, the traffic signal recognition ECU 10 does not necessarily use the position of the vehicle and the map information for the determination of the confirmation necessity determination unit 12. In this case, the signal recognition ECU 10 does not have to be connected to the GPS receiver 1 and the map database 4. The signal recognition ECU 10 does not necessarily have to have the learning unit 16.

  DESCRIPTION OF SYMBOLS 1 ... GPS reception part, 2 ... Camera, 3 ... Radar sensor, 4 ... Map database, 5 ... Operation detection part, 6 ... HMI, 10 ... Signal machine recognition ECU, 11 ... Signal machine recognition part, 12 ... Confirmation necessity determination part, 13 ... notification part, 14 ... driver operation recognition part, 15 ... lighting state judgment part, 16 ... learning part, 50 ... automatic driving ECU, 100 ... signal recognition device.

Claims (1)

A traffic signal recognition apparatus that recognizes the lighting state of a traffic signal in front of the vehicle based on an image captured by a camera of the vehicle.
A confirmation necessity determination unit that determines whether it is necessary for the driver of the vehicle to confirm the lighting state of the traffic light based on the recognition result of the lighting state of the traffic light;
A notification unit that notifies the driver of confirmation of the lighting state of the traffic light when it is determined by the confirmation necessity determination unit that the lighting state of the traffic light needs to be confirmed by the driver;
A driver operation recognition unit that recognizes the driver's brake operation and accelerator operation in response to the notification;
A lighting state determination unit that determines the lighting state of the traffic light based on the recognition result of the driver operation recognition unit;
A traffic signal recognition apparatus comprising:

Images