JP2018081405A - Traffic light recognition device, vehicle, and traffic light recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traffic light recognition device, vehicle, and traffic light recognition method capable of realizing at least one of improving recognition accuracy of a traffic light and quickening recognition determination of a traffic light.SOLUTION: A traffic light recognition device 36 determines a change of a height H or a change of a color C in subsequent circumferential images Fs2, Fs3, ... concerning a traffic light candidate Isc extracted from at least one circumferential image Fs1, determines recognition that the traffic light candidate Isc is a traffic light 330 on the basis of a change of a height H and a change of a color C, and outputs information of a traffic light 330.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a traffic signal recognition device, a vehicle, and a traffic signal recognition method for recognizing a traffic signal based on a plurality of surrounding images captured by a camera.

  Patent Document 1 aims to provide a traveling environment recognition device that can maintain sufficient responsiveness while suppressing an increase in the amount of calculation ([0009], summary). In order to achieve the object, in Patent Document 1 (abstract), a radar 3 is provided in addition to the imaging device 2, and the imaging device 2 obtains image information of the traveling direction of the vehicle. Get the distance and direction to a stationary object. The image processing device 1 associates the preceding vehicle with the distance and direction obtained from the radar 3 in the image obtained from the imaging device 2, thereby easily obtaining image signal processing necessary for recognition of the preceding vehicle. To be able to. Then, the position of a stationary object, traffic light, pedestrian crossing, etc. is estimated to recognize the traveling environment around the vehicle. Further, the road map device 4 associates the vehicle position obtained from the GIS antenna 5 on the road map, and outputs the presence / absence of an intersection ahead of the vehicle and distance information to the intersection to the image processing device 1.

  Patent Document 1 discloses a traffic signal recognition process for recognizing a traffic signal provided on a road. In this traffic signal recognition process, an area (signal area 110) where a traffic signal is estimated to be present is determined from the distance to the intersection and the preset height of the traffic signal, and the signal state is determined from the color information inside the area. Is recognized ([0111]).

JP 2004-265432 A

  As described above, in Patent Document 1, the traffic light area 110 is determined based on the distance to the intersection and the height of the traffic light, and the signal state is recognized from the color information inside the traffic light area [0111]. However, for example, when the vehicle travels on a downhill and the preceding vehicle travels on a flat road ahead, the preceding vehicle exists in a relatively high position in the surrounding image. If the preceding vehicle enters the traffic signal area 110 in such a state, the host vehicle may erroneously recognize the preceding vehicle as a traffic signal. The same applies to the case where the host vehicle is traveling on a flat road and the preceding vehicle is traveling on an uphill ahead.

  Moreover, in patent document 1, since the signal state is recognized from the color information in the traffic signal area 110 determined based on the distance to the intersection and the height of the traffic signal, the determination of the signal state is performed for each peripheral image at each time point. It is understood that this is done individually. For example, it is considered that the color information obtained from a single peripheral image is less than that of a plurality of consecutive peripheral images. For this reason, in Patent Document 1, there is room for improvement in signal state recognition accuracy.

  The present invention has been made in view of the above-described problems, and provides a traffic signal recognition device, a vehicle, and a traffic signal recognition method capable of realizing at least one of improvement of traffic signal recognition accuracy and speeding up of traffic signal recognition. The purpose is to do.

The traffic signal recognition apparatus according to the present invention is:
Performing a first traffic signal recognition process for recognizing a traffic signal based on a plurality of peripheral images captured by a camera provided in the vehicle;
In the first traffic signal recognition process, the traffic signal recognition device includes:
For the traffic signal candidate extracted from at least one of the peripheral images, determine a change in height in the subsequent peripheral images,
Confirming that the candidate traffic signal is the traffic signal based on the change in height,
The information of the traffic light is output.

  According to the present invention, recognition of a traffic signal is determined based on a change in height of a traffic signal candidate. For this reason, it becomes possible to confirm the recognition of the traffic light based on a larger amount of information compared to the case where the recognition of the traffic light is determined from only one peripheral image, for example. Therefore, it becomes possible to improve the recognition accuracy of the traffic signal.

  In addition, when the first traffic signal recognition process is used in parallel with other processes such as a process for determining the traffic signal recognition from only one peripheral image, the first traffic signal recognition process is based on a larger amount of information. Since the recognition is confirmed, it becomes possible to speed up the recognition of the traffic light.

  The traffic signal recognizing device includes a first gradient threshold that is a gradient threshold indicating that the traveling path of the host vehicle is a downhill, and a gradient threshold that indicates that the traveling path of the host vehicle is an uphill. At least one of the second gradient threshold values may be set. The traffic signal recognizing device may acquire gradient information of the traveling path of the host vehicle stored in a map information database or detected by a gradient sensor. The traffic signal recognizing device may use the recognition result of the first traffic signal recognition process when the first usage condition is satisfied. The traffic light recognition device may not use the recognition result of the first traffic light recognition processing when the first usage condition is not satisfied. The first usage condition may include at least one of a gradient indicated by the gradient information falling below the first gradient threshold and a gradient exceeding a second gradient threshold.

  Thereby, when the traveling path of the own vehicle is a downhill or an uphill, the traffic signal can be recognized with high accuracy by using the recognition result of the first traffic signal recognition process.

  When the first usage condition is not satisfied, the traffic signal recognition device extracts the traffic signal candidate from at least one of the peripheral images and the traffic signal candidate appearing in the peripheral image used for the extraction of the traffic signal candidate. A third traffic signal recognition process for confirming that the traffic signal candidate is the traffic signal based on the characteristics (height, color, shape, etc.) may be executed.

  In the first traffic signal recognition process, the traffic signal recognition apparatus determines that the height of the traffic signal candidate in the surrounding image increases as the distance between the vehicle and the traffic signal candidate decreases. One may be used. As a result, it is possible to prevent a light source other than a traffic light that is located far away, a taillight of a preceding vehicle, and the like from being erroneously recognized as a traffic light.

  In the first traffic light recognition process, the traffic light recognition device may set one of the traffic light recognition conditions that the traffic signal candidate is located on or near a road on which the host vehicle travels. Thereby, it is possible to reduce the calculation load required for the recognition of the traffic light.

  In the first traffic signal recognition process, the traffic signal recognition device, for the traffic signal candidate extracted from the at least one peripheral image, changes in the distance between the vehicle and the traffic signal candidate in the subsequent peripheral image and the traffic signal candidate You may determine the change in height. The traffic signal recognizing device may determine recognition that the traffic signal candidate is the traffic signal based on the change in the distance and the change in the height. Thereby, in addition to the change in the height of the candidate traffic signal, the recognition of the traffic signal is determined based on the change in the distance between the vehicle and the candidate traffic signal. Therefore, it becomes possible to recognize the traffic signal with higher accuracy.

  The traffic signal recognizing device may set a traffic signal assumed position, which is a position where the traffic signal should be present, or a traffic signal assumed area, which is a region where the traffic signal should be present, based on map information. The traffic signal recognizing device may determine that the distance between the position of the traffic signal candidate and the assumed traffic signal position is less than a distance threshold, or that the candidate traffic signal is within the traffic signal assumption area as one of the recognition confirmation conditions of the traffic signal. Good. Thereby, it is possible to further improve the recognition accuracy of the traffic signal by recognizing the traffic signal on the basis of the position or area where the traffic signal exists.

  The traffic signal recognition device may continuously output the traffic signal information while tracking the traffic signal that has been recognized by the first traffic signal recognition process. Thereby, it becomes possible to use the traffic light recognized with high accuracy while associating with the passage of time.

The traffic signal recognition apparatus according to the present invention performs a second traffic signal recognition process for recognizing a traffic signal based on a plurality of surrounding images captured by a camera provided in the vehicle,
In the second traffic signal recognition process, the traffic signal recognition device includes:
For a candidate traffic light extracted from at least one of the peripheral images, determine a color change in the subsequent peripheral images,
Confirming that the candidate traffic signal is the traffic signal based on the color change,
The information of the traffic light is output.

  According to the present invention, recognition of a traffic signal is determined based on a change in the color of a traffic signal candidate. For this reason, it becomes possible to confirm the recognition of the traffic light based on a larger amount of information compared to the case where the recognition of the traffic light is determined from only one peripheral image, for example. Therefore, it becomes possible to improve the recognition accuracy of the traffic signal.

  In addition, when the second traffic signal recognition process is used in parallel with other control such as a control for determining the traffic signal recognition from only one peripheral image, the second traffic signal recognition process is based on a larger amount of information. Since the recognition is confirmed, it becomes possible to speed up the recognition of the traffic light.

  In the second traffic signal recognition process, the traffic signal recognition device may start determination of a color change of the traffic signal candidate in a subsequent peripheral image when blue is detected as the color of the traffic signal candidate. . In a tunnel or the like, a sodium lamp that emits light containing yellow and red components may be used. For this reason, compared with yellow and red, it is thought that the probability that blue is showing the traffic signal is high. Therefore, when blue is detected as the color of the traffic signal candidate, the determination of the color change of the traffic signal candidate in the subsequent peripheral image is started, thereby suppressing the reduction in the recognition accuracy of the traffic signal and reducing the calculation load. It becomes possible to reduce.

  In the second traffic signal recognition process, the traffic signal recognition device may set one of the traffic signal recognition conditions that the traffic signal candidate is located on or near the road on which the vehicle travels. Thereby, it is possible to reduce the calculation load required for the recognition of the traffic light.

  In the second traffic signal recognition process, the traffic signal recognition device, for the traffic signal candidate extracted from the at least one peripheral image, changes in the distance between the vehicle and the traffic signal candidate in the subsequent peripheral image and the traffic signal candidate The color change may be determined. The traffic signal recognizing device may determine recognition that the traffic signal candidate is the traffic signal based on the change in the distance and the change in the color. Thereby, in addition to the change in the color of the candidate traffic signal, the recognition of the traffic signal is determined based on the change in the distance between the vehicle and the candidate traffic signal. Therefore, it becomes possible to recognize the traffic signal with higher accuracy.

  The traffic signal recognizing device may set a traffic signal assumed position, which is a position where the traffic signal should be present, or a traffic signal assumed area, which is a region where the traffic signal should be present, based on map information. The traffic signal recognizing device may determine that the distance between the position of the traffic signal candidate and the assumed traffic signal position is less than a distance threshold, or that the candidate traffic signal is within the traffic signal assumption area as one of the recognition confirmation conditions of the traffic signal. Good. Thereby, it is possible to further improve the recognition accuracy of the traffic signal by recognizing the traffic signal on the basis of the position or area where the traffic signal exists.

  The traffic signal recognition device may continuously output the traffic signal information while tracking the traffic signal that has been recognized by the second traffic signal recognition process. Thereby, it becomes possible to use the traffic light recognized with high accuracy while associating with the passage of time.

The vehicle according to the present invention is
The above signal recognition device;
And an acceleration / deceleration control unit that automatically controls acceleration / deceleration using information on the traffic signal recognized by the traffic signal recognition device.

  Thereby, at least one of the improvement of the recognition accuracy of the traffic signal and the speeding up of the confirmation of the traffic signal recognition can be realized, so that the merchantability of the vehicle can be improved.

The traffic signal recognition method according to the present invention uses a traffic signal recognition device that recognizes a traffic signal based on a plurality of surrounding images captured by a camera provided in the vehicle,
The traffic signal recognition apparatus determines a change in height in a subsequent peripheral image for a traffic signal candidate extracted from at least one of the peripheral images, and the traffic signal candidate is the traffic signal based on the change in height. The first traffic signal recognition process for confirming the recognition of the signal is executed.

The traffic signal recognition method according to the present invention uses a traffic signal recognition device that recognizes a traffic signal based on a plurality of surrounding images captured by a camera provided in the vehicle,
The traffic signal recognition apparatus determines a color change in a subsequent peripheral image with respect to a traffic signal candidate extracted from at least one of the peripheral images, and recognizes that the traffic signal candidate is the traffic signal based on the color change. The second traffic light recognition process for determining

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement | achieve at least one of the improvement of the recognition accuracy of a traffic signal, and quickness of the recognition confirmation of a traffic signal.

1 is a block diagram showing a configuration of a vehicle (hereinafter referred to as “own vehicle”) including a travel electronic control device as a travel control device according to an embodiment of the present invention. It is a figure which shows the scene where the said own vehicle of the said embodiment is driving | running the uphill. It is a figure which shows simply an example of the periphery image which the camera of the said own vehicle of FIG. 2 acquired. It is a figure which shows the scene where the said vehicle of the said embodiment is drive | working the downhill. It is a flowchart which shows the whole flow of the traffic signal recognition control of the said embodiment. It is a flowchart of the normal recognition process in the said embodiment. It is a flowchart of the 1st exception recognition process in the said embodiment. It is a flowchart of the 2nd exception recognition process in the said embodiment.

A. One Embodiment <A-1. Configuration>
[A-1-1. overall structure]
FIG. 1 is a block diagram showing a configuration of a vehicle 10 including a travel electronic control device 36 (hereinafter referred to as “travel ECU 36” or “ECU 36”) as a travel control device according to an embodiment of the present invention. In addition to the travel ECU 36, the vehicle 10 (hereinafter also referred to as “own vehicle 10”) includes a vehicle peripheral sensor group 20, a vehicle body behavior sensor group 22, a driving operation sensor group 24, a communication device 26, a human machine machine. It has an interface 28 (hereinafter referred to as “HMI 28”), a driving force control system 30, a braking force control system 32, and an electric power steering system 34 (hereinafter referred to as “EPS system 34”).

[A-1-2. Vehicle Perimeter Sensor Group 20]
The vehicle periphery sensor group 20 detects information related to the periphery of the vehicle 10 (hereinafter also referred to as “vehicle periphery information Ic”). The vehicle periphery sensor group 20 includes a plurality of outside cameras 50, a plurality of radars 52, a LIDAR 54 (Light Detection And Ranging), and a global positioning system sensor 56 (hereinafter referred to as “GPS sensor 56”). included.

  A plurality of exterior cameras 50 (hereinafter also referred to as “cameras 50”) output image information Iimage relating to a peripheral image Fs obtained by imaging the periphery (front, side, and rear) of the vehicle 10. Each camera 50 continuously acquires the peripheral image Fs. In other words, each camera 50 is a video camera that acquires the peripheral image Fs as a moving image. However, the camera 50 may be, for example, a still camera that can continuously acquire the peripheral image Fs from the viewpoint of use in a first exception recognition process or a second exception recognition process described later.

  The plurality of radars 52 output radar information Iradar indicating reflected waves with respect to electromagnetic waves transmitted to the periphery (front, side, and rear) of the vehicle 10. The LIDAR 54 continuously emits a laser in all directions of the vehicle 10, measures the three-dimensional position of the reflection point based on the reflected wave, and outputs it as three-dimensional information Ilidar. The GPS sensor 56 detects the current position Pcur of the vehicle 10. The outside camera 50, the radar 52, the LIDAR 54, and the GPS sensor 56 are peripheral recognition devices that recognize the vehicle peripheral information Ic.

[A-1-3. Body behavior sensor group 22]
The vehicle body behavior sensor group 22 detects information related to the behavior of the vehicle 10 (particularly, the vehicle body) (hereinafter also referred to as “vehicle body behavior information Ib”). The vehicle body behavior sensor group 22 includes a vehicle speed sensor 60, an acceleration sensor 62, a yaw rate sensor 64, and a tilt sensor 66.

  The vehicle speed sensor 60 detects the vehicle speed V [km / h] and the traveling direction Dt of the vehicle 10. The acceleration sensor 62 detects the acceleration G [m / s / s] of the vehicle 10. The acceleration G includes a longitudinal acceleration α, a lateral acceleration Glat, and a vertical acceleration Gv (may be acceleration G in only a part of directions).

  The yaw rate sensor 64 detects the yaw rate Y [rad / s] of the vehicle 10. The inclination sensor 66 (gradient sensor) detects an inclination angle A [°] of the vehicle 10 (vehicle body) with respect to the traveling direction Dt of the vehicle 10. If the pitching of the vehicle body is ignored, the inclination angle A means the gradient Gr [°] of the travel lane 302 of the host vehicle 10.

[A-1-4. Driving operation sensor group 24]
The driving operation sensor group 24 detects information related to driving operation by the driver (hereinafter also referred to as “driving operation information Io”). The driving operation sensor group 24 includes an accelerator pedal sensor 80, a brake pedal sensor 82, a steering angle sensor 84, and a steering torque sensor 86.

  An accelerator pedal sensor 80 (hereinafter also referred to as “AP sensor 80”) detects an operation amount θap of the accelerator pedal 90 (hereinafter also referred to as “AP operation amount θap”) [%]. The brake pedal sensor 82 (hereinafter also referred to as “BP sensor 82”) detects an operation amount θbp of the brake pedal 92 (hereinafter also referred to as “BP operation amount θbp”) [%]. The steering angle sensor 84 detects the steering angle θst [deg] of the steering handle 94. The steering torque sensor 86 detects the steering torque Tst [N · m] applied to the steering handle 94.

[A-1-5. Communication device 26]
The communication device 26 performs wireless communication with an external device. The external devices here include, for example, an external server 250 as a traffic information server. The traffic information server provides traffic information such as traffic jam information, accident information, and construction information to each vehicle 10. Alternatively, the external server 250 may be a route guidance server. The route guidance server generates or calculates a planned route Rv to the target point Pgoal instead of the travel ECU 36 based on the current position Pcur of the vehicle 10 received from the communication device 26 and the target point Pgoal.

  In addition, although the communication apparatus 26 of this embodiment assumes what is mounted in the vehicle 10 (or always fixed), for example, it is a thing which can be carried out of the vehicle 10 like a mobile telephone or a smart phone. There may be.

[A-1-6. HMI28]
The HMI 28 receives an operation input from the occupant and presents various information to the occupant visually, audibly, and tactilely. The HMI 28 includes an automatic operation switch 110 (hereinafter also referred to as “automatic operation SW 110”), a speaker 112, and a display unit 114.

  The automatic operation SW 110 is a switch for instructing the start and end of automatic operation control by the operation of the occupant. In addition to or instead of the automatic operation SW 110, the start or end of the automatic operation control can be instructed by other methods (such as voice input via a microphone (not shown)). The display unit 114 includes, for example, a liquid crystal panel or an organic EL panel. The display unit 114 may be configured as a touch panel.

[A-1-7. Driving force control system 30]
The driving force control system 30 includes an engine 120 (drive source) and a drive electronic control device 122 (hereinafter referred to as “drive ECU 122”). The AP sensor 80 and the accelerator pedal 90 described above may be positioned as a part of the driving force control system 30. The drive ECU 122 executes drive force control of the vehicle 10 using the AP operation amount θap and the like. In driving force control, the drive ECU 122 controls the driving force Fd of the vehicle 10 through the control of the engine 120.

[A-1-8. Braking force control system 32]
The braking force control system 32 includes a brake mechanism 130 and a braking electronic control device 132 (hereinafter referred to as “braking ECU 132”). The BP sensor 82 and the brake pedal 92 described above may be positioned as a part of the braking force control system 32. The brake mechanism 130 operates a brake member by a brake motor (or a hydraulic mechanism) or the like.

  The braking ECU 132 executes the braking force control of the vehicle 10 using the BP operation amount θbp and the like. In the braking force control, the braking ECU 132 controls the braking force Fb of the vehicle 10 through the control of the brake mechanism 130 and the like.

[A-1-9. EPS system 34]
The EPS system 34 includes an EPS motor 140 and an EPS electronic control unit 142 (hereinafter referred to as “EPS ECU 142” or “ECU 142”). The steering angle sensor 84, the steering torque sensor 86, and the steering handle 94 described above may be positioned as a part of the EPS system 34.

  The EPS ECU 142 controls the EPS motor 140 in accordance with a command from the travel ECU 36 to control the steering angle θst of the vehicle 10.

[A-1-10. Traveling ECU 36]
(A-1-10-1. Overview of travel ECU 36)
The traveling ECU 36 performs automatic driving control for driving the vehicle 10 to the target point Pgoal without requiring a driving operation by the driver, and includes, for example, a central processing unit (CPU). The ECU 36 includes an input / output unit 150, a calculation unit 152, and a storage unit 154.

  A part of the function of the travel ECU 36 can be assigned to the external server 250 existing outside the vehicle 10. For example, the vehicle 10 itself may have a configuration in which the planned route Rv and / or the map information Imap is acquired from the route guidance server without having the action plan unit 172 and / or the map database 200 described later.

(A-1-10-2. Input / output unit 150)
The input / output unit 150 performs input / output with devices other than the ECU 36 (sensor groups 20, 22, 24, communication device 26, etc.).

(A-1-10-3. Calculation unit 152)
The computing unit 152 performs computation based on signals from the sensor groups 20, 22, 24, the communication device 26, the HMI 28, the ECUs 122, 132, 142, and the like. And the calculating part 152 produces | generates the signal with respect to the communication apparatus 26, drive ECU122, brake ECU132, and EPS ECU142 based on a calculation result.

  As illustrated in FIG. 1, the calculation unit 152 of the travel ECU 36 includes a periphery recognition unit 170, an action plan unit 172, and a travel control unit 174. Each of these units is realized by executing a program stored in the storage unit 154. The program may be supplied from the external server 250 via the communication device 26. A part of the program can be configured by hardware (circuit parts).

  The periphery recognition unit 170 displays lane marks (see lane mark images 514a, 514b, and 514c in FIG. 3) and peripheral obstacles (such as the preceding vehicle 320 in FIG. 2) based on the vehicle periphery information Ic from the vehicle periphery sensor group 20. recognize. For example, the lane mark is recognized based on the image information Iimage. The periphery recognition unit 170 recognizes the traveling lane 302 (FIG. 2 and the like) of the vehicle 10 based on the recognized lane mark.

  The peripheral obstacle is recognized using the image information Iimage, the radar information Iradar, and the three-dimensional information Ilidar. The peripheral obstacles include moving objects such as other vehicles (such as the preceding vehicle 320 in FIG. 2) and stationary objects such as buildings and road signs (for example, the traffic lights 330 (FIG. 2)). When the peripheral obstacle is the traffic light 330, the peripheral recognition unit 170 determines the color C, the arrow, and the like of the traffic light 330.

  The peripheral recognition unit 170 includes a traffic signal recognition unit 180 that performs traffic signal recognition control for recognizing the traffic signal 330. The traffic signal recognition control will be described later with reference to FIGS.

  The action planning unit 172 calculates the planned route Rv of the host vehicle 10 to the target point Pgoal input via the HMI 28, and performs route guidance along the planned route Rv.

  The travel control unit 174 controls the output of each actuator that controls the vehicle behavior. The actuator here includes the engine 120, the brake mechanism 130, and the EPS motor 140. The travel control unit 174 controls the behavior amount of the vehicle 10 (particularly the vehicle body) (hereinafter referred to as “vehicle body behavior amount Qb”) by controlling the output of the actuator. The vehicle body behavior amount Qb here includes the vehicle speed V, the longitudinal acceleration α, the steering angle θst, the lateral acceleration Glat, and the yaw rate Y.

  The travel control unit 174 includes an acceleration / deceleration control unit 190 that controls acceleration / deceleration of the host vehicle 10 and a turn control unit 192 that controls the turn of the host vehicle 10. The acceleration / deceleration control unit 190 executes automatic driving force control and automatic braking force control. The automatic driving force control mainly controls the output of the engine 120 to thereby control the traveling driving force Fd (or positive acceleration α) of the vehicle 10. ) To control. In the automatic braking force control, the braking force Fb (or negative acceleration α or deceleration β) of the vehicle 10 is controlled mainly by controlling the output of the brake mechanism 130. The turning control unit 192 performs automatic turning control. The automatic turning control mainly controls the steering angle θst (or the lateral acceleration Glat or the yaw rate Y) of the vehicle 10 by controlling the output of the EPS motor 140.

(A-1-10-4. Storage unit 154)
The storage unit 154 stores programs and data (including the map database 200) used by the calculation unit 152. The map database 200 (hereinafter referred to as “map DB 200”) stores road map information (map information Imap).

  The storage unit 154 includes, for example, a random access memory (hereinafter referred to as “RAM”). As the RAM, a volatile memory such as a register and a non-volatile memory such as a flash memory can be used. The storage unit 154 may include a read only memory (hereinafter referred to as “ROM”) in addition to the RAM.

<A-2. Automatic operation control of this embodiment>
[A-2-1. Overview of automatic operation control of this embodiment]
As described above, the travel ECU 36 of the present embodiment executes automatic operation control. In the automatic driving control, the vehicle 10 is driven to the target point Pgoal without requiring a driving operation by the driver. However, in the automatic driving control, when the driver operates the accelerator pedal 90, the brake pedal 92, or the steering handle 94, the operation may be reflected in the driving. In the automatic operation control of this embodiment, automatic driving force control, automatic braking force control, and automatic turning control are used in combination.

  In the automatic driving force control, the vehicle 10 travels by controlling the traveling driving force Fd. At this time, the ECU 36 sets a target value (for example, target engine torque) of the travel driving force Fd, and controls the actuator (engine 120) according to the target value.

  In the automatic braking force control, the vehicle 10 is decelerated by controlling the braking force Fb of the vehicle 10. At this time, the ECU 36 sets a target value (for example, target deceleration βtar) of the braking force Fb, and controls the actuator (brake mechanism 130) according to the target value.

  In the automatic turning control, the vehicle 10 is turned by controlling the steering angle θst of the vehicle 10. The turning of the vehicle 10 here includes not only the case of traveling on a curved road, but also includes turning the vehicle 10 left and right, changing the driving lane, joining to another lane, and maintaining the driving lane. The ECU 36 sets a target value (for example, target yaw rate Ytar or target lateral acceleration Glatar) of the steering angle θst, and controls the actuator (EPS motor 140) according to the target value. The automatic turning control includes lane maintenance assist system control (hereinafter also referred to as “LKAS control”, LKAS: Lane Keeping Assist System).

[A-2-2. Traffic light recognition control]
(A-2-2-1. Considerations)
FIG. 2 is a diagram illustrating a scene in which the vehicle 10 of the present embodiment is traveling on an uphill 310. Uphill 310 forms part of road 300 (hereinafter also referred to as “traveling path 300”). The road 300 includes a travel lane 302 of the host vehicle 10 and an opposite lane (not shown, see an opposite lane image 512 in FIG. 3). In FIG. 2, another vehicle 320 (hereinafter referred to as “preceding vehicle 320”) is traveling in front of the host vehicle 10 in the travel lane 302 of the host vehicle 10. In addition, a traffic light 330 is present on a downhill 312 existing ahead of the uphill 310. A virtual line Lx in FIG. 2 is an auxiliary line indicating the traveling direction Dt of the host vehicle 10.

  FIG. 3 is a diagram simply showing an example of the peripheral image Fs acquired by the camera 50 of the host vehicle 10 of FIG. The peripheral image Fs in FIG. 3 includes an image 500 of the road 300 (hereinafter also referred to as “road image 500”), an image 520 of the preceding vehicle 320 (hereinafter also referred to as “preceding vehicle image 520”), and a traffic light 330. Image 530 (hereinafter also referred to as “traffic signal image 530”).

  The road image 500 includes a travel lane image 502 showing the travel lane 302 of the host vehicle 10 and an opposite lane image 512 showing the opposite lane (not shown). The traveling lane image 502 is defined by lane mark images 514a and 514b, and the opposite lane image 512 is defined by lane mark images 514b and 514c. The preceding vehicle image 520 includes an image 522 of the taillight 322 (hereinafter referred to as “taillight image 522”).

  In the scene of FIG. 2, the traffic light 330 is located below the virtual line Lx. For this reason, when the traffic light 330 is red, the ECU 36 may misrecognize the light emission of the traffic light 330 as the light emission of the taillight 322 (FIG. 2) of the preceding vehicle 320 and may not recognize the traffic light 330 as a traffic light.

  FIG. 4 is a diagram illustrating a scene in which the vehicle 10 of the present embodiment is traveling on the downhill 312. In FIG. 4, a traffic light 330 different from that in FIG. 2 is present on the flat road 340 that exists ahead of the downhill 312. Similar to FIG. 2, the virtual line Lx in FIG. 4 is an auxiliary line indicating the traveling direction Dt of the host vehicle 10.

  In FIG. 4, the traffic light 330 is positioned relatively above the virtual line Lx as compared to a scene where the host vehicle 10 is traveling on a flat road 340. For this reason, when the recognition condition of the traffic signal 330 is that the height H (described later) of the traffic signal 330 is within a predetermined range, the ECU 36 may not be able to recognize the traffic signal 330 as a traffic signal.

  Therefore, in the present embodiment, the first tracking recognition control and the second tracking recognition control that can accurately recognize the traffic light 330 even when the gradient Gr of the road 300 of the host vehicle 10 is relatively large are used (for details, see FIG. 5 and FIG. 5). 7 and will be described later with reference to FIG.

(A-2-2-2. Specific flow of traffic light recognition control)
FIG. 5 is a flowchart showing an overall flow of traffic signal recognition control according to the present embodiment. As described above, the traffic light recognition control is control for recognizing the traffic light 330. In step S11, the ECU 36 determines whether or not the host vehicle 10 is approaching the traffic light 330 based on the current position Pcur of the host vehicle 10, the traveling direction Dt, and the map information Imap. For example, the determination is based on the distance Dx between the current position Pcur of the host vehicle 10 (or the position of a traffic signal candidate Isc described later) and the position of the intersection where the traffic signal 330 in front of the host vehicle 10 is provided (signal expected position). Judgment is made based on whether or not it is within the threshold THd. Alternatively, the ECU 36 sets a traffic light assumption area Rsa that is an area where the traffic light 330 should be present based on the map information Imap, and the current position Pcur of the host vehicle 10 (or the position of a traffic light candidate Isc described later) is the traffic light assumption area. The determination may be made based on whether or not it is within Rsa.

  If the host vehicle 10 is approaching the traffic light 330 (S11: true), the process proceeds to step S12. If the host vehicle 10 is not approaching the traffic light 330 (S11: false), the current process is terminated, and the process returns to step S11 after a predetermined time has elapsed.

  In step S12, the ECU 36 acquires gradient information Ig indicating the gradient Gr of the travel lane 302 of the host vehicle 10. In the present embodiment, the gradient Gr indicating an uphill (such as the uphill 310 in FIG. 2) is a positive value, and the gradient Gr indicating a downhill (such as the downhill 312 in FIG. 2) is a negative value. However, the opposite definition is possible. The acquisition of the gradient information Ig can be acquired as information corresponding to the current position Pcur from the map DB 200, for example. Alternatively, the inclination angle A detected by the inclination sensor 66 (gradient sensor) can be used as it is or corrected to obtain the gradient Gr.

  In step S13, the ECU 36 determines whether or not the traveling lane 302 of the host vehicle 10 is a flat road (such as the flat road 340 in FIG. 4) based on the gradient information Ig. Specifically, when the gradient Gr indicated by the gradient information Ig is equal to or greater than the negative first gradient threshold THg1 and equal to or less than the positive second gradient threshold THg2 (THg1 ≦ Gr ≦ THg2), the ECU 36 It is determined that the traveling lane 302 is a flat road. On the contrary, when the gradient Gr is less than the first gradient threshold THg1 (Gr <THg1), in other words, when the travel lane 302 is a downhill, the ECU 36 determines that the travel lane 302 is not the flat road 340. When the second gradient threshold THg2 is exceeded (Gr> THg2), in other words, when the travel lane 302 is on an uphill, the ECU 36 determines that the travel lane 302 is not a flat road.

  When the traveling lane 302 of the host vehicle 10 is a flat road (S13: true), the process proceeds to step S14. In step S <b> 14, the ECU 36 executes a normal recognition process (third signal recognition process) for determining the recognition of the traffic light 330 from the single (in other words, one frame) peripheral image Fs from the camera 50. Here, “confirm recognition of the traffic light 330” means that the recognized traffic light 330 is assumed to be used for acceleration / deceleration control. The normal recognition process will be described later with reference to FIG.

  When the traveling lane 302 of the host vehicle 10 is not a flat road (S13: false), the process proceeds to steps S15 and S16. In this embodiment, steps S15 and S16 are processed in parallel. Alternatively, one of steps S15 and S16 can be performed first and the other can be performed later.

  In step S <b> 15, the ECU 36 executes a first exception recognition process (first signal recognition process) for determining the recognition of the traffic light 330 from the plurality of surrounding images Fs from the camera 50. The first exception recognition process will be described later with reference to FIG.

  In step S <b> 16, the ECU 36 executes a second exception recognition process (second signal recognition process) for determining the recognition of the traffic light 330 from the plurality of surrounding images Fs from the camera 50. The second exception recognition process will be described later with reference to FIG.

  After step S14 or steps S15 and S16, in step S17, the ECU 36 determines whether or not the traffic light 330 has been recognized in the current process (S14 to S16). The recognition of the traffic signal 330 here means that the recognition that the later-described traffic signal candidate Isc is the traffic signal 330 is confirmed. When the traffic light 330 is recognized (S17: true), the process proceeds to step S18. When the traffic light 330 is not recognized (S17: false), the current traffic light recognition control is terminated, and the process returns to step S11 after a predetermined time has elapsed.

  In step S18, the ECU 36 determines whether or not the association (that is, tracking) with the traffic light 330 recognized in the previous process (previous S14 to S16) is possible. The determination is based on the elapsed time from the previous process to the current process and the vehicle speed V, and the presence of the currently recognized traffic signal 330 is present at a position (or within a region) where the presence of the previously recognized traffic signal 330 is predicted. It is determined by whether or not to do so.

  If tracking with the traffic light 330 recognized last time is possible (S18: true), the ECU 36 performs tracking in step S19. When the traffic light 330 is recognized in both the first and second exception recognition processes (S15 and S16), the ECU 36 uses the result of the tracking used in the previous process, for example, in the current process. On the other hand, if tracking with the traffic light 330 recognized last time is not possible (S18: false), the ECU 36 starts tracking in step S20.

  After step S19 or S20, the ECU 36 outputs a recognition result (that is, information Is of the tracked traffic light 330 (hereinafter also referred to as “traffic signal information Is”)). The traffic signal information Is here includes the position Ps (or distance D) and the color C of the traffic signal 330.

  The ECU 36 keeps tracking until the traffic signal 330 being tracked is removed from the use target of the automatic operation control. Therefore, for the traffic light 330 being tracked, the ECU 36 continues to select each of the consecutive surrounding images Fs (or at a predetermined interval) as a usage target (in other words, whether the usage target exclusion condition is satisfied). ).

As the use target exclusion condition here, for example, the following conditions can be used alone or in combination.
・ Some or all of the traffic light 330 no longer exists in the surrounding image Fs due to the passing of the intersection by the own vehicle 10 ・ All the traffic light 330 disappeared from the surrounding image Fs for some reason ・ Recognized as the traffic light 330 Have exhibited characteristics different from those of the traffic light 330 (for example, the same requirements as in step S54 in FIG. 7 and / or step S74 in FIG. 8 described later are not satisfied).

[A-2-3. Normal recognition processing]
FIG. 6 is a flowchart of normal recognition processing in the present embodiment. As described above, in the normal recognition process, the recognition (or detection) of the traffic light 330 is determined from a single peripheral image Fs (one frame) from the camera 50.

  In step S31, the ECU 36 determines whether or not there is a candidate Isc for the traffic light 330 (hereinafter also referred to as “traffic light candidate Isc”) in a single peripheral image Fs (herein referred to as “peripheral image Fs1”). judge. In the present embodiment, the candidate Isc is extracted as a region having a color C unique to the traffic light 330. For example, in the case of a vehicle traffic signal 330 in Japan, it is normal to emit light in three colors of blue, yellow and red. Therefore, when any one of these three colors C is detected in the peripheral image Fs1, it is set as a candidate Isc.

  When there are a plurality of candidate Iscs, the ECU 36 performs a normal recognition process for each candidate Isc. If there is a traffic signal candidate Isc (S31: true), the process proceeds to step S32. When the traffic signal candidate Isc does not exist (S31: false), the current normal recognition process is terminated.

  In step S <b> 32, the ECU 36 determines whether or not the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10. The “lane LN in the same direction as the host vehicle 10” here refers to the traveling lane 302 of the host vehicle 10 on a one-lane road. In addition, in a two-lane road on one side, it refers to a traveling lane 302 of the host vehicle 10 and a lane adjacent thereto (for traveling in the same direction). Further, “near” refers to a range where the traffic light 330 can exist other than “on lane LN”.

  Whether the candidate Isc is on or near the lane LN is determined as follows, for example. That is, the ECU 36 projects the lane mark LM and the traffic signal candidate Isc on a two-dimensional space in plan view based on the peripheral image Fs1 acquired by the camera 50. Then, it is specified whether or not the candidate Isc is located within the range of the lane LN defined by the lane mark LM. In other words, the ECU 36 determines whether the candidate Isc is located on the lane LN.

  If the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10 (S32: true), the process proceeds to step S33. When the traffic signal candidate Isc is not on the lane LN in the same direction as the host vehicle 10 (S32: false), the process proceeds to step S35.

  In step S <b> 33, the ECU 36 determines whether or not the distance D and the height H of the traffic signal candidate Isc in the peripheral image Fs <b> 1 indicate the characteristics of the traffic signal 330. The distance D is a distance [m] between the current position Pcur of the host vehicle 10 and the position Psc of the candidate traffic light Isc. When the outside camera 50 is configured as a stereo camera, the position Psc of the traffic signal candidate Isc is calculated based on the peripheral image Fs of each stereo camera. Alternatively, the position Psc may be calculated by combining the peripheral image Fs and the radar information Iradar. Alternatively, when the outside camera 50 is configured as a monocular camera, the ECU 36 may obtain the distance D by monocular stereoscopic vision from two peripheral images Fs with different imaging times of the monocular camera. The height H of the traffic signal candidate Isc is defined as the distance [number of pixels or m] between the candidate Isc and the lane LN in the peripheral image Fs.

  In the peripheral image Fs1, the height H of the traffic light 330 changes corresponding to the distance D. That is, when the distance D is relatively long, the height H is low, and when the distance D is relatively short, the height H is high. Therefore, the ECU 36 selects a range that the height H can take in accordance with the distance D, and when the height H in the peripheral image Fs1 is included in this range, it is determined that the candidate Isc indicates the characteristics of the traffic light 330. To do.

  When the distance D and the height H indicate the characteristics of the traffic light 330 (S33: true), the process proceeds to step S34. When the distance D or the height H does not indicate the characteristics of the traffic light 330 (S33: false), the process proceeds to step S35.

  In step S <b> 34, the ECU 36 determines that the traffic signal candidate Isc is the traffic signal 330. Thereby, the ECU 36 controls acceleration / deceleration of the host vehicle 10 according to the color C of the traffic light 330. In step S <b> 35, the ECU 36 determines that the traffic signal candidate Isc is not the traffic signal 330.

[A-2-4. First exception recognition process]
FIG. 7 is a flowchart of the first exception recognition process in the present embodiment. As described above, in the first exception recognition process, the recognition (or detection) of the traffic light 330 is determined from a plurality of surrounding images Fs (frames) from the camera 50. The difference from the second exception recognition process is that the first exception recognition process uses a change in the height H of the traffic signal candidate Isc, whereas the second exception recognition process uses a change in the color C of the traffic signal candidate Isc. It is in. If there are a plurality of candidate Iscs, the ECU 36 executes a first exception recognition process for each candidate Isc.

  Steps S51 and S52 in FIG. 7 are the same as steps S31 and S32 in FIG. That is, in step S51, the ECU 36 determines whether or not the traffic signal candidate Isc exists in the single peripheral image Fs1. In the present embodiment, the candidate Isc is extracted as a region having a color C unique to the traffic light 330.

  As described above, in the case of the vehicle traffic signal 330 in Japan, it is normal to emit light in three colors of blue, yellow and red. Therefore, when any one of these three colors C is detected in the peripheral image Fs1, it is set as a candidate Isc.

  When there are a plurality of candidate Iscs, the ECU 36 executes a first exception recognition process for each candidate Isc. In step S52, the ECU 36 determines whether the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10. Steps S31 and S32 in FIG. 6 and steps S51 and S52 in FIG. 7 may be common steps.

  When the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10 (S52: true), the process proceeds to step S53. When the traffic signal candidate Isc is not on or near the lane LN in the same direction as the host vehicle 10 (S52: false), the process proceeds to step S57.

  In step S53, the ECU 36 changes the distance D and the height H of the candidate Isc in the peripheral image Fs (Fs2, Fs3,...) After the peripheral image Fs (Fs1) used in steps S51 and S52. judge. The number of the surrounding images Fs used here can be a number (for example, any value of 2 to 300) that can significantly determine the change in the distance D and the change in the height H. Note that the initial values of the distance D and the height H may be calculated in step S51 or the like.

  In step S54, the ECU 36 determines whether or not the change in the distance D and the change in the height H of the traffic signal candidate Isc determined in step S53 indicate the characteristics of the traffic signal 330. The definitions of the distance D and the height H are the same as those in the normal recognition process (FIG. 6).

  When the host vehicle 10 approaches the traffic signal 330, the traffic light 330 on the peripheral image Fs has a shorter distance D to the host vehicle 10 and a higher height H. Therefore, the feature of the traffic light 330 regarding the change of the distance D and the change of the height H can be used that the distance D becomes shorter and the height H becomes higher as the host vehicle 10 travels.

  When the change in the distance D and the change in the height H indicate the characteristics of the traffic light 330 (S54: true), in step S55, the ECU 36 determines that the candidate Isc is the traffic light 330. When the change in the distance D or the change in the height H does not indicate the characteristics of the traffic light 330 (S54: false), the process proceeds to step S56.

  In step S56, the ECU 36 determines whether a candidate exclusion condition for excluding the candidate Isc from the determination target is satisfied. The candidate exclusion condition can be used, for example, that a change in the distance D or a change in the height H does not clearly indicate the traffic light 330. Alternatively, it may be used that the distance D itself or the height H itself does not clearly indicate the traffic light 330.

  If the candidate exclusion condition is not satisfied (S56: false), the process returns to step S52. If the candidate exclusion condition is satisfied (S56: true), in step S57, the ECU 36 determines that the traffic signal candidate Isc is not the traffic signal 330.

[A-2-5. Second exception recognition process]
FIG. 8 is a flowchart of the second exception recognition process in the present embodiment. As described above, in the second exception recognition process, the recognition (or detection) of the traffic light 330 is determined from a plurality of surrounding images Fs (frames) from the camera 50. The difference from the first exception recognition process is that the first exception recognition process uses a change in the height H of the traffic signal candidate Isc, whereas the second exception recognition process uses a change in the color C of the traffic signal candidate Isc. It is in. When there are a plurality of candidate Iscs, the ECU 36 executes a second exception recognition process for each candidate Isc.

  Steps S71 and S72 in FIG. 8 are the same as steps S31 and S32 in FIG. 6 and steps S51 and S52 in FIG. That is, in step S71, the ECU 36 determines whether or not the traffic signal candidate Isc exists in the single peripheral image Fs1. In the present embodiment, the candidate Isc is extracted as a region having a color C unique to the traffic light 330. The unique colors here can be three colors of blue, yellow and red, as in step S51.

  Alternatively, only the blue color among the above three colors may be determined in step S71 for the following reason. That is, in the normal recognition process (FIG. 6), the extraction of the candidate Isc and the confirmation of the traffic light 330 are performed only in the single peripheral image Fs1. On the other hand, in the second exception recognition process (FIG. 8), the traffic light 330 is recognized using a specific change of the candidate Isc in the plurality of surrounding images Fs (Fs1, Fs2, Fs3,...). In addition, for yellow and red among the above three colors, those components are included in the light of a sodium lamp provided in a tunnel or the like, while for blue, the light containing that component is rare. Therefore, by setting the color determined in step S71 to only blue, it is possible to improve the accuracy of extracting the candidate Isc.

  In addition, when using blue preferentially for said reason, it is also possible to comprise so that the timing which starts step S73 in the case of blue may be advanced rather than yellow and red. For example, when adding whether or not the distance from the vehicle 10 to the traffic signal candidate Isc is equal to or smaller than the distance threshold value in the determination in step S71, the distance threshold value in the case of blue is longer than the distance threshold value in the case of yellow and red. May be. Alternatively, the distance condition may not be provided for blue, and the distance condition (threshold value, etc.) may be provided for yellow and red.

  Similarly, when adding whether or not the area of the candidate Isc in the peripheral image Fs (on the peripheral image Fs) is equal to or larger than the area threshold to the determination in step S71, the case where the area is blue rather than the area threshold in the case of yellow and red The area threshold may be reduced. Alternatively, an area condition may not be provided for blue, and an area condition (threshold, etc.) may be provided for yellow and red.

  In step S72, the ECU 36 determines whether the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10. Steps S31 and S32 in FIG. 6 and steps S51 and S52 in FIG. 7 and steps S71 and S72 in FIG. 8 may be common steps.

  If the traffic signal candidate Isc is on or near the lane LN in the same direction as the host vehicle 10 (S72: true), the process proceeds to step S73. If the traffic signal candidate Isc is not on or near the lane LN in the same direction as the host vehicle 10 (S72: false), the process proceeds to step S77.

  In step S73, the ECU 36 determines the change in the distance D and the change in the color C of the candidate Isc in the peripheral image Fs (Fs2, Fs3,...) After the peripheral image Fs (Fs1) used in steps S71 and S72. To do. The number of the peripheral images Fs used here can be a number (for example, any value of 2 to 300) that can significantly determine the change in the distance D and the change in the color C. Note that the initial values of the distance D and the color C may be calculated in step S71 and the like.

  In step S74, the ECU 36 determines whether or not the change in the distance D and the change in the color C of the traffic signal candidate Isc determined in step S73 indicate the characteristics of the traffic signal 330. The definition of the distance D is the same as in the normal recognition process (FIG. 6) and the first exception recognition process (FIG. 7). Moreover, the feature of the traffic light 330 regarding the change of the distance D can use that the distance D becomes short as the own vehicle 10 advances.

  Regarding the color C, for example, in the case of a vehicle traffic signal 330 in Japan, it is normal to change in the order of blue → yellow → red → blue → yellow → red →. For this reason, if the change of the color C follows this rule, it can be recognized that the candidate Isc is the traffic light 330. Therefore, in the present embodiment, it is determined that the change in the color C indicates the characteristic of the traffic light 330 when the change in two colors (for example, blue → yellow) among the three colors occurs. Alternatively, it may be determined that the change in the color C indicates the characteristics of the traffic light 330 when a change in three colors (for example, blue → yellow → red) occurs among the three colors. Alternatively, when the traffic light 330 includes an arrow signal, the lighting pattern is complicated. Considering this point, the lighting pattern may be acquired from the map information Imap, and the ECU 36 may determine whether the lighting pattern and the change in the color C match.

  In the traffic light 330, a single light does not emit light in different colors, but a plurality of lights emit light in different colors (fixed colors). For this reason, it can be made a part of the conditions of the change of the color C that the change of the color C is accompanied by a position shift.

  When the change in the distance D and the change in the color C indicate the characteristics of the traffic light 330 (S74: true), in step S75, the ECU 36 determines that the candidate Isc is the traffic light 330. When the change in the distance D or the change in the color C does not indicate the characteristics of the traffic light 330 (S74: false), the process proceeds to step S76.

  In step S76, the ECU 36 determines whether a candidate exclusion condition for excluding the candidate Isc from the determination target is satisfied. Candidate exclusion conditions can be used, for example, that a change in distance D or a change in color C does not clearly indicate traffic light 330. Alternatively, it may be used that the distance D itself or the color C itself does not clearly indicate the traffic light 330.

  If the candidate exclusion condition is not satisfied (S76: false), the process returns to step S72. If the candidate exclusion condition is satisfied (S76: true), in step S77, the ECU 36 determines that the traffic signal candidate Isc is not the traffic signal 330.

<A-3. Effects of this embodiment>
As described above, according to the first exception recognition process of the present embodiment, the recognition of the traffic light 330 is determined based on the change in the height H of the traffic light candidate Isc (S15 in FIG. 5 and FIG. 7). For this reason, for example, the recognition of the traffic light 330 can be confirmed based on a larger amount of information as compared with the case where the recognition of the traffic light 330 is confirmed only from one peripheral image Fs (the normal recognition processing or the like in FIG. 6). It becomes possible. Therefore, the recognition accuracy of the traffic light 330 can be increased.

  Further, when the first exception recognition process is used in parallel with other processes such as the process of determining the recognition of the traffic light 330 from only one peripheral image Fs, the first exception recognition process is based on a larger amount of information. Since the recognition of the traffic light 330 is confirmed, the recognition confirmation of the traffic light 330 can be accelerated.

  In the present embodiment, the travel ECU 36 (signal recognition device) has a first gradient threshold value that is a threshold value of the gradient Gr indicating that the road 300 of the host vehicle 10 is a downhill (for example, the downhill 312 in FIGS. 2 and 4). THg1 and a second gradient threshold value THg2 that is a threshold value of the gradient Gr indicating that the road 300 of the host vehicle 10 is an uphill (for example, the uphill 310 in FIG. 2) are set (S13 in FIG. 5). The traveling ECU 36 acquires the gradient information Ig of the road 300 of the host vehicle 10 that is stored in the map DB 200 (map information database) or detected by the inclination sensor 66 (gradient sensor) (S12 in FIG. 5). When the gradient Gr indicated by the gradient information Ig is lower than the first gradient threshold THg1 or when the gradient Gr is higher than the second gradient threshold THg2 (S13: false), in other words, when the first usage condition is satisfied, the travel ECU 36 The recognition results of the exception recognition process (S15) and the second exception recognition process (S16) are used (S21). When the gradient Gr exceeds the first gradient threshold THg1 and falls below the second gradient threshold THg2 (S13: true), in other words, when the first usage condition is not satisfied, the recognition results of the first exception recognition process and the second exception recognition process Is not used.

  Thereby, when the road 300 of the own vehicle 10 is the downhill 312 or the uphill 310, the traffic lights 330 can be recognized with high accuracy by using the recognition results of the first exception recognition process and the second exception recognition process. Become.

  In the first exception recognition process (FIG. 7), the travel ECU 36 (signal recognition device) increases the height H of the traffic signal candidate Isc in the peripheral image Fs as the distance D between the vehicle 10 and the traffic signal candidate Isc becomes shorter. Is one of the recognition confirmation conditions of the traffic light 330 (S54). As a result, it is possible to prevent a light source other than the traffic light 330 that is located far away, the tail light 322 of the preceding vehicle 320, and the like from being erroneously recognized as the traffic light 330.

  In the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8), the travel ECU 36 (signal recognition device) has the traffic signal candidate Isc located on or near the road 300 on which the vehicle 10 travels. This is one of the recognition confirmation conditions of the traffic light 330 (S52, S72). Thereby, it becomes possible to reduce the calculation load required for the recognition of the traffic light 330.

  In the first exception recognition process (FIG. 7), the travel ECU 36 (signal recognition device) determines the traffic signal candidate Isc extracted from one peripheral image Fs1 and the vehicle 10 and the traffic signal in the subsequent peripheral images Fs2, Fs3, and so on. A change in the distance D from the candidate Isc and a change in the height H of the traffic signal candidate Isc are determined (S53). The ECU 36 determines that the candidate traffic signal Isc is the traffic signal 330 based on the change in the distance D and the change in the height H (S54). Thereby, in addition to the change in the height H of the traffic signal candidate Isc, the recognition of the traffic light 330 is determined based on the change in the distance D between the vehicle 10 and the traffic signal candidate Isc. Therefore, the traffic light 330 can be recognized with higher accuracy.

  In the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8), the travel ECU 36 (signal recognition device) is a traffic light assumption area Rsa that is an area where the traffic light 330 should be present based on the map information Imap. Is set as one of the recognition confirmation conditions of the traffic signal 330 that the traffic signal candidate Isc is in the expected traffic signal area Rsa (S11 in FIG. 5). Accordingly, the recognition accuracy of the traffic light 330 can be further improved by performing the recognition of the traffic light 330 with reference to the area where the traffic light 330 exists.

  In the present embodiment, the travel ECU 36 (signal recognition device) continuously tracks the signal information Is while tracking the signal 330 that has been recognized by the first signal recognition process (FIG. 7) or the second exception recognition process (FIG. 8). (FIGS. 5, 7 and 8). Thereby, it is possible to use the traffic light 330 recognized with high accuracy while associating it with the passage of time.

  According to the second exception recognition process (FIG. 8) of this embodiment, the recognition of the traffic light 330 is determined based on the change in the color C of the traffic light candidate Isc (S16 in FIG. 5 and FIG. 8). For this reason, for example, the recognition of the traffic light 330 can be confirmed based on a larger amount of information as compared with the case where the recognition of the traffic light 330 is confirmed only from one peripheral image Fs1 (eg, the normal recognition processing of FIG. 6). It becomes possible. Therefore, the recognition accuracy of the traffic light 330 can be increased.

  Further, when the second exception recognition process is used in parallel with other control such as the control for determining the recognition of the traffic light 330 from only one peripheral image Fs1, the second exception recognition process is based on a larger amount of information. Since the recognition of the traffic light 330 is confirmed, the recognition confirmation of the traffic light 330 can be accelerated.

  In the second exception recognition process (FIG. 8) of the present embodiment, the travel ECU 36 (signal recognition device) detects that blue is detected as the color C of the traffic signal candidate Isc (S71: true), and the subsequent surrounding image. The determination (S73) of the change of the color C of the traffic signal candidate Isc in Fs may be started.

  In a tunnel or the like, a sodium lamp that emits light containing yellow and red components may be used. For this reason, it is considered that the probability that the blue color indicates the traffic light 330 is higher than that of yellow and red. Therefore, when the blue color is detected as the color of the traffic signal candidate Isc, the determination of the change in the color C of the traffic signal candidate Isc in the subsequent peripheral image Fs is started to suppress a decrease in the recognition accuracy of the traffic light 330. However, it is possible to reduce the calculation load.

  In the second exception recognition process (FIG. 8), the travel ECU 36 (signal recognition device) determines the traffic signal candidate Isc extracted from one peripheral image Fs1 and the vehicle 10 and the traffic signal in the subsequent peripheral images Fs2, Fs3, and so on. A change in the distance D from the candidate Isc and a change in the color C of the traffic signal candidate Isc are determined (S73). The ECU 36 determines that the candidate traffic signal Isc is the traffic signal 330 based on the change in the distance D and the change in the color C (S74). Thereby, in addition to the change of the color C of the traffic signal candidate Isc, the recognition of the traffic light 330 is determined based on the change of the distance D between the vehicle 10 and the traffic signal candidate Isc. Therefore, the traffic light 330 can be recognized with higher accuracy.

  The vehicle 10 of this embodiment includes a travel ECU 36 (signal recognition device) and an acceleration / deceleration control unit 190 that automatically controls acceleration / deceleration using information of the traffic signal 330 recognized by the travel ECU 36 (FIG. 1). Thereby, at least one of the improvement of the recognition accuracy of the traffic light 330 and the speeding up of the recognition confirmation of the traffic light 330 can be realized, so that the merchantability of the vehicle 10 can be improved.

B. Modifications It should be noted that the present invention is not limited to the above-described embodiment, and it is needless to say that various configurations can be adopted based on the description of the present specification. For example, the following configuration can be adopted.

<B-1. Applicable object>
In the above embodiment, it is assumed that the travel ECU 36 (travel control device) is used for a vehicle 10 as a car (FIG. 1). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, the vehicle 10 (or vehicle) may be a moving object such as a railroad, a ship, or an aircraft. Alternatively, the vehicle 10 can be used for other devices (for example, various robots and manufacturing devices).

<B-2. Configuration of Vehicle 10>
[B-2-1. Sensor group 20, 22, 24]
The vehicle surrounding sensor group 20 of the above embodiment includes a plurality of outside cameras 50, a plurality of radars 52, a LIDAR 54, and a GPS sensor 56 (FIG. 1). However, for example, from the viewpoint of determining a change in the height H or color C of the traffic signal candidate Isc, the present invention is not limited to this. When the plurality of outside cameras 50 include a stereo camera that detects the front of the vehicle 10, the radar 52 and / or the LIDAR 54 can be omitted. Alternatively, when using monocular stereoscopic vision or when distance information is not acquired by the outside camera 50, the outside camera 50 may be configured as a monocular camera.

  The vehicle body behavior sensor group 22 of the above embodiment includes a vehicle speed sensor 60, an acceleration sensor 62, a yaw rate sensor 64, and a tilt sensor 66 (FIG. 1). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited thereto. For example, any one or more of the vehicle speed sensor 60, the acceleration sensor 62, the yaw rate sensor 64, and the tilt sensor 66 may be omitted.

  The driving operation sensor group 24 of the embodiment includes the AP sensor 80, the BP sensor 82, the steering angle sensor 84, and the steering torque sensor 86 (FIG. 1). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, any one or more of the AP sensor 80, the BP sensor 82, the steering angle sensor 84, and the steering torque sensor 86 may be omitted.

[B-2-2. Actuator]
In the above-described embodiment, the engine 120, the brake mechanism 130, and the EPS motor 140 are used as actuators that are targets in automatic operation control (FIG. 1). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, any one or two of the engine 120, the brake mechanism 130, and the EPS motor 140 can be excluded from the targets of automatic operation control. When one of the actuators is excluded from the target of the automatic operation control, the driver performs control related to the actuator that is excluded from the target. Further, as described above, instead of the EPS motor 140, it is also possible to turn using the torque difference between the left and right wheels (so-called torque vectoring).

<B-3. Control of traveling ECU 36>
[B-3-1. Automatic operation control]
In the above embodiment, the present invention is applied to the vehicle 10 that automatically performs acceleration / deceleration and turning (FIG. 1). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, the present invention can be applied to the vehicle 10 that automatically performs only one or two of acceleration, deceleration (including only automatic braking) and turning. Alternatively, the present invention can be applied to a navigation device (not shown). In that case, by determining the position of the traffic light 330, it is possible to notify the presence / absence of the traffic light 330 (or an intersection), to correct the current position Pcur detected by the GPS sensor 56, and the like.

[B-3-2. Traffic light recognition control]
In the above embodiment, only the normal recognition process (FIG. 6) is executed on the flat road 340 (S13 in FIG. 5: true) (S14). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, in addition to or instead of the normal recognition process, one or both of the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8) can be performed.

  In the above embodiment, the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8) are executed on the uphill 310 or the downhill 312 (S13 in FIG. 5: false) (S15, S16). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this.

  For example, only one of the uphill 310 or the downhill 312 can be determined. It is also possible to perform only one of the first exception recognition process and the second exception recognition process. Alternatively, when the height H that can be taken by the traffic light 330 is changed in accordance with the gradient Gr, the normal recognition process is performed in addition to one or both of the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8). It is also possible to do this. Alternatively, the normal recognition process, the first exception recognition process, and the second exception recognition process may be continuously performed, and whether or not the determination result is accepted may be determined based on the gradient information Ig.

  In the above embodiment, the processing is switched according to the gradient Gr (S13 to S16 in FIG. 5). However, for example, from the viewpoint of recognizing the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, it is possible to perform processing independently of the gradient Gr. Further, a range of the gradient Gr that performs only the normal recognition process, a range of the gradient Gr that performs only the first exception recognition process, and a range of the gradient Gr that performs both the normal recognition process and the first exception recognition process may be provided. The same applies to the second exception recognition process.

  In the normal recognition process (FIG. 6), it is determined whether or not there is a traffic signal candidate Isc in the single peripheral image Fs1 (S31). However, for example, from the viewpoint of extracting the traffic signal candidate Isc, it is not limited to this. For example, it may be determined whether there is a traffic signal candidate Isc in a plurality of surrounding images Fs. The same applies to step S51 of the first exception recognition process (FIG. 7) and step S71 of the second exception recognition process (FIG. 8).

  In the above embodiment, the first exception recognition process (FIG. 7) and the second exception recognition process (FIG. 8) are executed separately (FIG. 5). However, for example, from the viewpoint of recognizing the traffic light 330 based on the change in the height H and the color C of the traffic light candidate Isc, it is possible to combine both exception recognition processes into one. For example, the determination that the candidate Isc is the traffic light 330 may be confirmed based on whether the change in the distance D, the height H, and the color C of the traffic light candidate Isc indicates the characteristics of the traffic light 330.

  In the traffic signal recognition control of the above embodiment, it is determined whether or not the host vehicle 10 is approaching the traffic signal 330 based on the distance D between the current position Pcur of the host vehicle 10 and the position of the intersection where the traffic signal 330 is provided ( S11 of FIG. However, for example, it is determined that the host vehicle 10 is approaching the traffic signal 330 (the host vehicle 10 is approaching an area where the traffic signal 330 should be present or an area where the traffic signal 330 is likely to be present). From a viewpoint, it is not limited to this. For example, when an external device such as a road-side beacon exists around the traveling road 300 of the host vehicle 10, the ECU 36 may acquire information regarding the presence of the traffic light 330 from the external device. Alternatively, when the traffic light recognition unit 180 cannot recognize information that can be automatically operated (for example, the color C of the traffic light 330), but can determine the presence or absence of the traffic light 330 (traffic sign) (such as shape recognition of the traffic light 330 as a whole) Step S11 can also be performed based on the determination.

  In the traffic signal recognition control (FIGS. 5 to 8) of the above embodiment, a conclusion was made by accumulating individual determinations. However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H or the color C of the traffic light candidate Isc, the present invention is not limited to this. For example, the determination items in each step shown in FIGS. 5 to 8 (for example, S11 in FIG. 5, S31 to S33 in FIG. 6, S51, S52 and S54 in FIG. 7, S71, S72 and S74 in FIG. 8) are pointed respectively. When the total value of the points exceeds the point threshold value, the recognition that the traffic signal is 330 may be confirmed.

[B-3-3. Normal recognition processing]
In the normal recognition processing of the above embodiment, the recognition confirmation that the candidate Isc is the traffic light 330 is determined based on the color C, the distance D, and the height H of the traffic light candidate Isc (S31 and S33 in FIG. 6). However, for example, for the traffic signal candidate Isc extracted from at least one peripheral image Fs, the traffic light 330 is based on the characteristics (height, color, shape, etc.) of the candidate Isc appearing in the peripheral image Fs used for extraction of the candidate Isc. From the viewpoint of confirming the recognition that there is, it is not limited to this. For example, in the normal recognition processing, the candidate Isc is recognized as the traffic light 330 based on the shape or area of the candidate Isc in the peripheral image Fs in addition to or instead of the color C, the distance D, and the height H. Confirmation may be determined.

[B-3-4. First exception recognition process]
In the first exception recognition process of the above embodiment, both the change in the distance D and the change in the height H of the traffic signal candidate Isc are determined (S53 in FIG. 7). However, for example, from the viewpoint of determining the recognition of the traffic light 330 based on the change in the height H of the traffic signal candidate Isc, the determination of the change in the distance D is not limited to this, and the determination of the change in the distance D may be omitted.

[B-3-5. Second exception recognition process]
In the second exception recognition process of the above embodiment, both the change in the distance D and the change in the color C of the traffic signal candidate Isc are determined (S73 in FIG. 8). However, for example, from the viewpoint of confirming the recognition of the traffic light 330 based on the change in the color C of the traffic light candidate Isc, the determination of the change in the distance D is not limited thereto, and the determination of the change in the distance D may be omitted.

<B-4. Other>
In the above embodiment, the steps are executed in the order shown in FIGS. However, for example, as long as the purpose of each step can be realized (in other words, the effect of the present invention can be obtained), the order of the steps can be changed. For example, step S11 can be performed after step S13 in FIG. 5 (before each of steps S14 to S16). Also, the order of steps S51 and S52 in FIG. 7 can be changed or executed simultaneously.

  In the above embodiment, there is a case where the equal sign is included and a case where the equal sign is not included in the comparison of numerical values (S13 in FIG. 5). However, for example, if there is no special meaning including or removing the equal sign (in other words, the effect of the present invention can be obtained), it is optional whether or not the equal sign is included in the comparison of numerical values. Can be set.

  In that sense, for example, it is determined whether or not the gradient Gr in step S13 of FIG. 5 is equal to or greater than the first gradient threshold THg1 (Gr ≧ THg1), and whether or not the gradient Gr is greater than the first gradient threshold THg1. (Gr> THg1) can be substituted. Similarly, the determination whether the gradient Gr is equal to or smaller than the second gradient threshold THg2 (Gr ≦ THg2) may be replaced with the determination whether the gradient Gr is smaller than the second gradient threshold THg2 (Gr <THg2). it can.

10 ... Vehicle (own vehicle)
36 ... Traveling ECU (signal recognition device) 50 ... Outside camera (camera)
66 ... Inclination sensor (gradient sensor) 190 ... Acceleration / deceleration control unit 200 ... Map DB (map information database) 300 ... Road (traveling road)
310 ... Uphill 312 ... Downhill 330 ... Traffic light C ... Color D ... Distance between own vehicle and traffic light candidate Dx ... Distance between current position of own vehicle and expected traffic light position Fs, Fs1, Fs2, Fs3 ... Peripheral image Gr ... Gradient H ... Height Ig ... Gradient information Imap ... Map information Is ... Traffic signal information Isc ... Traffic signal candidate Rsa ... Traffic signal assumption area THd ... Distance threshold THg1 ... First gradient threshold THg2 ... Second gradient threshold

Claims (16)

A traffic signal recognition device that executes a first traffic signal recognition process for recognizing a traffic signal based on a plurality of peripheral images captured by a camera provided in the host vehicle,
In the first traffic signal recognition process, the traffic signal recognition device includes:
For the traffic signal candidate extracted from at least one of the peripheral images, determine a change in height in the subsequent peripheral images,
Confirming that the candidate traffic signal is the traffic signal based on the change in height,
A signal recognition device, characterized in that the information on the signal is output. In the traffic signal recognition device according to claim 1,
The traffic signal recognition device is:
At least a first gradient threshold that is a gradient threshold indicating that the traveling path of the host vehicle is a downhill and a second gradient threshold that is a threshold of gradient indicating that the traveling path of the host vehicle is an uphill Set one side,
Acquire gradient information of the traveling path of the vehicle stored in the map information database or detected by the gradient sensor;
When the first usage condition is satisfied, the recognition result of the first traffic signal recognition process is used,
If the first use condition is not satisfied, the recognition result of the first traffic light recognition process is not used,
The first usage condition includes at least one of a slope indicated by the slope information being lower than the first slope threshold and a slope being higher than a second slope threshold. In the traffic signal recognition device according to claim 1 or 2,
In the first traffic signal recognition process, the traffic signal recognition apparatus determines that the height of the traffic signal candidate in the surrounding image increases as the distance between the vehicle and the traffic signal candidate decreases. A traffic signal recognizing device characterized by comprising one. In the traffic signal recognition device according to any one of claims 1 to 3,
In the first traffic signal recognition process, the traffic signal recognition device sets one of the traffic signal recognition conditions that the traffic signal candidate is located on or near a road on which the vehicle travels. Traffic light recognition device. In the traffic signal recognition device according to any one of claims 1 to 4,
In the first traffic signal recognition process, the traffic signal recognition device includes:
For the traffic light candidate extracted from the at least one peripheral image, determine a change in the distance between the vehicle and the traffic light candidate in the subsequent peripheral image and a change in the height of the traffic light candidate,
A traffic signal recognition apparatus, wherein recognition that the traffic signal candidate is the traffic signal is determined based on the change in the distance and the change in the height. In the traffic signal recognition device according to any one of claims 1 to 5,
The traffic signal recognition device is:
Based on the map information, set a traffic light assumed position that is a position where the traffic light should be present or a traffic light assumed area that is a region where the traffic light should be present,
One of the signal recognition recognition conditions is that the distance between the position of the signal candidate and the assumed position of the signal is less than a distance threshold, or that the candidate signal is within the assumed signal area. apparatus. In the traffic signal recognition device according to any one of claims 1 to 6,
The traffic signal recognizing device continuously outputs information of the traffic signal while tracking the traffic signal that has been recognized by the first traffic signal recognition process. A traffic signal recognition device that executes a second traffic signal recognition process for recognizing a traffic signal based on a plurality of surrounding images captured by a camera provided in the host vehicle,
In the second traffic signal recognition process, the traffic signal recognition device includes:
For a candidate traffic light extracted from at least one of the peripheral images, determine a color change in the subsequent peripheral images,
Confirming that the candidate traffic signal is the traffic signal based on the color change,
A signal recognition device, characterized in that the information on the signal is output. In the traffic signal recognition device according to claim 8,
In the second traffic signal recognition process, the traffic signal recognition device starts determination of a color change of the traffic signal candidate in a subsequent peripheral image when blue is detected as the color of the traffic signal candidate. A traffic light recognition device. In the traffic signal recognition device according to claim 8 or 9,
In the second traffic signal recognition process, the traffic signal recognition device sets one of the traffic signal recognition conditions that the candidate traffic signal is located on or near the road on which the vehicle travels. Traffic light recognition device. In the traffic signal recognition device according to any one of claims 8 to 10,
In the second traffic signal recognition process, the traffic signal recognition device includes:
For the traffic light candidate extracted from the at least one peripheral image, determine a change in the distance between the vehicle and the traffic light candidate in the subsequent peripheral image and a change in the color of the traffic light candidate,
A traffic signal recognition apparatus, wherein recognition that the traffic signal candidate is the traffic signal is determined based on the change in distance and the change in color. In the traffic signal recognition device according to any one of claims 8 to 11,
The traffic signal recognition device is:
Based on the map information, set a traffic light assumed position that is a position where the traffic light should be present or a traffic light assumed area that is a region where the traffic light should be present,
One of the signal recognition recognition conditions is that the distance between the position of the signal candidate and the assumed position of the signal is less than a distance threshold, or that the candidate signal is within the assumed signal area. apparatus. In the traffic signal recognition device according to any one of claims 8 to 12,
The traffic signal recognizing device continuously outputs information of the traffic signal while tracking the traffic signal that has been recognized by the second traffic signal recognition process. A traffic light recognition device according to any one of claims 1 to 13,
A vehicle comprising: an acceleration / deceleration control unit that automatically controls acceleration / deceleration using information on the traffic signal recognized by the traffic signal recognition device. A traffic signal recognition method using a traffic signal recognition device for recognizing a traffic signal based on a plurality of surrounding images captured by a camera provided in the own vehicle,
The traffic signal recognition apparatus determines a change in height in a subsequent peripheral image for a traffic signal candidate extracted from at least one of the peripheral images, and the traffic signal candidate is the traffic signal based on the change in height. A traffic signal recognition method, comprising: executing a first traffic signal recognition process for confirming recognition of a signal. A traffic signal recognition method using a traffic signal recognition device for recognizing a traffic signal based on a plurality of surrounding images captured by a camera provided in the own vehicle,
The traffic signal recognition apparatus determines a color change in a subsequent peripheral image with respect to a traffic signal candidate extracted from at least one of the peripheral images, and recognizes that the traffic signal candidate is the traffic signal based on the color change. A signal recognition method comprising: executing a second signal recognition process for determining

Images