JP2014215697A - Outside vehicle environment recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately determine the type of signal devices detected in a photographed image in order to appropriately execute vehicle control and issue warnings.SOLUTION: An outside vehicle environment recognition device detects the presence of signal devices in a photographed image obtained by photographing an image in a direction of travel of a vehicle, performs processing to determine a distance distribution from the vehicle on the detected one or more signal devices, and performs type determination whether the signal devices detected in the photographed image are traffic lights or instruction signal devices by using a result of the determination of the distance distribution so as to create determination information.

Description

  The present invention relates to a technical field relating to a vehicle exterior environment recognition apparatus that determines a traffic signal ahead using a captured image obtained by capturing a traveling direction of a host vehicle.

JP2007-34693 JP-A-2005-170154

For example, driving assistance technologies for assisting the driver, such as collision avoidance such as detecting the approach of a preceding vehicle and other obstacles to brake the vehicle, and auto-cruising, are known. In such driving support technology, the distance to an object existing in front of the vehicle is detected, and based on the result, vehicle control such as brake control and throttle control is performed, and a warning is given to the driver.
The detection of traffic signals that are in front of the vehicle while driving is also important for vehicle control. For example, traffic signal detection is performed for an alarm or automatic stop control according to a red signal.

Techniques for detecting a forward red signal from an image captured by a camera attached to a vehicle are disclosed in Patent Documents 1 and 2 above.
In Patent Document 1, the shape of a red signal is used to determine whether or not a traffic light on the traveling path of the host vehicle is a red signal. In other words, the fact that the traffic signal in front of the host vehicle is closer to a perfect circle and the further away it is, the closer it is to an ellipse.
Patent Document 2 discloses that a traffic signal is detected as a condition for vehicle stop control. Specifically, when two or more red signals are detected at an intersection in the traveling direction of the host vehicle, the intersection is detected. It is described that the driving support control is performed with the stop line as a stop point.

By the way, there are various traffic lights on the road on which the vehicle travels in addition to the normal traffic lights that perform traffic control for smooth traffic at intersections and the like. For example, a traffic signal indicating opening and closing of a toll gate on a toll road, a traffic signal indicating a lane, and the like. These will be referred to as “instruction traffic lights” for the sake of explanation. When these instruction traffic lights are detected, it may not be appropriate to perform vehicle control or warning as in the case of traffic lights such as intersections.
For example, the techniques of Patent Documents 1 and 2 both detect a red signal, but do not consider the type of traffic signal. There may be a number of red traffic lights that indicate the toll gate and the central line on the toll road, so it may be misdetected as a traffic signal at an intersection, or a red signal in the lane adjacent to the toll gate May be determined as a red traffic signal and stop control may occur. For this reason, for appropriate driving support control, it is necessary to distinguish between traffic lights and light emitters (indicating traffic lights) having characteristics similar to those of other traffic signals.

  Accordingly, an object of the present invention is to make it possible to determine whether a traffic signal detected in a captured image obtained by capturing the traveling direction of the host vehicle is a traffic signal or an instruction signal.

1stly, the external environment recognition apparatus which concerns on this invention is the own vehicle about the traffic signal detection part which detects presence of a traffic signal in the picked-up image which imaged the advancing direction of the own vehicle, and the traffic signal detected by the said traffic signal detection part. A traffic signal type determination unit that performs a type determination of whether the detected traffic signal is a traffic signal or an instruction traffic signal using a distance distribution from and generates a determination information indicating a result of the type determination. is there.
In most cases, there are a plurality of instruction signals for toll gates, lane instructions, and the like at positions where the forward distance from the vehicle is substantially equal. By determining this point based on the distance distribution from the host vehicle, it is possible to distinguish the indication signal from the traffic signal provided at an installation location such as an intersection or a crosswalk.

Secondly, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic light type determination unit further determines the light emission shape of the traffic light detected in the captured image, and the light emission shape determination result is also the type determination. It is desirable to use it.
Some indication traffic lights have a light emission shape that is different from the traffic light ● shape, such as ◎ or ×. Therefore, the light emission shape can also be used as an element for determining the type of traffic signal and instruction signal.

Third, in the above-described outside environment recognition device according to the present invention, the traffic signal type determination unit further includes a bulletin board-like shape in which the distance from the own vehicle is determined to be the same as the traffic signal around the detected traffic signal. It is desirable to determine whether or not an object exists and use the determination result of the presence of a bulletin board-like object for the type determination.
In the case of an instruction signal at a toll gate or the like, a bulletin board is often provided along with the signal. Therefore, the presence of a bulletin board-like object can also be used as an element for type determination.

Fourthly, in the above-described outside environment recognition device according to the present invention, the traffic light type determination unit acquires the emission color information of the traffic light and the three-dimensional position information of the traffic light from the own vehicle, and is detected by the traffic light detection unit. The number of other traffic signals that exist within a predetermined distance before and after the specific traffic signal, the horizontal distance between the traffic signals when there is another traffic signal, and the traffic signal and the other traffic signal It is desirable to determine whether the detected traffic light is a traffic light or an instruction traffic light based on the mutual emission color information.
The number of other traffic signals existing within a predetermined distance is obtained from the distance distribution of the detected traffic signals. By using this information, the horizontal distance (interval) between the traffic lights, and the emission color information, more accurate traffic signal type determination is realized.

Fifth, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic light type determination unit acquires the emission color information of the traffic light and the three-dimensional position information of the traffic signal from the host vehicle, and the traffic light detection unit detects the traffic light. If two or more are detected, the traffic light with the closest forward distance from the host vehicle within the distance range to be processed is identified, and the light emission color is the same as that of the traffic light within a predetermined distance before and after the identified traffic light. If there is another traffic signal of the same color, if the horizontal distance between the traffic signal and the other traffic signal is greater than or equal to a predetermined interval, the traffic signal and the other traffic signal are used as traffic signals. In addition, when the horizontal distance between the traffic signal and the other traffic signal is less than a predetermined interval, it is desirable to determine the traffic signal and the other traffic signal as the instruction traffic signal.
From the information obtained as the distance distribution, identifying the traffic signal having the closest forward distance from the host vehicle within the distance range to be processed specifies the traffic signal to be determined with the highest priority for the host vehicle. In this case, by using the presence or absence of other traffic signals of the same color and the horizontal distance of the signal period, more accurate traffic signal type determination is realized.

Sixth, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic signal type determination unit acquires a three-dimensional position of the traffic signal from the host vehicle, and three or more traffic signals are detected by the traffic signal detection unit. If the three or more traffic signals are all present within a predetermined distance in the front and rear, the three or more traffic signals are determined as indication traffic signals. As the front distance of the host vehicle, there are three or more traffic signals within the predetermined front and rear distance. It can be determined that there is a traffic signal at the toll gate, that is, an instruction signal with a high probability.

Seventhly, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic light type determination unit acquires the light emission color information of the traffic light and the three-dimensional position information of the traffic light from the host vehicle, and at least the blue light is detected by the traffic light detection unit. If two or more traffic lights including a light signal in the light emission state are detected and there is a traffic light in the red light emission state within a predetermined range from the signal light in the blue light emission state, the traffic light in the blue light emission state is excluded from the traffic light signal. Therefore, it is desirable to determine the type of traffic signal based on the distribution of distance from the host vehicle.
If there is a red light emitting signal within a predetermined range from the blue light emitting signal, it is likely that the blue light emitting signal is an arrow signal, so it is appropriate to exclude it.

Eighth, in the above-described vehicle environment recognition apparatus according to the present invention, when the traffic signal type determination unit is determined to be the command signal from at least the determination result of the distance distribution, the command signal is the width of the host vehicle. When it is detected that a red light is present within a predetermined left-to-right distance from the center of the direction and the front distance to the instruction signal is within a predetermined distance, the instruction signal is re-determined as a traffic signal. Is desirable.
For example, even in the case of an instruction signal, when the vehicle is traveling toward the instruction signal when it is in a red light emission state, vehicle control, warning, etc. equivalent to the case of a traffic light in the red light emission state are performed. Is appropriate. Therefore, determination information corresponding to the red light approach situation is obtained.

Ninthly, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic light type determination unit acquires the light emission color information and the luminance information of the traffic light, and when the traffic light detection unit detects one traffic light, It is desirable to determine whether the traffic signal is a traffic signal or an instruction signal using the luminance value of at least one sampling area set within a predetermined area of the traffic signal.
When there is one traffic light, it is preferable to use the light emission shape determination result for the traffic light type determination. The light emission shape of the traffic light can be determined by setting one or more sampling areas and using the luminance value of each sampling area.
Tenth, in the above-described vehicle environment recognition apparatus according to the present invention, the predetermined area is a rectangular area surrounding a light emitting portion of the traffic light, and the sampling area is a central area set at a central portion of the rectangular area. And determining whether the traffic signal is a traffic signal or an instruction traffic signal based on the luminance value of each sampling region and the emission color information of the traffic light, which is an intermediate region set in the middle of at least one of the 4 sides. It is desirable.
By setting such a sampling region, it is possible to easily determine the shape of ◎, ×, or ●.

Eleventh, in the above-described vehicle environment recognition apparatus according to the present invention, the traffic signal type determination unit has a bulletin board set above the traffic signal in a captured image when the traffic signal detection unit detects one traffic signal. Refer to the distance information for each pixel in the search range, and refer to the first comparison result for comparing the distance information for each pixel and the forward distance of the traffic light from the vehicle and the color information of the bulletin board search range. And determining that there is a bulletin board-like object above the traffic light based on a second comparison result for comparing whether or not the color information in the bulletin board search range is predetermined color information. It is desirable to judge as.
When there is one traffic signal, it is preferable to use the determination result of the presence of the bulletin board-like object for the traffic signal type determination. In that case, by using the first and second comparison results, it can be accurately determined that a bulletin board-like object exists above the traffic light.

  12thly, in the above-mentioned outside environment recognition device according to the present invention, when the traffic signal type determination unit determines that the traffic signal is a traffic signal, the brake is performed based on the emission color information and distance information of the traffic signal. It is desirable to further include a driving support control unit that performs control or alarm control. Thereby, the brake control and alarm control based on the red signal of the traffic signal can be executed.

  According to the present invention, when a traffic light is detected from a captured image, it is possible to generate determination information that distinguishes a traffic light from an instruction traffic light. This determination information is useful information for driving support control such as appropriate vehicle control and warning.

It is explanatory drawing of the vehicle system structure containing the external environment recognition apparatus of embodiment. It is a flowchart of traffic signal determination processing and traffic signal error detection suppression processing of the embodiment. It is a flowchart of distance distribution determination processing of an embodiment. It is explanatory drawing of distance distribution determination of embodiment. It is a flowchart of the light emission shape determination process of embodiment, and a bulletin board determination process. It is explanatory drawing of the light emission shape determination of embodiment. It is explanatory drawing of bulletin board determination of embodiment. It is a flowchart of an erroneous detection suppression release determination process and an erroneous detection suppression determination process of the embodiment. It is explanatory drawing of the conditions of false detection suppression cancellation | release determination of embodiment.

Hereinafter, embodiments of the present invention will be described. First, “traffic traffic light” and “instruction traffic light” used in the present specification and claims will be described.
"Traffic signal" is installed for traffic smoothness and safety at intersections and pedestrian crossings, etc. on roads, and traffic is controlled by switching red light emission / blue light emission etc. to indicate vehicle progress / stop It is a traffic light that performs.
The “indicator traffic signal” refers to a traffic signal for indicating closing / opening of a toll gate or the like, a traffic signal installed on a road for lane indication, and the like other than the traffic signal.

<1. Vehicle control system configuration>
With reference to FIG. 1, a vehicle control system 1 including an outside environment recognition unit 2 as an embodiment of the present invention will be described. The vehicle environment recognition unit 2 is a device having a function of discriminating between traffic signals and instruction signals in order to perform driving support control by the vehicle control system 1, and corresponds to the vehicle environment recognition device in the claims.
As shown in FIG. 1, the vehicle control system 1 includes an outside environment recognition unit 2, a vehicle control unit 10, a display control unit 11, an actuator 12, a display 13, and sensors 14.

The actuator 12 comprehensively represents an actuator for running and braking the vehicle. For example, the engine-related actuator 12 includes a throttle actuator that drives a throttle valve, an injector that performs fuel injection, and the like. Further, as a transmission-related actuator, for example, a control valve that performs shift control of an automatic transmission, a lockup actuator that locks up a lockup clutch, and the like are provided. Further, as a brake-related actuator, for example, a hydraulic pressure control actuator for controlling the hydraulic pressure output from the brake booster to the master cylinder and the hydraulic pressure in the brake fluid piping is provided.
The display 13 is an MFD (Multi Function Display), for example, and presents various information to the driver. For example, the display 13 can display travel speed, engine speed, fuel gauge, gear display, gear shift instruction, various alarms, and the like.
The sensors 14 are an engine speed sensor, an intake air amount sensor that detects an intake air amount, an accelerator opening sensor that detects an accelerator opening amount from the depression amount of an accelerator pedal, and each cylinder of the engine that is interposed in an intake passage. Sensors for detecting the state of the vehicle and the engine operation, such as a throttle opening sensor for detecting the opening of a throttle valve for adjusting the amount of intake air supplied to the engine, a water temperature sensor for detecting the coolant temperature indicating the engine temperature, a brake switch, etc. Comprehensive.

The vehicle control unit 10 controls the state of each unit related to vehicle travel, and controls the operation of the actuator 12 based on detection signals from the sensors as the sensors 14. For example, the fuel injection timing, the injection pulse width, the opening degree of the throttle valve, and the like are controlled. Further, actuator control for driving support, for example, brake control, is also performed in response to a control request from the outside environment recognition unit 2 (driving support control unit 6 described later).
The display control unit 11 performs display control based on requests and information from the vehicle control unit 10 and the vehicle exterior environment recognition unit 2 so that the display 13 executes various displays as described above. Also, control such as alarm display is performed in response to a display request from the vehicle exterior environment recognition unit 2 (a driving support control unit 6 described later).
The vehicle control unit 10, the display control unit 11, and the vehicle environment recognition unit 2 can perform mutual data communication via the bus 15.

In the vehicle control system 1, the vehicle environment recognition unit 2 recognizes an object existing in front of the vehicle, and can perform driving support control according to the recognized object.
For this reason, the vehicle environment recognition unit 2 includes imaging units 3A and 3B, an image processing unit 4, a memory unit 5, and a driving support control unit 6.

  The imaging units 3A and 3B are installed so as to be able to image the traveling direction (front) of the vehicle. In particular, two cameras as the imaging units 3A and 3B are installed so as to enable distance measurement by a so-called stereo method, and the two image data captured by the imaging units 3A and 3B respectively. (Referred to as first captured image data and second captured image data) are output for each frame.

The image processing unit 4 is composed of a semiconductor integrated circuit including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) as a work area, and follows a program stored in the ROM. Perform various processes.
In the case of the present embodiment, the image processing unit 4 stores the first captured image data and the frame image data as the second captured image data supplied from the imaging units 3A and 3B in the memory unit 5. Based on the first captured image data and the second captured image data of each frame, various processes for recognizing an object existing in front of the vehicle as an external environment are executed.
Although various types of object recognition and vehicle control corresponding to the various types are various, in the present embodiment, a signal is detected as an object and processing for determining the type is performed. In order to detect the traffic signal and determine the type, the image processing unit 4 performs the following processing.

First, distance image generation processing for generating a distance image based on the first captured image data and the second captured image data held in the memory unit 5 is executed. A distance image is a pattern matching that detects corresponding points between a pair of image data that have been captured in stereo, finds the shift in coordinates between the detected corresponding points as a parallax, and uses the parallax to implement the principle of triangulation. This is data representing the distance to the corresponding point in space on the image.
Next, an area selection process based on the distance image generated by the distance image generation process is executed, and an object detection process for detecting an object existing in the image with respect to the distance image is executed using the result of the area selection process. In connection with the operation of the present embodiment, a traffic light that is present in front of the vehicle is detected.
Further, a traffic signal type determination process is performed for the traffic signal detected by the above processing. That is, a process for determining a distance distribution from the own vehicle for the detected traffic light is performed. Then, using the determination result of the distance distribution, it is determined whether the detected traffic signal is a traffic signal or an instruction signal, and determination information indicating the classification determination result is generated.
In the case of this example, the distance image generation process, the object (traffic signal) detection process, and the traffic signal type determination process are realized by software processing by the CPU according to a program stored in the ROM. In FIG. 1, for the sake of convenience, it is assumed that there is hardware for executing these processes, and they are represented as a distance image generation unit 4a, a traffic signal detection unit 4b, and a traffic signal type determination unit 4c, respectively.

Determination information generated as a result of the traffic signal type determination process and other information related to object recognition generated by the processing of the image processing unit 4 are supplied to the driving support control unit 6.
The driving support control unit 6 includes a semiconductor integrated circuit including a CPU, a ROM, and a RAM, and executes various processes according to programs stored in the ROM. Information relating to object recognition from the image processing unit 4 and sensor signals such as an accelerator opening sensor, a vehicle speed sensor, and a brake switch as sensors 14 are input to the driving support control unit 6. Based on these inputs, various driving assistance, for example, control for assisting the driver, such as collision avoidance and auto cruise, is performed.
In particular, according to the determination information as a result of the above-described signal type determination processing, for example, the vehicle control unit 10 is requested to perform a stop operation according to traffic signal determination as a red signal (a traffic light in a red light emitting state) or the display 13 Control processing such as requesting the display control unit 11 to execute alarm display is performed. Since the determination information from the image processing unit 4 includes the determination result of the traffic signal and the instruction signal, control according to the signal type can be performed.

<2. Overall processing of traffic signal judgment>
Hereinafter, the traffic signal determination process executed in the image processing unit 4 of the outside environment recognition unit 2 will be described. The traffic signal determination process is a process of recognizing a traffic signal as an object existing in front of the vehicle, further determining the type of the traffic signal, and generating the determination information.

FIG. 2A is a flowchart showing the overall signal determination process executed by the image processing unit 4.
The image processing unit 4 acquires image data in step S1. That is, one frame is acquired as the above-described first captured image data and second captured image data. Note that the traffic signal determination process of FIG. 2A is repeatedly executed every time one frame is acquired as the first captured image data and the second captured image data.

In step S2, a distance image is generated by the function of the above-described distance image generation unit 4a using the first captured image data and the second captured image data.
In step S3, a traffic light is detected as an object present in the distance image by the function of the traffic light detection unit 4b described above. In particular, for the detected traffic light, information on the color (light emission color) of the traffic light at that time and the actual three-dimensional position of the traffic light with respect to the host vehicle is obtained.
Various specific methods of signal detection processing are conceivable. For example, it can be detected by using pattern matching for a traffic signal shape, high luminance region detection as a signal light emission portion, or the like, but other methods may be used.

If no traffic light is detected in step S3, the image processing unit 4 ends the process of FIG. 2A from step S4, and starts the process using the first and second captured image data of the next frame from step S1 again. .
If one or more traffic lights are detected in step S3, the image processing unit 4 proceeds from step S4 to step S5, and executes traffic signal false detection suppression processing by the function as the traffic signal type determination unit 4c described above. This is a process for determining whether the traffic light detected in step S3 is a traffic light or an instruction traffic light. In principle, it has a meaning as a process for suppressing erroneous detection that recognizes an instruction signal as a traffic signal.

The traffic signal error detection suppression processing in step S5 is shown in detail in FIG. 2B.
As shown in the figure, the traffic signal erroneous detection suppression process includes a distance distribution determination process (S10), a light emission shape determination process (S12), a bulletin board determination process (S13), an erroneous detection suppression release determination process (S14), and an erroneous detection suppression determination. Processing (S16) and branch processing (S11, S15) according to conditions are included.
The distance distribution determination process (S10) is a process for discriminating the traffic signal and the instruction signal from the distance and the number of the detected signals, that is, the distance distribution of the signals.
The light emission shape determination process (S12) is a process for determining the type of traffic signal based on the light emission shape as the shape ● or X shape when the determination is impossible in the distance distribution determination process (S10).
The bulletin board determination process (S13) is a process for determining the type of traffic signal by detecting an indicator mark (bulletin board) such as a toll gate existing around the traffic signal when the distance distribution determination process (S10) is not possible. .
The erroneous detection suppression determination process (S16) is a process of generating determination information for finally determining whether the signal is a traffic signal or an instruction signal based on the results of the above processes.
The erroneous detection suppression cancellation determination process (S14) is a process for canceling the instruction signal determination only when the red signal is advanced from the front and approaches a certain distance or more even when it is determined that the signal is an instruction signal. .
Although each process will be described later, as a result, determination information is generated at a certain point in step S5 of FIG. 2A. The determination information includes type determination information that indicates whether the detected traffic signal is a traffic signal or an instruction signal, and that it should be regarded as a traffic signal for driving support control even if it is an instruction signal. The red signal approach determination information shown is included.

  Then, the image processing unit 4 outputs the determination information generated as a result of the traffic signal error detection detection process to the driving support control unit 6 in step S6 of FIG. 2A. This determination information is used for alarm display and vehicle control as driving support control.

<3. Distance distribution judgment processing>
Details of the distance distribution determination process executed as step S10 in FIG. 2B are shown in FIG.
In step S100, the image processing unit 4 acquires traffic signal information. The traffic signal information is information on the emission color of the traffic signal detected in step S3 in FIG. 2 and information on the three-dimensional position from the host vehicle in the actual environment. Such information can be obtained from the captured image and the distance image.
There are a plurality of instruction signals for indicating the toll gate and the central line, which are the main detection targets, at positions where the front distance from the vehicle is equal. The distance distribution determination process uses this feature to distinguish between traffic signals and instruction signals according to the detected distance and number distribution of traffic signals.

Here, the non-traffic signal flag NSF is used in the traffic signal error detection suppression processing of FIG. 2B.
The non-traffic signal flag NSF is set to “SET” when it is determined that the signal to be judged is not a traffic signal at each time point, and “CLR (clear)” when it is determined to be a traffic signal. A flag whose value is set. If the determination is impossible, the “NEXT” value is set. Then, each time the non-traffic signal flag NSF = “SET” is set at each time point, the image processing unit 4 adds one point to a point held as an internal counter (hereinafter “NFS point”). When the non-traffic signal flag NSF = “CLR” is set, the counter (NFS point) is not incremented. When NSF = “CLR”, NSF point decrement or zero reset may be performed.

In step S101 in FIG. 3, the image processing unit 4 determines whether or not the number of detected traffic lights is one. If the number is 1, the process proceeds to step S102, and the non-traffic signal flag NSF is set to the “NEXT” value. Then, the distance distribution determination process in FIG.
In step S11 of FIG. 2B, it is determined whether or not the non-traffic traffic signal flag NSF is “NEXT”. However, in this case, it is not possible to determine the type of traffic signal in the distance distribution determination process in step S10. The process proceeds to step S12 (light emission shape determination process) and step S13 (bulletin board determination process).

Returning to FIG. When the detected number of traffic lights is plural, the process proceeds from step S101 to S103, and the image processing unit 4 performs distance group assignment.
In the distance grouping, traffic lights are grouped based on the forward distance from the host vehicle acquired in step S100. This group is called a distance group. A distance group is 15 groups (GP1-GP15) which divided | segmented the distance from 0 m-150m from the own vehicle for every 10 m, as shown, for example in FIG.
The image processing unit 4 distributes the detected plurality of traffic lights to any one of 15 distance groups depending on the distance from the host vehicle. In one distance group GP, traffic lights whose front distance from the host vehicle is within a predetermined distance in the front-rear direction are assigned. In addition, when a traffic signal far from 150 m or more is detected, it is all assigned to the distance group GP15.
At the same time, the image processing unit 4 counts the number of traffic lights assigned to each distance group.
In FIG. 4, a red signal is indicated by ●, and a blue signal (a traffic light in a blue light emission state) is indicated by a hatched circle. For example, FIG. 4A shows a state in which one red traffic light is assigned to the distance group GP7 and one red traffic light is assigned to the distance group GP10.

Subsequently, the image processing unit 4 performs arrow determination in step S104. In this process, an arrow signal is detected from the blue signal distributed to each distance group. If there is one or more green signals in the distance group, there is a possibility of an arrow signal, so a search is made for a red signal within a predetermined range from the green signal, for example, within a few meters around. If a red signal exists, it is determined that the blue signal is an arrow signal, and an arrow signal flag is set.
In step S105, the distance group distribution update is performed. In this process, the traffic signal set with the arrow signal flag in the arrow determination in step S104 is excluded from each distance group. Accordingly, information on the distance distribution of the traffic lights other than the arrow signals is obtained as the distance groups GP1 to GP15.

In step S106, the image processing unit 4 performs distribution pattern determination. In this process, the distribution pattern of the detected traffic signal is determined, and it is determined whether the latest traffic signal is a traffic signal or an instruction signal. In FIG. 4A to FIG. 4E, the distance group including the detected latest traffic signal is set as the latest group Ng. The latest group Ng is specified, and the determination is made by paying attention to the traffic lights included in the latest group Ng.
In the present embodiment, the distance groups GP1 to GP4 and GP15 are excluded from the processing targets. The distance group GP15 is not used because all signals with a length of 150 m or more are distributed and the group is meaningless and cannot be grouped. Since the distance groups GP1 to GP4 are too close to the host vehicle, there is often a case where a signal that has already been determined exists, or even if it is determined, the processing is not performed. The reason why the processing is performed on the nearest group Ng other than these is to make a determination on the traffic light that the host vehicle is about to pass through. Therefore, specifying the traffic signal included in the nearest group Ng having the closest forward distance from the host vehicle within the distance range to be processed (distance groups GP4 to GP14) is the traffic signal to be determined with the highest priority for the host vehicle. Will be specified.

In the distribution pattern determination, specifically, the number of traffic lights included in the latest group Ng, the color, and the traffic light interval are used as determination conditions.
As shown in FIG. 4A and FIG. 4B, when the number of traffic lights assigned to the latest group Ng is 1, it is determined that the traffic signal. FIG. 4A shows a case where one signal is detected most recently and another signal is detected a little further away. FIG. 4B shows a case where the green signal and the red signal are allocated to the latest group Ng, but are excluded in step S105 because the blue signal is an arrow. In such cases, the processing of FIG. 3 proceeds from step S107 to S109, and the non-traffic signal flag NSF = “CLR” is set. In other words, NFS points are not added.
As shown in FIG. 4E, when three or more traffic lights are assigned to the latest group Ng, it is determined that the traffic light is an instruction traffic light. In this case, the process proceeds from step S107 to S108, and the non-traffic signal flag NSF = “SET” is set. That is, 1 is added to the NSF point. The reason why there are three or more traffic lights in the nearest group Ng as the front distance of the host vehicle is that it can be determined to be a toll gate indication signal with high probability.

When two traffic lights are assigned to the nearest group Ng as shown in FIG. 4C and FIG. 4D, the process first proceeds from step S107 to S110, and the image processing unit 4 calculates the horizontal distance (traffic distance) between the two traffic lights. . In the case of traffic signals, most of them are installed at intervals wider than 5 m. Therefore, the interval between the two detected traffic lights is used as one element for determining whether the traffic signal or the instruction traffic light. The emission color acquired in step S100 is also used as a determination element. This is because, in the case of a traffic signal, it is normal that signals having the same distance from the same vehicle position have the same emission color.
Therefore, in step S111, the image processing unit 4 determines the type of the traffic signal based on the traffic signal interval and the emission color. That is, if the luminescent colors are the same and the traffic light interval is 5 m or more, it is determined as a traffic light, and the process proceeds to step S109 to set the non-traffic traffic light flag NSF = “CLR”. If this condition is not satisfied, it is determined that the signal is an instruction signal, and the process proceeds to step S108 to set the non-traffic signal flag NSF = “SET”.
Even if it is a traffic signal, there are rare cases where the distance between the two traffic signals is short, but this does not correspond to a red signal as driving support control by an erroneous detection suppression release determination process (S14) described later. This situation can be avoided.

<4. Light emission shape determination and bulletin board determination processing>
Next, the light emission shape determination processing in step S12 of FIG. 2B will be described with reference to FIG. 5A. When the non-traffic traffic signal flag NSF is set to “NEXT” in step S102 in the distance distribution determination process in FIG. 3 and the process proceeds from step S11 to S12 in FIG. 2B, that is, only one traffic signal is detected. Is a process performed when the type of traffic signal cannot be determined.
Some indicator traffic lights do not simply emit light with the shape ●, but emit light with the shape ◎ or shape X. In the light emission shape determination process, this feature is used to distinguish the traffic signal from the instruction signal by determining that it is lit in the shape of ◎ or ×.

In this light emission shape determination process, as step S200 of FIG. 5A, the image processing unit 4 acquires a signal area and color information on the image. The signal area is an area on the image corresponding to the light emitting portion of the traffic light, and is, for example, a square area indicated by a broken line in FIG. 6A. The emission shape observed in the signal area is the blue shape in FIG. 6C, the blue shape in FIG. 6D (hatched in the figure to distinguish it from red), the red × shape in FIG. 6F, and the red shape in FIG. 6G. ● There is a shape.
In order to discriminate these, sampling areas P1 to P5 are set as shown in FIG. 6A, and in step S201, the luminance of the pixels in each of the sampling areas P1 to P5 is sampled. Each sampling region P1 to P5 may be a one-pixel region or a block region having a plurality of pixels.

  In step S202, the light emission shape is determined from the luminance values of the sampling regions P1 to P5. In the case of blue light emission, the blue 青 shape in FIG. 6C and the blue ● shape in FIG. 6D can be discriminated from the luminance difference shown in FIG. 6E. That is, in the case of the blue ● shape, a high luminance value is obtained in each of the sampling regions P1 to P5, but in the case of the blue ◎ shape, the luminance value of the sampling region P5 is extremely low. In the case of red light emission, the red × shape in FIG. 6F and the red shape in FIG. 6G can be discriminated from the luminance difference shown in FIG. 6H. That is, in the case of the red shape, a high luminance value is obtained in each of the sampling regions P1 to P5. However, in the case of red × shape, the luminance value of the sampling regions P1 to P4 is extremely low.

  When a luminance pattern as a blue light emission shape or a red × shape is observed as the light emission shape, the traffic light can be determined as an instruction traffic light. Therefore, the image processing unit 4 proceeds to step S205 and sets the non-traffic traffic signal flag NSF = “SET” when the luminance pattern satisfies the condition of blue shape or red × shape in step S203. If the luminance pattern condition of blue ◎ shape or red × shape is not satisfied, the non-traffic traffic signal flag NSF = “CLR” is set in step S204. In this case, the determination as an instruction signal is reserved rather than the determination as a traffic signal.

Note that sampling of the luminance values of the sampling areas P1 to P5 for light emission shape determination can correctly cope with blinking of an LED (Light Emitting Diode) traffic light. The traffic signal determination process in FIG. 2A is performed in units of frame image data. For this reason, there is a case where the image to be processed corresponds to a timing at which the LED traffic light is not emitted due to the relationship between the system processing cycle and the blinking cycle of the LED signal. is there. In such a case, the luminance values of the sampling areas P1 to P5 are all extremely low. In this case, it is determined in step S203 that the luminance pattern condition is not satisfied (the LED signal is not captured), and the non-traffic signal flag NSF is not set to the “SET” value. Thereby, it can prevent making a misjudgment with an instruction | indication signal apparatus.
Further, the sampling areas P1 to P5 may be set as shown in FIG. 6B, for example. Also in this case, it is possible to distinguish between the blue shape and the blue shape and the red × shape and the red shape according to the luminance pattern.
Further, the sampling area may be set to at least one and set at a position where the luminance varies depending on the light emission shape. For example, if only one is set at an intermediate position between the sampling areas P1 and P5 in FIG. 6A, it is possible to distinguish the shape ●, shape ◎, and shape X from the luminance value. Further, for example, even in the two sampling regions P3 and P5, it is possible to distinguish between ● shape, ◎ shape, and × shape. As described above, the sampling area can be set in various ways. However, more accurate shape determination can be performed by setting a plurality of sampling areas.
In addition, in determining the luminance values of the sampling regions P1 to P5 as shown in FIGS. 6A and 6B, it is only necessary to determine whether the luminance is high or low by actually using a predetermined threshold as a comparison reference. The threshold value may be a constant value regardless of red light emission and blue light emission, but different threshold values may be used for the blue light emission state and the red light emission state in consideration of the difference in luminance between red and blue.

Subsequently, bulletin board determination processing is performed as step S13 of FIG. 2B.
In the case of a toll gate on the toll road that is the main detection target as an instruction signal, there are often signs indicating the state of the toll gate above the signal. Therefore, in the bulletin board determination process, it is determined whether or not it is a toll gate instruction signal using the distance and color characteristics of a bulletin board such as a sign.

The bulletin board determination process as step S13 is shown in FIG. 5B. The image processing unit 4 determines the bulletin board position in step S300. In this process, the size of the space above the detected traffic light is confirmed. The bulletin board search range is determined according to the size of the upper space. If the upper space is too wide or too narrow, the bulletin board search range is changed.
FIG. 7 shows a state in which the bulletin board 71 is installed above the toll gate instruction signal 70. A broken line SA1 is an example of a determination result of the space above the traffic light. Based on this result, for example, the bulletin board search range indicated by the broken line SA2 is determined.
The bulletin board search range SA2 may be determined based on the bulletin board size of the actual toll road toll gate. The relationship between the traffic light upper space SA1 and the bulletin board search range SA2 is also preferably determined based on the bulletin board actually installed.

In step S301, the image processing unit 4 checks the distance information of the determined bulletin board search position SA2. That is, the distance information for each pixel in the bulletin board search range SA2 is referred to from the distance image, and the distance information for each pixel is compared with the front distance of the traffic light from the vehicle to determine whether the distance is the same. . Strictly speaking, even if there is a slight difference in distance information, it may be determined that the distance information is the same if it is within a predetermined difference.
If the object (pixels that copied the object) included in the bulletin board search range SA2 is the same distance (including substantially the same) as the traffic signal at a certain ratio or more, there is an object at the same distance as the traffic signal in step S302. And the process proceeds to step S304.
On the other hand, if the part of the same distance as the traffic signal is less than a certain value, there is no equidistant object (bulletin board) around the traffic signal, so it cannot be determined as an instruction traffic signal, and the process proceeds from step S302 to S303. The flag NSF is set to “CLR”. That is, it means that the determination with the instruction signal is reserved.

If it is determined that there is an object that appears to be a bulletin board at the same distance as the traffic light and the process proceeds to step S304, the image processing unit 4 performs bulletin board position color check. That is, the color information of the bulletin board search range SA2 determined in step S300 is checked. That is, referring to the color information of each pixel in the bulletin board search range SA2, a process is performed to compare whether the color information of each pixel in the bulletin board search range SA2 is predetermined color information.
The colors to be searched are green for a general gate, purple for an ETC gate, and white for a closed gate. If the search color (either green, purple, or white, which is the color of the bulletin board) is included in the bulletin board search range SA2 at a certain ratio or higher, in step S305, the objects in the bulletin board search range SA2 are displayed. The toll road bulletin board is determined, and the non-traffic signal flag NSF = “SET” is set in step S306. That is, it is determined as an instruction signal for a toll road toll gate or the like. If the search color is not included at a certain ratio or higher, it cannot be determined that the traffic signal is an instruction traffic light such as a toll road toll gate or the like, the process proceeds from step S305 to S303, and the non-traffic traffic signal flag NSF = “CLR”.

The bulletin board determination process may be performed only for a red signal or only for a blue signal in consideration of the calculation amount.
In the signboard position distance check in step S301, a fixed ratio for determining whether the distance is the same as the traffic signal is 60% or more, for example, and in the signboard position color check in step S304, a fixed ratio for determining the search color is set, for example. Although 20% to 40% can be considered, an appropriate ratio can be arbitrarily set.

<5. Error detection suppression release determination process and error detection suppression determination process>
Next, the erroneous detection suppression cancellation determination process in step S14 and the erroneous detection suppression determination process in step S16 of FIG. 2B will be described with reference to FIGS. 8A and 8B, respectively.
The erroneous detection suppression determination process in step S16 is based on the determination with the instruction signal, that is, the determination of the non-traffic signal flag NSF = “SET” in each of the above-described processes. It is a process which produces | generates the determination result of the classification of a traffic signal. In other words, this is a process for ensuring the reliability of the determination result as the instruction signal device.
On the other hand, the erroneous detection suppression release determination process in step S14 is a process for canceling the result of the “false detection suppression determination process”, and even when the determination with the instruction signal is made, the determination with the instruction signal is canceled in a specific case. In this process, driving assistance equivalent to that for traffic signals is performed.

  First, the erroneous detection suppression determination process in step S16 will be described with reference to FIG. 8B. When the light emission shape determination process in step S12 and the bulletin board determination process in step S13 in FIG. 2B are performed and the process proceeds to step S16, or the distance distribution determination process in step S10 is performed, and an erroneous detection suppression release determination in step S14 described later is performed. When the process proceeds to step S16 without setting the cancel flag CF in the process, an erroneous detection suppression determination process as shown in FIG. 8 is performed.

In step S500, the image processing unit 4 calculates the determination results so far. Specifically, the NFS point added by the internal counter is checked according to the non-traffic signal flag NSF = “SET”. If the NFS point is equal to or greater than the threshold value in step S501, the process proceeds to step S503 to set the type determination information RSL = “instruction signal”. If the NFS point is less than the threshold value, the process proceeds to step S502, where the type determination information RSL = “traffic signal” is set.
The threshold value is set to an appropriate value such as “5”.
As described above, the processing of FIG. 2A is repeated for the current frame image data. When an instruction signal is detected, the NFS point is added one or more times in one frame processing. . Only when the NSF point becomes equal to or greater than the threshold value in the process of repeating the process, the type determination information RSL = “instruction signal” is set. Thereby, the reliability in the case of being determined as the instruction traffic light is improved.

The image processing unit 4 transmits such type determination information RSL to the driving support control unit 6 in step S6 of FIG. 2A.
Thus, the accuracy of the instruction signal determination is increased because the determination information with the instruction signal is transmitted to the driving support control unit 6 in a state where the determination accuracy is low, and driving support (stop control or the like) according to the signal detection is performed. This is to avoid being not executed.

Regarding the distance distribution determination processing in step S10 of FIG. 2B, the non-traffic signal flag NSF is commonly set for all signals detected as the latest group Ng at that time. In the light emission shape determination process in step S12 and the bulletin board determination process in step S13, the non-traffic signal flag NSF is individually set for the detected traffic signal.
Therefore, when the distance distribution determination process is performed and the process proceeds to step S16, the type determination information RSL is generated in common for all the signals detected as the latest group Ng. When the light emission shape determination process and the bulletin board determination process are performed, type determination information RSL is generated for each traffic light.

The erroneous detection suppression release determination process in step S14 of FIG. 2B will be described with reference to FIG. 8A. This is executed by proceeding from step S11 to S14 when it is determined in the distance distribution determination process of step S10 whether or not it is an instruction signal. And even if it is determined that the false detection suppression release determination process is an instruction traffic light in the distance distribution determination process (S10), driving assistance equivalent to a traffic signal is performed only when traveling from the front to the red signal. Therefore, the process does not proceed to step S16 (that is, the type determination information RSL as an instruction signal is not output).
In short, even if it is determined to be an instruction traffic light, it is often impossible to enter when it is lit red, so it is more appropriate to treat it as a traffic light. Therefore, only when a certain distance or more is approached while proceeding toward the red light, it has the meaning of canceling the instruction traffic light determination and re-determination as a traffic light.

In step S400 of FIG. 8A, the image processing unit 4 acquires traffic signal information. In this case, left-right distance information indicating how many meters the traffic light is deviated from the center of the host vehicle and color information are acquired.
In step S402, the emission color is determined. If the color is not red, the process is terminated in step S403 with the clear flag CF = “CLR (clear)”. In this case, the erroneous detection suppression determination process in step S16 is subsequently performed.

If the emission color is red, in step S404, the image processing unit 4 confirms whether or not a red signal exists in front of the host vehicle as the host vehicle traveling direction determination. That is, it is confirmed whether or not a red signal is detected within the width Wm of the host vehicle 91 in FIG. 9 on the captured image data. Furthermore, the distance Dx with the own vehicle is confirmed.
Note that whether or not the vehicle 91 is within the width Wm may be determined as whether or not it is a red signal within 1 m from the center of the vehicle, for example, but the range is arbitrarily set. It is possible.

  When the red signal is not within the width Wm of the own vehicle 91 or when the distance Dx is not within the predetermined distance, it is not determined that the own vehicle is traveling toward the red signal, and the process proceeds from step S405 to S403. The process is ended by setting the clear flag CF to “CLR”.

  If a red signal is detected within the width Wm of the host vehicle 91 and the distance Dx is within a predetermined distance, it is determined that the host vehicle is traveling toward the red signal. In this case, the process proceeds from step S405 to S406, and the process is terminated with the cancel flag CF = “SET”. In this case, step S16 is skipped by the branch process of step S15 in FIG. 2B.

That is, even if the erroneous detection suppression release determination process is performed in step S14, if the cancel flag CF = “CLR”, the erroneous detection suppression determination process is performed in step S16, and the type determination information RSL is generated, and FIG. In step S6, the type determination information RSL indicating whether the signal is a traffic signal or a traffic signal is output to the driving support control unit 6.
On the other hand, when the cancel flag CF = “SET”, the erroneous detection suppression determination process in step S16 is not performed. In this case, the determination information output to the driving support control unit 6 in step S6 in FIG. Information of the cancel flag CF = “SET”, that is, red signal approach determination information (information obtained by re-determination of the instruction signal signal determination as a traffic signal).

  The driving support control unit 6 performs driving support control according to the determination information (type determination information RSL or red signal approach determination information) supplied in the process of step S6 of the image processing unit 4. For example, when the type determination information RSL = “traffic signal” or when the red light approach determination information is supplied, stop control and alarm control are executed. When the type determination information RSL = “instruction signal”, stop control and alarm control are not executed.

  Note that the processing in FIG. 8A may be performed only when it is determined as an instruction signal (non-traffic signal flag NSF = “SET”) in the immediately preceding step S10, but should be performed regardless of the determination in step S10. Also good.

<6. Summary>
In this Embodiment, the following effects are acquired by performing the above process in the image processing part 4 of the environment recognition unit 2 outside a vehicle.
First, there is an effect that accurate driving support control can be performed by obtaining determination information for distinguishing between traffic signals and instruction signals. In many cases, the instruction traffic light emits red light, and if it is mistakenly processed as a traffic signal, it is recognized as a red traffic light, which adversely affects subsequent driving support control processing. For example, at the toll gate, if the indicator signal in the lane adjacent to the own vehicle lane emits red light, it may occur that the stop signal is controlled as a red signal. On the other hand, in this embodiment, a traffic signal and an instruction signal can be distinguished. As a result, the driving support control unit 6 can realize accurate driving support control.

Further, in the present embodiment, the type determination of whether the detected traffic signal is a traffic signal or an instruction signal is performed using the distance distribution of the traffic signal. The traffic light is similar in size and installation position to the traffic light, and it is difficult to distinguish by simple light emitter detection from the captured image, but it is possible to determine the traffic light and the traffic light with high accuracy by distance distribution determination It becomes.
In particular, the traffic signal type determination unit 4c of the image processing unit 4 acquires the light emission color information of the traffic signal and the three-dimensional position information of the traffic signal from the own vehicle, and among the traffic signals detected by the traffic signal detection unit 4b, the specific traffic signal and the specific traffic signal are detected. The number of other traffic signals existing within a predetermined front-rear distance of the traffic signal (the same distance group GP), the horizontal distance between the traffic signals when there is another traffic signal, and the mutual light emission of the traffic signal and the other traffic signal Based on the color information, it is determined whether the detected traffic light is a traffic light or an instruction traffic light. This realizes highly accurate traffic signal type determination using the characteristics of the traffic signal and the instruction signal on the arrangement and light emission operation.

When two or more traffic lights are detected, the traffic light with the closest forward distance from the host vehicle is identified within the distance range to be processed, and within the predetermined distance before and after the identified traffic light (the nearest distance group GP) In the group Ng), it is determined whether or not there is another traffic light having the same color as that of the traffic light. If there is another traffic light of the same color, the horizontal distance between the traffic light and the other traffic light is a predetermined interval. At the above time, by determining the traffic signal and other traffic signals as traffic signals, it is possible to make a determination based on the installation characteristics of the traffic signal. This is because traffic signals having the same emission color are rarely installed on the road.
On the other hand, when the horizontal distance is less than the predetermined interval, the traffic signal and other traffic signals are determined as instruction traffic signals. Thereby, for example, it is possible to make a determination based on the characteristics of an instruction signal arranged at a relatively narrow interval such as an instruction signal at a tollgate.
Then, from the information obtained as the distance distribution, by identifying and determining the traffic signal with the closest forward distance from the host vehicle within the distance range to be processed, the traffic signal to be determined with the highest priority for the host vehicle The determination is made and is suitable as a determination for driving support.
In addition, when all three or more traffic signals are present within a predetermined distance in the front and rear (in the latest group Ng), it is also possible to determine the three or more traffic signals as instruction signal devices with accuracy based on the characteristics of the instruction signal device at the toll gate. High judgment.

  In addition, when two or more traffic lights including a blue light emitting signal are detected and there is a red light emitting signal within a predetermined range from the blue light emitting signal, the blue light emitting signal is designated as a traffic signal. The traffic light type is determined based on the distance distribution from the host vehicle. As a result, the arrow signal is excluded, and the determination accuracy based on the distance distribution can be improved.

When the number of detected traffic lights is one and distance distribution cannot be determined, a shape specific to the indication traffic light is determined as the light emission shape. The determination result of the light emission shape is also used for determining the type of the traffic light. As a result, it is possible to determine the instruction signal with high accuracy.
Particularly in the light emission shape determination, by setting the sampling regions (P1 to P5) as described above, it is possible to easily determine the ◎ shape, the X shape, or the ● shape using the luminance value of each sampling region.
Similarly, if the number of detected traffic signals is one and the distance distribution cannot be determined, as a bulletin board determination process, is there a bulletin board around the detected traffic signal where the distance from the host vehicle is determined to be the same as the traffic signal The determination result of the presence of the bulletin board is also used for determining the type of the traffic signal. This makes it possible to determine the toll gate instruction signal with high accuracy.
As for the bulletin board determination, as in the above-described example, the distance information for each pixel in the bulletin board search range SA2 set above the traffic light in the captured image is referred to, and the distance information for each pixel and the vehicle of the traffic light The first comparison result comparing the forward distance from the second reference and the color information of the bulletin board search range SA2 and the second comparison comparing whether the color information of the bulletin board search range SA2 is predetermined color information By determining whether or not a bulletin board-like object exists above the traffic light based on the result, it is possible to make a determination based on the position and color characteristics of the bulletin board attached to the instruction traffic light.

  In addition, by the erroneous detection suppression determination process (S16, S500 to S503 in FIG. 8B), the NSF point corresponding to the determination with the instruction signal device in the distance distribution determination process, the light emission shape determination process, and the bulletin board determination process becomes equal to or greater than the threshold. For the first time, the type determination information RSL = “instruction signal” is set. Thereby, the determination accuracy as an instruction | indication signal apparatus can be raised significantly.

In addition, even when a determination as an instruction signal is made in the distance distribution determination process by the erroneous detection suppression release determination process (S14, S400 to S406 in FIG. 8A), the vehicle travels toward the red signal and approaches within a predetermined distance. Is detected, the red light approach determination information is output so that the driving support control similar to the traffic signal is performed.
That is, the traffic light type determination unit 4c is in a red light emitting state in which the instruction signal is present within a predetermined left-right distance from the center in the width direction of the host vehicle, at least when the instruction signal is determined from the determination result of the distance distribution. When it is detected that the front distance to the instruction signal is approaching within a predetermined distance, the instruction signal is re-determined as a traffic signal, and red signal approach determination information is output.
Thereby, even if it is an instruction | indication signal, required driving assistance control (stop control, an alarm, etc.) can be performed and driving safety can be improved more.
Further, even if an erroneous determination is made in the distance distribution determination process (S10, steps S100 to S109 in FIG. 3), appropriate driving support control can be performed. For example, in an environment where two traffic signals are installed on a narrow road, there is a possibility of erroneous determination as an instruction signal in the distance distribution determination. However, in this case as well, the driving support control similar to the traffic signal can be executed by the red signal approach determination information.

  By performing each of the above processes according to the situation, detection performance (robustness) can be improved, and safety can be further improved.

The outside environment recognition device of the present invention is not limited to the configuration and processing of the embodiment, and various modifications can be considered. For example, in the process of FIG. 2B, only one of the light emission shape determination process (S12) and the bulletin board determination process (S13) may be performed. Further, the erroneous detection suppression release determination process (S14) may be performed after the light emission shape determination process (S12) and the bulletin board determination process (S13) are performed.
In FIG. 1, the driving support control unit 6 is shown in the outside environment recognition unit 2, but the driving support control unit 6 may be provided outside the outside environment recognition unit 2. The driving support control unit 6 may be integrated with the vehicle control unit 10. In addition, the vehicle control unit 10 may actually be configured by a plurality of microcomputers such as an engine ECU (electronic control unit), a brake ECU, a transmission ECU, and the like. Further, the display control unit 11 and the vehicle control unit 10 may be integrated.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle control system, 2 ... Outside environment recognition unit, 3A, 3B ... Imaging part, 4 ... Image processing part, 4a ... Distance image generation part, 4b ... Traffic signal detection part, 4c ... Traffic signal type determination part, 5 ... Memory part , 6 ... Driving support control unit, 10 ... Vehicle control unit, 11 ... Display control unit, 12 ... Actuator, 13 ... Display, 14 ... Sensors

Claims (12)

A traffic light detector for detecting the presence of a traffic light in a captured image obtained by capturing the traveling direction of the host vehicle;
Using the distance distribution from the host vehicle with respect to the traffic signal detected by the traffic signal detection unit, it is determined whether the detected traffic signal is a traffic signal or an instruction traffic signal, and determination information indicating a classification determination result is provided. A vehicle exterior environment recognition device comprising: a traffic signal type determination unit to be generated. The outside environment recognition device according to claim 1, wherein the traffic light type determination unit further determines a light emission shape of the traffic light detected in the captured image, and uses a light emission shape determination result for the type determination. The traffic signal type determination unit further determines whether or not there is a bulletin board-like object whose front distance from the host vehicle is determined to be the same as the traffic signal around the detected traffic signal. The outside environment recognition apparatus according to claim 1 or 2, wherein the determination result is also used for the type determination. The traffic light type determination unit obtains the light emission color information of the traffic light and the three-dimensional position information of the traffic signal from the host vehicle, and within a predetermined distance before and after the specific traffic light among the traffic lights detected by the traffic light detection unit. The traffic signal detected is a traffic signal based on the number of other traffic signals present in the traffic signal, the horizontal distance between the traffic signals in the presence of other traffic signals, and the color information of the other traffic lights. The external environment recognition device according to claim 1, wherein a type determination is made as to whether there is an instruction signal or not. The traffic light type determination unit obtains the light emission color information of the traffic light and the three-dimensional position information of the traffic signal from the own vehicle, and when two or more traffic lights are detected by the traffic light detection unit, within the distance range to be processed. Identify the traffic signal with the closest distance from the host vehicle, determine whether there is another traffic signal of the same color as the traffic signal within the predetermined distance before and after the identified traffic signal, and determine the other traffic signal of the same color. When the horizontal distance between the traffic signal and another traffic signal is greater than or equal to a predetermined interval, the traffic signal and the other traffic signal are determined as traffic signals and the horizontal distance between the traffic signal and the other traffic signal is predetermined. The outside environment recognition device according to claim 1 or 4, wherein when the distance is less than the interval, the traffic signal and the other traffic signal are determined as instruction traffic signals. The traffic signal type determination unit acquires a three-dimensional position of the traffic signal from the own vehicle, and when the traffic signal detection unit detects three or more traffic signals, all the three or more traffic signals exist within a predetermined distance in the front and rear. 6. The vehicle exterior environment recognition device according to claim 1, wherein the three or more traffic signals are determined as instruction traffic signals. The traffic light type determination unit obtains the light emission color information of the traffic light and the three-dimensional position information of the traffic light from the own vehicle, and the traffic light detection unit detects two or more traffic lights including at least a blue light emitting traffic light, and the blue light When there is a red light emitting signal within a predetermined range from the light emitting signal, the blue light emitting signal is excluded from the traffic signal and the traffic light type is determined based on the distance distribution from the host vehicle. The outside environment recognition device according to any one of claims 1, 4, 5, and 6. The traffic light type determination unit is in a red light emission state in which the signal traffic light is present within a predetermined left-right distance from the center in the width direction of the host vehicle, at least when the traffic light type determination unit is determined as the command traffic light from the determination result of the distance distribution. The outside environment recognition device according to any one of claims 1 to 7, wherein when it is detected that the front distance to the instruction signal is approaching within a predetermined distance, the instruction signal is re-determined as a traffic signal. The traffic light type determination unit obtains light emission color information and luminance information of the traffic signal, and when there is one traffic signal detected by the traffic signal detection unit, the brightness of at least one sampling region set within a predetermined region of the traffic signal The vehicle exterior environment recognition apparatus according to claim 2, wherein the type is used to determine whether the signal is a traffic signal or an instruction signal. The predetermined area is a rectangular area surrounding a light emitting portion of the traffic light, and the sampling area is a central area set at a central portion of the rectangular area and an intermediate area set at an intermediate portion of at least one of four sides The vehicle exterior environment recognition device according to claim 9, wherein the type determination as to whether the traffic signal is a traffic signal or an instruction signal is performed based on the luminance value of each sampling region and the emission color information of the traffic signal. The traffic signal type determination unit refers to the distance information for each pixel in the bulletin board search range set above the traffic signal in the captured image when there is one traffic signal detected by the traffic signal detection unit. Whether the color information of the bulletin board search range is predetermined color information by referring to the first comparison result comparing the distance information of the traffic light and the forward distance of the traffic light from the own vehicle and the color information of the bulletin board search range. The vehicle exterior environment recognition apparatus according to claim 3, wherein it is determined that a bulletin board-like object is present above the traffic light based on a second comparison result for comparing whether or not, and the traffic light is determined as an instruction traffic light. The driving support control part which performs brake control or alarm control based on the luminescent color information and distance information of the traffic signal when the traffic signal type determination part determines that the traffic signal is a traffic signal. The vehicle environment recognition apparatus according to any one of claims 11 to 11.

Images