JP6085522B2 - Image processing device - Google Patents

Description

  The present invention relates to a technical field of an image processing apparatus that performs image processing on a captured image obtained by imaging the front of a host vehicle.

Japanese Patent No. 4666049

As headlight light distribution control, so-called AHB (Auto High Beam) and ADB (Adaptive Driving Beam) are known. In performing these headlight light distribution controls, it is required to recognize a preceding vehicle and an oncoming vehicle that are in front of the host vehicle. If a high beam is irradiated to the preceding vehicle and the oncoming vehicle, the driver of the preceding vehicle and the oncoming vehicle may be dazzled and hinder driving. This is to prevent this.
In order to detect the preceding vehicle, it is conceivable to detect the tail lamp of the preceding vehicle on a captured image obtained by imaging the front of the host vehicle.
Patent Document 1 discloses a technique for identifying a light emitter such as a headlight of a vehicle and a reflector.

For detection of a preceding vehicle, it is effective to detect a light emitter as a tail lamp on a captured image. For example, a high luminance area on the captured image is detected as a light emitter. However, the high-intensity area detected on the captured image may actually be a sign or the like that reflects light such as a headlight.
As a technique for separating the light emitter and the reflector, in Patent Document 1 described above, based on the difference in how the light emitter and the reflector are reflected, the frequency distribution tendency of the target luminance (the frequency decreases as the luminance increases from low to high). Or the like). This method is effective in a scene where the target has a sufficient number of pixels (that is, the distance is short), but when the number of pixels is small, the frequency distribution does not become a reliable number of frequencies and does not work. Moreover, since the frequency distribution processing is performed for each target, the processing amount is also an issue.

  Therefore, an object of the present invention is to realize tail lamp detection / identification with high accuracy and with a small processing load.

1stly, the image processing apparatus which concerns on this invention is the imaging part which acquires the captured image which imaged the front of the own vehicle, the red area detection process which detects a red area in the said captured image, and the said red area detection process A determination process for obtaining a divergence value between a high luminance representative value and a low luminance representative value among luminance values of a plurality of pixels included in the detected red region, and determining a magnitude relationship between the divergence value and a threshold value is performed. And an image processing unit that uses the result of the determination processing for identification processing as to whether or not the red region is a tail lamp image of a preceding vehicle.
In this case, the preceding vehicle detection first extracts the red region of the captured image. By targeting the red region, the influence of light emitters other than red is excluded, and the processing range on the image and the luminance range that is the red detection range are limited.
However, the detected red area is not necessarily the tail lamp of the preceding vehicle, and there is an area where the headlight of the own vehicle is reflected by a sign having red color such as a speed limit.
Here, a self-luminous light source such as a tail lamp of a vehicle tends to have a wide luminance distribution because light diffuses, while a light distribution reflected by a headlight of the own vehicle on a sign or the like tends to have a narrow luminance distribution. It is in. For this reason, the difference value between the high luminance representative value and the low luminance representative value in the red region is different between the tail lamp light and the reflected light. Using this, tail lamp detection is performed.

Second, in the above-described image processing apparatus according to the present invention, it is preferable that the image processing unit changes the threshold value in accordance with the value of the high luminance representative value.
That is, considering that the divergence value between the high luminance representative value and the low luminance representative value increases or decreases under the influence of the luminance value, the threshold value to be compared with the divergence value is changed according to the value of the high luminance representative value.

Thirdly, in the above-described image processing apparatus according to the present invention, it is desirable that the image processing unit executes the identification process during a period in which the headlight of the host vehicle is in a high beam state.
This is because the reflected light of the marker lamp mainly appears on the image during the high beam period.

Fourthly, in the above-described image processing apparatus according to the present invention, it is preferable that the image processing unit sets a tail lamp detection range in the captured image and detects a red region within the tail lamp detection range.
That is, the red region detection is performed by limiting the detection range that can exist as a tail lamp.

Fifth, in the above-described image processing device according to the present invention, the image processing unit adds or subtracts the reliability that the red region is the tail lamp image of the preceding vehicle in the identification processing according to various conditions, It is desirable to identify whether the red region is a tail lamp image of a preceding vehicle according to the reliability value, and to subtract or add the reliability value according to the result of the determination process.
The reliability of tail lamp identification is improved by adjusting the reliability value according to various conditions. In this case, the comparison result of the deviation value with the threshold value is reflected in the reliability value.

  According to the present invention, it is possible to appropriately identify the tail lamp of the preceding vehicle on the image by processing with less burden.

It is the figure which showed the structure of the vehicle control system of embodiment. It is a figure for demonstrating the image processing performed in embodiment. It is the figure which showed the example of the bright image and the dark image. It is explanatory drawing about a tail lamp detection range, a headlight detection range, and a streetlight detection range. It is the flowchart which showed the whole flow of each process which concerns on embodiment. It is a flowchart for demonstrating a tail lamp detection process. It is explanatory drawing of an element group and an object group. It is a flowchart for demonstrating the whole flow of object recognition and identification processing. It is the flowchart which showed the flow of the object area | region calculation process. It is the flowchart which showed the flow of the object group tracking process. It is the flowchart which showed the flow of the object identification process. It is the flowchart which showed the content of the control information calculation process. It is explanatory drawing about the irradiation aspect of the high beam implement | achieved according to the control information showing high beam ON other than object. It is explanatory drawing of the relationship between the oncoming vehicle and headlight detection range at the time of proximity | contact.

<1. Overall system configuration>
FIG. 1 shows a configuration of a vehicle control system 1 including an image processing apparatus as an embodiment according to the present invention. In FIG. 1, only the configuration of the main part according to the present invention is extracted and shown from the configuration of the vehicle control system 1.
The vehicle control system 1 includes an imaging unit 2, an image processing unit 3, a memory 4, a driving support control unit 5, a display control unit 6, an engine control unit 7, a transmission control unit 8, and a brake control unit provided for the host vehicle. 9. Light control unit 10, display unit 11, engine-related actuator 12, transmission-related actuator 13, brake-related actuator 14, headlight 15, ADB (Adaptive Driving Beam) actuator 16, sensor / operator 17 and bus 18. I have.

  The imaging unit 2 includes a first camera unit 2A and a second camera unit 2B that are installed so as to be able to capture the traveling direction (front) of the vehicle. The first camera unit 2A and the second camera unit 2B are arranged, for example, at a predetermined interval in the vehicle width direction in the vicinity of the upper part of the windshield of the host vehicle so that distance measurement by a so-called stereo method is possible. The optical axes of the first camera unit 2A and the second camera unit 2B are parallel, and the focal lengths are the same. Also, the frame periods are synchronized and the frame rates are the same. The number of pixels of the image sensor is, for example, about 640 × 480.

Electrical signals (captured image signals) obtained by the image sensors of the first camera unit 2A and the second camera unit 2B are each A / D converted, and digital image signals (imaging images) representing luminance values with a predetermined gradation in pixel units. Image data). In the present embodiment, these captured image data are color image data, and therefore, three data (luminance values) of R (red), G (green), and B (blue) are obtained for each pixel. The gradation of the luminance value is, for example, 256 gradations.
Hereinafter, the captured image data obtained by the first camera unit 2A is referred to as “first captured image data”, and the captured image data obtained by the second camera unit 2B is referred to as “second captured image data”.

  The imaging unit 2 in this example has an automatic adjustment function for the shutter speed and gain (ISO sensitivity) of the first camera unit 2A and the second camera unit 2B. The imaging unit 2 is also capable of adjusting the shutter speed and gain of the first camera unit 2A and the second camera unit 2B based on an instruction from the image processing unit 3.

The image processing unit 3 is configured by a microcomputer including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) as a work area, and various types of programs according to programs stored in the ROM. Execute the process.
The image processing unit 3 stores the first captured image data and the frame image data as the second captured image data obtained by the imaging unit 2 imaging the front of the host vehicle in the memory unit 4. Based on the first captured image data and the second captured image data of each frame, various processes for recognizing and identifying an object existing in front of the vehicle as an external environment are executed.
Details of specific processing executed by the image processing unit 3 will be described later.

The driving support control unit 5 is composed of, for example, a microcomputer, and is based on the results of image processing by the image processing unit 3, detection information obtained by the sensor / operator 17 and operation input information, and the like. Execute control processing.
The driving support control unit 5 is connected to each control unit of the display control unit 6, the engine control unit 7, the transmission control unit 8, the brake control unit 9, and the light control unit 10, which are also configured by a microcomputer, via the bus 18. Thus, data communication can be performed with each of these control units. The driving support control unit 5 instructs a necessary control unit among the above control units to execute an operation related to driving support.
In the case of the present embodiment, the driving support control unit 5 performs light distribution control for the headlight 15. In the drawing, the processing function for the light distribution control of the driving support control unit 5 is represented by a functional block as “light distribution control processing unit 5A”. The light distribution control processing unit 5A instructs the light control unit 10 for ADB control based on the control information generated from the recognition / identification result of the oncoming vehicle, the preceding vehicle, the streetlight, and the like by the image processing unit 3.

The sensors / operators 17 comprehensively represent various sensors and operators provided in the host vehicle. Sensors / operators 17 include, for example, an engine speed sensor, an intake air amount sensor that detects the intake air amount, an accelerator opening sensor that detects the accelerator opening amount from the depression amount of the accelerator pedal, and an intake passage. A throttle opening sensor that detects the opening of a throttle valve that adjusts the amount of intake air supplied to each cylinder of the engine, a water temperature sensor that detects the cooling water temperature that indicates the engine temperature, and an outside air temperature sensor that detects the temperature outside the vehicle Etc.
In addition, as an operator, an ignition switch for instructing start / stop of the engine, selection of an automatic transmission mode / manual transmission mode in an AT (automatic transmission) vehicle, and an instruction for up / down in the manual transmission mode are given. There are a select lever for performing, a display changeover switch for switching display information in an MFD (Multi Function Display) provided in the display unit 11 described later, and the like.
Particularly in the case of the present embodiment, the sensor / operator 17 is provided with a vehicle speed sensor 17A, a rudder angle sensor 17B, an accelerator opening sensor 17C, a headlight switch 17D, and a winker switch 17E. The headlight switch 17D represents an operator for instructing ON / OFF of the low beam of the headlight 15 and ON / OFF of the high beam. Here, in the case of this example, the ADB function is also turned ON / OFF according to the ON / OFF operation of the high beam.

  The display unit 11 comprehensively represents various meters such as a speedometer and a tachometer provided in a meter panel installed in front of the driver, an MFD, and other display devices for presenting information to the driver. Various information such as the total travel distance of the host vehicle, the outside air temperature, and instantaneous fuel consumption can be displayed on the MFD simultaneously or by switching.

  The display control unit 6 controls the display operation by the display unit 11 based on a detection signal from a predetermined sensor in the sensor / operation element 17, operation input information by the operation element, and the like.

The engine control unit 7 controls various actuators provided as the engine-related actuator 12 based on a detection signal from a predetermined sensor in the sensor / operator 17 or operation input information by the operator. As the engine-related actuator 12, for example, various actuators related to engine driving such as a throttle actuator that drives a throttle valve and an injector that performs fuel injection are provided.
For example, the engine control unit 7 performs engine start / stop control according to the operation of the ignition switch described above. The engine control unit 7 also controls the fuel injection timing, the fuel injection pulse width, the throttle opening, and the like based on detection signals from predetermined sensors such as an engine speed sensor and an accelerator opening sensor.

The transmission control unit 8 controls various actuators provided as the transmission-related actuator 13 based on a detection signal from a predetermined sensor in the sensor / operator 17 or operation input information by the operator. As the transmission-related actuator 13, there are provided various transmission-related actuators such as a control valve that performs shift control of an automatic transmission and a lockup actuator that locks up a lockup clutch.
For example, when the automatic shift mode is selected by the select lever described above, the transmission control unit 8 performs a shift control by outputting a shift signal to the control valve according to a predetermined shift pattern. Further, when the manual shift mode is set, the transmission control unit 8 performs a shift control by outputting a shift signal according to a shift up / down instruction from the select lever to the control valve.

The brake control unit 9 controls various actuators provided as the brake-related actuator 14 based on a detection signal from a predetermined sensor in the sensor / operator 17 or operation input information by the operator. As the brake-related actuator 14, for example, various brake-related actuators such as a hydraulic pressure control actuator for controlling the hydraulic pressure output from the brake booster to the master cylinder and the hydraulic pressure in the brake fluid piping are provided.
For example, when an instruction to turn on the brake is issued from the driving support control unit 5, the brake control unit 9 controls the hydraulic pressure control actuator to brake the host vehicle. Further, the brake control unit 9 calculates a wheel slip ratio from detection information of a predetermined sensor (for example, an axle rotation speed sensor or a vehicle speed sensor), and increases or decreases the hydraulic pressure by the hydraulic pressure control actuator according to the slip ratio. By so doing, so-called ABS (Antilock Brake System) control is realized.

The light control unit 10 controls turning on / off of the headlight 15 and control of the ADB actuator 16 based on a detection signal from a predetermined sensor in the sensor / operator 17 or operation input information by the operator.
Specifically, the light control unit 10 performs auto headlight control for turning on and off the headlight 15 based on a detection signal from a predetermined sensor such as an illuminance sensor. The light control unit 10 also performs ON / OFF control of the low beam and the high beam of the headlight 15 based on the operation input information from the headlight switch 17D described above. In particular, the light control unit 10 of the present embodiment realizes the ADB function by controlling the ADB actuator 16 based on an instruction from the light distribution control processing unit 5A in the driving support control unit 5. The ADB actuator 16 in this example is, for example, an actuator that drives a light shielding plate. Whether the light shielding region is formed in a part of the high beam light distribution region by driving the light shielding plate based on the control from the light control unit 10. Alternatively, the light-shielding region is not formed (that is, the high beam is fully illuminated).

<2. Overview of processing executed in this embodiment>
With reference to FIG. 2, an outline of various processes executed in the present embodiment will be described.
In FIG. 2, various types of image processing executed by the image processing unit 3 based on the first captured image data and the second captured image data are shown in blocks for each function. 2 also shows the light distribution control processing unit 5A and the memory 4 included in the driving support control unit 5.

  As shown in the figure, the image processing unit 3 is roughly classified according to function. The distance image generation processing unit 3A, the lane detection processing unit 3B, the lane model formation processing unit 3C, the tail lamp detection processing unit 3D, the headlight detection processing unit 3E, It can be expressed as having a streetlight detection processing unit 3F, an object recognition / identification processing unit 3G, a scene determination processing unit 3H, and a control information calculation processing unit 3I.

  In the image processing unit 3, the distance image generation processing executed by the distance image generation processing unit 3 </ b> A is a process of generating a distance image based on the first captured image data and the second captured image data held in the memory 4. Specifically, the distance image generation process detects the corresponding points between the first captured image data and the second captured image data (that is, a pair of image data captured in stereo) by pattern matching, and between the detected corresponding points. Is calculated as a parallax M, and distance image data is generated using the parallax M and representing the distance to the corresponding point in real space on the image by the principle of triangulation.

  The lane detection processing executed by the lane detection processing unit 3B is generated by a reference image (that is, image data set in advance among the first captured image data or the second captured image data) and the distance image generation processing described above. Based on the distance image data (distance information for each pixel as a corresponding point), the lane formed on the road surface on which the host vehicle travels is detected. Specifically, in the lane detection process, first, a lane candidate point is detected on the reference image based on the luminance value of each pixel of the reference image and the distance in the real space of each pixel, and the lane detection point is automatically detected based on the detected lane candidate point. The left and right lane positions of the vehicle are detected. For example, a search is performed by offsetting a horizontal line having a width of one pixel on the reference image by one pixel in the left-right direction, and the luminance differential value (= edge strength) of each pixel is a threshold value based on the luminance value of each pixel of the reference image. Pixels that satisfy the conditions that change greatly as described above are detected as lane candidate points. This process is sequentially performed while offsetting the horizontal line to be searched for by one pixel width upward, for example, from the lower side of the reference image. Thereby, a lane candidate point is detected in each of the right region and the left region of the host vehicle.

The lane model formation process executed by the lane model formation processing unit 3C is based on the information on the left and right lane candidate points detected by the above lane detection, and the X, Y, and Z axes (the X axis is the left and right direction, the Y axis Is a process for forming a lane model in a three-dimensional space defined by the height direction and the Z-axis the vehicle traveling direction). Specifically, the lane model in the three-dimensional space is formed by linearly approximating the position (X, Y, Z) of the lane candidate point detected by the lane detection unit in the real space by, for example, the least square method. .
With the lane model formed in this way, the height information of the road surface on which the host vehicle travels can also be obtained.
Note that the method of the distance image generation process, the lane detection process, and the lane model formation process is the same as the technique disclosed in Japanese Patent Application Laid-Open No. 2008-33750.

  Tail lamp detection processing unit 3D, headlight detection processing unit 3E, streetlight detection processing unit 3F, object recognition / identification processing unit 3G, scene determination processing unit 3H, and control information calculation processing unit 3I respectively execute tail lamp detection processing, headlights The detection process, the streetlight detection process, the object recognition / identification process, the scene determination process, and the control information calculation process are particularly processes according to the present embodiment. Each processing according to these embodiments will be described later.

  Here, the memory 4 stores low beam time distance threshold relationship information 4A and high beam time distance threshold relationship information 4B. These pieces of information are used in the object recognition / identification process, and details thereof will be described later.

<3. Bright and dark images and detection range>
First, prior to description of each process according to the embodiment, two types of captured images (frame images) handled in each process and detection ranges of each target will be described.

As will be described later, in the present embodiment, the preceding vehicle and the oncoming vehicle are recognized and identified as targets that should not be irradiated with the high beam. The preceding vehicle is recognized and identified based on the detection result of the tail lamp, and the oncoming vehicle is recognized and identified based on the detection result of the headlight.
Here, since the headlight and the tail lamp have different amounts of light, there is a problem that a clear image cannot be detected by using both images captured at the same shutter speed. For example, in an image captured at a shutter speed according to the tail lamp, the brightness of the headlight is saturated and proper detection cannot be performed.

Therefore, in the present embodiment, the shutter speed is changed for each frame, and each object is detected using images captured with the shutter speed matched to the tail lamp and the shutter speed matched to the headlight. Hereinafter, the captured image data obtained by capturing at the shutter speed for the tail lamp (slower shutter speed than that for the headlight) is “bright image G1”, and the shutter speed for the headlight (faster shutter speed than that for the taillight). The captured image data obtained by imaging is denoted as “dark image G2”.
Examples of a bright image G1 and a dark image G2 captured for the same scene are shown in FIGS. 3A and 3B, respectively.

The image processing unit 3 instructs the imaging unit 2 so that the first camera unit 2A and the second camera unit 2B alternately output the bright image G1 and the dark image G2, respectively. As a result, as the first captured image data obtained by the first camera unit 2A and the second captured image data obtained by the second camera unit 2B, the bright image G1 and the dark image G2 are alternately displayed every frame period. It will be switched. At this time, the bright image G1 is imaged at the shutter speed set by the automatic adjustment function described above. The dark image G2 is imaged at a shutter speed obtained by adding a predetermined offset to the shutter speed of the bright image G1.
Note that the above-described distance image is generated based on the bright image G1.

Here, in the case of this example, the dark image G2 is used for headlight detection and also for streetlight detection processing by the streetlight detection processing unit 3F. Considering this point, the dark image G2 in this example is an image offset upward from the bright image G1.
In many cases, the brightness of the streetlight on the captured image is about halfway between the taillight and the headlight. Therefore, the streetlight detection process is not necessarily performed based on the dark image G2, but is performed based on the bright image G1. You can also.

  Further, in the present embodiment, detection ranges are determined for each target of the tail lamp (preceding vehicle), the headlight (oncoming vehicle), and the streetlight. That is, the detection process for each of these targets is not performed for all the pixels of the bright image G1 and the dark image G2, but the tail lamp detection range As as the target range of the tail lamp detection process and the target range of the headlight detection process. Is performed for the headlight detection range At and the streetlight detection range Ag as the target range of the streetlight detection process.

  4A shows an example of the tail lamp detection range As defined for the bright image G1, and FIG. 4B shows an example of the headlight detection range At and the streetlight detection range Ag defined for the dark image G2. Each of these detection ranges is set as a rectangular range. The position of each detection range is set so as to cover the area where the target exists in the image.

  By setting the detection ranges of the tail lamp detection range As, the headlight detection range At, and the streetlight detection range Ag as described above, the target detection range is limited, and the processing time is shortened and the processing load is reduced. Can be reduced, and erroneous detection in a place where the detection target does not exist can be prevented.

<4. Overall flow of processing>
FIG. 5 is a flowchart showing the overall flow of each process according to the embodiment.
Note that the series of processing shown in FIG. 5 is repeatedly executed by the image processing unit 3 every frame period.

First, the image processing unit 3 determines whether or not it is nighttime in step S101. When it is not nighttime, it is not necessary to detect and recognize each object in the first place, and therefore it is determined whether or not each object needs to be detected and recognized by the determination processing in step S101.
Whether it is night or not is determined based on the shutter speed and gain value of the captured image data. Alternatively, whether or not it is nighttime can be determined based on the result of determining whether or not the high beam is ON.
In step S101, if a negative result indicating that it is not nighttime is obtained, the process in the current frame period is terminated, and if a positive result that it is nighttime is obtained, the process proceeds to step S102.

In step S102, the image type is determined. That is, it is determined whether the captured image data captured from the imaging unit 2 in the current frame period is the bright image G1 or the dark image G2.
If the image type is the bright image G1, after performing the tail lamp detection process in step S103, the process in the current frame period is terminated.

  On the other hand, when it is determined that the image type is the dark image G2, the headlight detection process is performed in step S104, and then the streetlight detection process is performed in step S105.

  In subsequent step S106, object recognition / identification processing is executed. Although details will be described later, the object recognition / identification process is based on the result of the tail lamp detection process executed in step S103 and the results of the headlight detection process and the street lamp detection process executed in steps S104 and S105, respectively. This is a process for recognizing and identifying each object of an oncoming vehicle and a streetlight.

  After executing the object recognition / identification process, a scene determination process is executed in step S107, and a control information calculation process is executed in step S108, and the process ends.

  The contents of the tail lamp detection process, headlight detection process, streetlight detection process, object recognition / identification process, scene determination process, and control information calculation process executed as steps S103 to S108 will be described below.

<5. Tail lamp detection processing>
The tail lamp detection process is a process for detecting an area (tail lamp area) that is estimated to be a tail lamp portion of a preceding vehicle. The general processing flow is as follows.
First, in the tail lamp detection process, a tail lamp detection range As is set for the bright image G1, and red pixels are detected for pixels in the tail lamp detection range As. Then, the detected red pixels are grouped to create an element group.

Then, the basic feature amount of the element group is obtained. As basic features,
・ Up / down / left / right coordinates of the element group (position of each side when the element group is enclosed in a rectangle)
-Number of pixels in element group-Maximum luminance value and minimum luminance value in element group-Average parallax of element group (average value of parallax M of each pixel in element group)
Note that the parallax M uses a value obtained by the above-described distance image generation processing.

  Then, the element group is selected based on the basic feature amount information of the element group. That is, element groups whose basic feature values are outside the setting conditions described later are deleted.

Based on the above, specific processing contents executed as tail lamp detection processing will be described with reference to the flowchart of FIG.
In FIG. 6, the image processing unit 3 sets the tail lamp detection range As for the bright image G1 in step S201, and sets the target pixel identifier P = 0 in the subsequent step S202. The target pixel identifier P is an identifier for specifying a pixel that is a target of the tail lamp detection process among the pixels in the tail lamp detection range As, and the upper limit value is the number of pixels −1 constituting the tail lamp detection range As.

Next, in step S203, it is determined whether or not the pixel specified by the pixel identifier P (currently targeted pixel) is red.
If a positive result is obtained that the pixel is red, it is determined in step S204 whether the pixel is adjacent to the existing group. In other words, it is determined whether or not the current pixel is adjacent to any element group that has already been created in the processing in the current frame period.
Here, whether or not the current target pixel is adjacent to the created element group is determined based on whether or not the distance on the image is A1 pixel or less. For example, A1 = 1.5 pixels.

  If a negative result is obtained in step S204 that it is not adjacent to the existing group, the process proceeds to step S205 to execute a new group creation process. That is, a new element group is created, and the pixel currently targeted (the pixel specified by the identifier P) is added as a constituent pixel of the element group. At this time, information on basic feature values for the element group is also newly created.

  On the other hand, if an affirmative result that it is adjacent to the existing group is obtained in step S204, the basic feature amount updating process is executed in step S206. That is, the basic feature amount information for the corresponding element group is updated. Of the basic feature information, the information on the maximum luminance value and the minimum luminance value in the element group is updated as necessary.

  After executing the creation process in step S205 or the update process in step S206, the process proceeds to step S2207.

  If a negative result that the pixel is not red is obtained in the previous step S203, the process proceeds to step S207 without performing the grouping process in steps S204 to S206 described above. Thereby, pixels other than red are excluded.

  In step S207, it is determined whether or not the processing range has ended, that is, whether or not the value of the target pixel identifier P has reached the above-described upper limit (the number of pixels constituting the tail lamp detection range As−1). If a negative result that the processing range has not ended is obtained, the value of the target pixel identifier P is incremented (P ← P + 1) in step S208, and the process returns to step S203. As a result, the processes in steps S203 to S206 described above are executed for new pixels in the tail lamp detection range As.

On the other hand, if an affirmative result that the processing range has ended is obtained, group selection processing is executed in step S209. That is, element groups whose basic feature values are outside the following setting conditions are deleted.
Condition 1): The vertical / horizontal size of the element group is B1 pixels or less. B1 is 2 pixels, for example.
Condition 2): The number of pixels in the element group is C1 pixels or less. C1 is, for example, 2 pixels.
In addition, deletion may be performed when the size of the element group is too large. However, the threshold value in that case is changed by the distance (parallax M).

  With the group selection process in step S209, the image processing unit 3 ends the tail lamp detection process.

FIG. 7A shows a schematic diagram of element groups detected (finally selected) by the tail lamp detection process as described above.
The area shown in gray in the figure represents the area corresponding to the preceding vehicle (the preceding vehicle area) in the tail lamp detection range As, and the area shown in black represents the area detected as a red pixel. According to the tail lamp detection process described above, rectangular regions indicated by broken lines in the drawing are grouped as element groups.

  As described above, according to the tail lamp detection process, a region corresponding to the tail lamp portion of the preceding vehicle is detected as an element group.

  As can be understood from the above description, an element group can be defined as a group of characteristic parts included in a target to be recognized. In object recognition / identification processing described later, based on the detection results of these element groups, ranges corresponding to objects to be recognized are grouped as object groups (see FIG. 7B).

<6. Headlight detection process>
The headlight detection process is a process for detecting an area (headlight area) that is estimated to be a headlight part of an oncoming vehicle.
In the headlight detection process, first, a headlight detection range At is set for the dark image G2, and then a binarization process using a threshold value D2 for the luminance value for each pixel in the headlight detection range At. I do. Then, with respect to the pixels extracted by the binarization process, element groups are grouped and element groups are selected as in the tail lamp detection process.
The basic flow of processing is: i) Detect a pixel whose luminance value is greater than or equal to the threshold value D2.
ii) Group the detected pixels and create an element group
iii) Find basic feature values of element groups
iv) Element group selection. In the case of this example, the processing of ii) to iv) is the same as that of the tail lamp detection processing, and thus the description thereof is omitted.

  Here, the threshold value D2 used in the process i) can be adaptively changed according to the imaging condition. Thereby, the robustness of detection can be improved.

<7. Streetlight detection processing>
The streetlight detection process is a process for detecting an area (a streetlight area) that is presumed to be a streetlight.
In the case of this example, the streetlight detection process is the same as the headlight detection process except that the pixel to be processed is a pixel in the streetlight detection range Ag, and thus description thereof is omitted.

<8. Object recognition / identification process>
The object recognition / identification process is a process for recognizing and identifying a target (preceding vehicle, oncoming vehicle) based on at least the processing results of the tail lamp detection process and the headlight detection process.
Here, “recognition” means recognizing a target range. “Identification” refers to whether or not an object existing in the “recognized” range is a target (for example, reliability α described later), and whether or not the target is based on the probability. It means to perform the separation.

[8-1. Overall flow of object recognition / identification process]
FIG. 8 is a flowchart for explaining the overall flow of object recognition / identification processing.
As shown in FIG. 8, in the target recognition / identification process, the target area calculation process in step S301, the target group three-dimensional position calculation process in step S302, the target group tracking process in step S303, the target identification process in step S304, and the step The previous result inheriting process of S305 is performed in order.
Among these, the target area calculation process in step S301 corresponds to the above-described “recognition” process, and the target identification process in step S304 corresponds to the “identification” process.

[8-2. Target area calculation process]
First, the target area calculation process in step S301 will be described.
The target area calculation process is a process of grouping each element group obtained by the tail lamp detection process and the headlight detection process as a target group based on the basic feature information. In the target area calculation processing, processing based on the result of the tail lamp detection processing (that is, “recognition” processing of the preceding vehicle) and processing based on the result of the headlight detection processing (“recognition processing” of the oncoming vehicle) are individually performed. In this example, since the specific processing contents are common, the following will be described collectively.

  In addition, since the element group itself as one light emitter is an object to be recognized for the street lamp, in this example, the target area calculation process for the street lamp is not executed, and the element group is handled as the target group as it is.

FIG. 9 is a flowchart showing the flow of the target area calculation process.
In FIG. 9, the image processing unit 3 sets the element group identifier E = 0 in step S401. The element group identifier E is an identifier for identifying individual element groups that are detected in the tail lamp detection process and the headlight detection process and are processed in the target area calculation process. The element group identifier E is individually used in the target area calculation process based on the result of the tail lamp detection process and the target area calculation process based on the result of the headlight detection process.

In step S402, it is determined whether or not there is another element group that satisfies the grouping condition. That is, for the element group specified by the element group identifier E, a determination process is performed as to whether there is another element group that satisfies the condition for grouping the target group.
If it is determined in step S402 that there is no other element group that satisfies the grouping condition, a new group is created in step S403. That is, a new target group including the element group specified by the element group identifier E is created.
On the other hand, if there is another element group that satisfies the grouping condition, group integration processing is performed in step S404. That is, the element group specified by the element group identifier E and another element group that satisfies the grouping condition are integrated as a target group.

Here, the determination process in step S402 is a process related to “recognition” of the range of the preceding vehicle or the oncoming vehicle as a target.
In this example, whether or not a certain element group and another element group constitute the same object is determined using the coordinate information in the vertical and horizontal directions and the average parallax value of each element group.
Here, if the average parallax values of the element groups are approximately the same, it can be said that the element groups are likely to constitute the same object. In addition, if the same object is configured, it can be said that the separation distance in the vertical and horizontal directions in the image of these element groups is within a predetermined range.
In the determination process in step S402, element groups that are regarded as having the same average parallax value and whose separation distances in the vertical and horizontal directions in the image of the element groups are within a predetermined range are set as target groups. This is a process for grouping. The “predetermined range” for the separation distance is variable according to the value of the average parallax in consideration of the fact that the size of the target in the captured image changes according to the distance from the host vehicle.

Based on this point, the determination process in step S402 will be described.
In the determination process of step S402, it is determined whether or not the relationship between the distance between the element groups on the image, the difference in average parallax, and the maximum luminance value satisfies all of the following conditions.

Condition 1): Image distance It is determined whether or not the distance in the horizontal direction and the vertical direction in the real world is included in the same object.
The threshold values are individually set for the vertical direction and the horizontal direction, and are set as the vertical direction threshold value KH and the horizontal direction threshold value KW, respectively. The vertical direction threshold value KH and the horizontal direction threshold value KW are values obtained by adding a margin to the vertical and horizontal sizes of the target in the real world (vertical: LH, horizontal: LW, respectively), parallax M (whichever The average parallax of the group may be used, or the average may be used.) If the base line length of the stereo camera is b, KH = LH * M / b
KW = LW * M / b
This is the value obtained by. In the condition 1), it is determined whether the vertical and horizontal separation distances of each element group are the vertical threshold value KH or less and the horizontal threshold value KW or less, respectively.

Note that the target recognition / identification processing of this example is performed based on the detection result of the illuminant existing in the captured image.
A light emitting part in a vehicle as a preceding vehicle or an oncoming vehicle is a lamp disposed on the rear surface of the vehicle such as a tail lamp or a high-mount stop lamp, or a lamp disposed on the front surface of the vehicle such as a headlight or a fog lamp. These lamps are generally within a certain range even for small cars and large cars. Therefore, even if the values of KH and KW are common values regardless of the size of the object as described above, there is no particular problem with respect to the accuracy of the object recognition / identification process.

Condition 2): Average parallax difference It is determined whether or not the depth distance in the real world is included in the same object.
The threshold value is KZ. The threshold value KZ is LZ (for example, 1 m) in the depth direction in the real world, the parallax of each element group to be processed is M1, M2, the horizontal size of one pixel of the camera is λi, and the focal length is Let f be KZ = λiM1 * M2 * KZ / (b * f)
This is the value obtained by. In the condition 2), it is determined whether or not the absolute value (| M1−M2 |) of the parallax difference is less than the threshold value KZ. However, in practice, it is desirable to add a margin to the threshold value KZ in consideration of the influence of a parallax error or the like.

Condition 3): Relationship of Maximum Luminance Values Since the processing here mainly assumes grouping of left and right lamps of the same vehicle, it is also determined whether or not the luminance is similar. When the maximum luminance value of each element group to be processed is N1, N2 (where N1 ≧ N2),
N1 / N2 <O1
It is determined whether or not. That is, it is a determination as to whether or not the ratio of N1 and N2 is within a certain range. At this time, O1 is set to 3, for example.

When it is determined that there is another element group that satisfies all of the above conditions 1) to 3), the element groups are grouped by the group integration process in step S404 described above.
Thereby, grouping of the target groups as shown in FIG. 7B referred to earlier is realized.

As can be seen with reference to FIG. 7B, the method of this example cannot appropriately surround the vertical direction of the object as the preceding vehicle and the oncoming vehicle. This is because detection is performed only for the light emitting portion of the preceding vehicle and the oncoming vehicle in the tail lamp and headlight detection processes.
However, since the ADB is not the vertical light distribution control but the horizontal light distribution control, there is no problem in the final light distribution control.

Here, when integrating each element group as a target group in step S404, the basic feature amount of the element group is inherited. Specifically, the upper, lower, left, and right coordinates of the target group Left and lower positions inherit the minimum value of each element group Right and upper positions inherit the maximum value of each element group ・ Number of pixels in the target group Sum of each element group Inheritance ・ Maximum brightness value and minimum brightness value in the target group The maximum brightness value inherits the maximum value of each element group The minimum brightness value inherits the minimum value of each element group ・ Average parallax (distance) of the target group
The average parallax inherits the maximum value of each element group (the one with the shorter distance). In addition, the number of element groups constituting the target group is also counted.

  In FIG. 9, the image processing unit 3 determines whether or not the processing for all element groups has been completed in step S405 after executing either the creation processing in step S403 or the integration processing in step S404. That is, when the target area calculation process is a process for the preceding vehicle, it is determined whether or not the processes for all the element groups detected in the tail lamp detection process have been completed. Or when the said target area | region calculation process is a process about an oncoming vehicle, it is discriminate | determined whether the process about all the element groups detected by the headlight detection process was complete | finished.

If a negative result is obtained in step S405 that the processing for all element groups has not been completed, the value of the element group identifier E is incremented (E ← E + 1) in step S406, and the process returns to step S402. Thereby, the process of step S402-S404 is performed about the following element group.
On the other hand, if a positive result is obtained that the processing for all element groups has been completed, the target region calculation processing is terminated.

[8-3. Target group 3D position calculation process]
The target group three-dimensional position calculation process in step S302 is a process of calculating the real space three-dimensional position of the target group.
The average parallax of the target group is Pd, and the average of the vertical and horizontal positions (the coordinates of the center point of the rectangle surrounding the target group) is Pj and Pi (where the coordinate of the optical axis center position is (0, 0)). . Each position of the target group in the depth direction, the horizontal direction, and the vertical direction (referred to as Qz, Qx, and Qy, respectively) is the base line length b, the vertical of the pixel size is λj, the horizontal is λi, and the focal length f is
Qz = b * f / (λi * Pd)
Qx = λi * Pi * Qz / f
Qy = λj * Pj * Qz / f
Is required. In addition, when the camera is installed with a pitch (forward tilt or backward tilt) with respect to the host vehicle, the pitch correction is performed on the above result.
Further, for the vertical direction, the road surface height at the point of the distance Qz calculated in the above-described lane detection model formation process is subtracted and converted to be the height from the road surface.

[8-4. Target group tracking process]
The target group tracking process in step S303 is a process of counting how many frames the target group has been able to recognize in the past (the number of times they exist). The target group tracking process is executed for the previous result inheritance process (S305).

FIG. 10 is a flowchart showing the flow of target group tracking processing.
The target group tracking processing is also performed individually for each target of the preceding vehicle and the oncoming vehicle, but the processing contents are common and will be described collectively with reference to FIG.
In FIG. 10, the image processing unit 3 sets the target group identifier T = 0 in step S501. The target group identifier T is an identifier for specifying individual target groups recognized in the target area calculation process described above.

In step S502, it is determined whether there is a target group that satisfies the condition one frame before. That is, regarding the target group of the current frame specified by the target group identifier T, it is determined whether there is a target group that satisfies the following conditions one frame before. Note that “one frame before” means the previous bright image G1 when the target group tracking process is a process for a preceding vehicle. Similarly, when the process is for an oncoming vehicle, it means the previous dark image G2.
The conditions used in the determination process in step S502 are the same as those used in the determination process in step S402. The parameters used in each condition are basically the same value. However, considering that there is a possibility that tracking may be lost depending on the traveling state of the host vehicle (for example, turning or traveling speed), the distance (KH, KW) on the image is a margin depending on the yaw rate of the host vehicle. Adjust. Further, the margin is adjusted according to the speed of the host vehicle with respect to the difference in average parallax.

  If a negative result is obtained in step S502 that there is no target group that satisfies the condition one frame before, after the number of existences = 1 in step S503, the process proceeds to step S505. On the other hand, if an affirmative result is obtained that there is a target group that satisfies the condition one frame before, the number of existence is added (+1) in step S504, and the process proceeds to step S505.

In step S505, it is determined whether or not the processing for all target groups has been completed. If a negative result is obtained that the processing has not been completed for all target groups, the value of the target group identifier T is incremented (T ← T + 1) in step S506, and the process returns to step S502.
On the other hand, if a positive result is obtained that the processing for all target groups has been completed, the target group tracking processing is ended.

[8-5. Object identification process]
The target identification process in step S304 is a process of calculating a reliability α for each target group and extracting a target group having a reliability α of a predetermined threshold or more.
In the case of this example, the reliability α is calculated for each frame. In the target identification process of this example, the target group is extracted by comparing the reliability α with the predetermined threshold value for each frame.

FIG. 11 is a flowchart showing the flow of the object identification process.
The target identification process is also performed separately for the preceding vehicle and the oncoming vehicle. Since the flow of the object identification process for the preceding vehicle and the flow of the object identification process for the oncoming vehicle are common, they are collectively shown in FIG.
However, since the contents of the process for adding the reliability α (reliability addition side process described later) and the process for the subtraction (reliability subtraction side process) are different, this point will be described separately.

In FIG. 11, the image processing unit 3 sets the target group identifier T = 0 in step S601 and sets the reliability α = 0 in step S602.
In step S603, a reliability addition side process is executed, and in step S604, a reliability subtraction side process is executed.

  In a succeeding step S605, an identification determination process is performed. That is, for the target group specified by the target group identifier T, it is determined whether or not the condition that the value of the reliability α is equal to or greater than a predetermined threshold is satisfied. If the vehicle is not satisfied, an identification result that the target group is other than the target is obtained.

  In the next step S606, it is determined whether or not the processing for all target groups has been completed. If a negative result is obtained that the processing has not been completed for all target groups, the value of the target group identifier T is incremented (T ← T + 1) in step S607, and the process returns to step S602. On the other hand, if a positive result is obtained that the processing for all target groups has been completed, the target identification processing is ended.

  The reliability addition side process in step S603 and the reliability subtraction side process in step S604 will be described. As described above, the contents of these processes differ between the preceding vehicle and the oncoming vehicle, and will be described separately below.

  In addition, as a premise in the following description, as a similar object of the preceding vehicle, a red light emitter such as a signboard, a light of a distant city, and a sign (red sign) reflected by the light of the own vehicle are mentioned. Also, similar objects of oncoming vehicles mainly include street lamps, reflectors provided on the road surface, and sign reflections.

(Reliability addition processing for the preceding vehicle)
In the reliability addition side processing for the preceding vehicle, the reliability α is added when the following conditions are satisfied.
Condition 1)
When the number of element groups constituting the target group is equal to or less than a threshold value (for example, 4). At this time, if the number of element groups is 2 (two tail lamps), and 3 (two tail lamps and a high-mount stop lamp), there is a high possibility that the vehicle is a preceding vehicle. 2)
When the horizontal size of the target group is within a certain range (for example, within 1 m to 6 m, this is converted into a pixel size).
Condition 3)
When the vertical size of the target group is within a certain range (for example, this is converted to a pixel size within 0.5 m to 4 m).
Condition 4)
When the vertical / horizontal size ratio of the target group is within a certain range (for example, the horizontal is 0.5 or more times the vertical).

(Reliability subtraction processing for the preceding vehicle)
In the reliability subtraction side processing of the preceding vehicle, the reliability α is subtracted when the following conditions are satisfied.
Condition 1)
When the height of the target group from the road surface is outside a certain range (the range is, for example, 0 m to 4 m). At this time, the height calculated from the lane model forming process is used as the height from the road surface. However, the calculation accuracy of the height value from the road surface due to the lane model formation processing deteriorates on the long distance side (because the distance measurement by the stereo method tends to deteriorate the distance accuracy in the long distance), so the above range Is variable according to the distance Qz so as to increase in the distance.
Condition 2)
The position of the target group in the horizontal direction is not within a certain range with respect to the image center (the range is, for example, -30 m to 30 m). This is primarily intended for discrimination from distant city lights.
Condition 3)
A case where the difference between the maximum luminance value and the minimum luminance value of the target group is equal to or greater than the threshold Ts in a state where the host vehicle is radiating a high beam. This is intended to be distinguished from the reflection of the red sign by the light of the vehicle. Since the tail lamp is self-luminous, the light diffuses and the brightness distribution tends to be widened, while the reflection of the sign tends to make the brightness distribution narrow. By providing the condition 3), it is possible to improve the accuracy of identification between the tail lamp (leading vehicle) and a red similar object such as a red sign.
Here, the threshold value Ts is variable depending on the maximum luminance value, and for example, the threshold value Ts = 50 is set when the maximum luminance value is 200 or more, and the threshold value Ts = 30 when the maximum luminance value is less than 200. The reason why the range is made variable is that the difference in luminance value is affected by the luminance value (the difference between the maximum value and the minimum value changes at high luminance and low luminance).
Condition 4)
When the number of times the target group exists is less than or equal to the set value (eg, 3).

(Reliability addition processing for oncoming vehicles)
In the oncoming vehicle reliability addition processing, the reliability α is added when the following conditions are satisfied.
Condition 1)
When the number of element groups constituting the target group is equal to or less than a threshold value (for example, 4). At this time, when the number of groups is 2 (two headlights), there is a higher possibility that the vehicle is an oncoming vehicle, so the reliability α is further added.
Condition 2)
When the horizontal size of the target group is within a certain range (for example, within 1 m to 6 m, this is converted into a pixel size).
Condition 3)
When the vertical size of the target group is within a certain range (for example, this is converted to a pixel size within 0.5 m to 4 m).
Condition 4)
When the vertical / horizontal size ratio of the target group is within a certain range (for example, the horizontal is 0.5 or more times the vertical).

(Reliability subtraction processing for oncoming vehicles)
In the on-vehicle reliability subtraction process, the reliability α is subtracted when the following conditions are satisfied.
Condition 1)
When the height of the target group from the road surface is outside a certain range (the range is, for example, 0 m to 4 m). At this time, as in the case of the reliability subtraction side process for the preceding preceding vehicle, the height value from the road surface uses the result of the oblique line model formation process, and the above range is determined by the distance Qz so that the above range becomes large at a distance. Variable.
Condition 2)
The position of the target group in the horizontal direction is not within a certain range with respect to the center of the image (the range is, for example, -30 m to 30 m) (mainly intended to be distinguished from distant city lights).
Condition 3)
The maximum brightness value of the target group is less than or equal to the brightness threshold.
At this time, considering that the luminance value when the headlight of the oncoming vehicle is imaged decreases as the distance increases, the above-described luminance threshold value is variable according to the distance Qz of the target group, and decreases as the distance increases. . The luminance threshold is corrected according to the imaging conditions such as the shutter speed and gain of the camera. For example, correction is performed such that the brightness threshold is increased if the shutter speed is long.
Condition 4)
When the number of times the target group exists is less than or equal to the set value (eg, 3).
Condition 5)
The number of times the target group exists is equal to or greater than a set value (for example, the set value is a value obtained by converting 20 seconds into the number of frames). Unlike the preceding vehicle, it is very unlikely that one oncoming vehicle exists in front of the host vehicle for an excessively long time. Therefore, in such a case, the control is performed so that the reliability α is subtracted and the discrimination result from the oncoming vehicle is covered.

[8-6. Inheritance of previous results]
In the inheritance process of the previous result in step S305, the result of the previous frame is inherited as a countermeasure when the element group is detected unidentified by the element group detection or target group identification performed for the current frame. The processing procedure is as follows.
i) From the result of the target group tracking process in step S303, a target group that is present in the previous frame but is not present in the current frame (a target group that does not match any target group in the previous frame) is extracted.
ii) For the target group extracted in i) above, extract a target group whose number of existence until the previous frame is a certain value (for example, 10) or more.
iii) The target group extracted in the above ii) is continued as a target by reducing the number of times of existence (for example, 1/2 times).
By such inheritance processing, a target group that has existed stably as a target until then is continued as a target even if it is not detected a certain number of times. That is, it is possible to prevent the target from being left undetected by some temporary factor, and the identification accuracy can be improved in this respect.

<9. Scene determination processing>
The scene determination process is a process for determining whether or not the current traveling scene is a scene that requires a high beam in the first place. In this example, scenes that do not require a high beam are urban areas (because they are bright enough), low speeds (because it is not necessary to irradiate light far away), and right / left turns (because it is not necessary to illuminate far away). If any of them corresponds, a determination result is obtained that the scene does not require a high beam.
Here, the determination as to whether or not it is an urban area is made based on the number of detected street lights. For example, this is performed based on a determination result of whether or not the number of detected street lamps is equal to or greater than a predetermined threshold.
Whether the vehicle is at low speed is determined by determining whether the vehicle speed detected by the vehicle speed sensor 17A is below a certain value (for example, 20 km / h). However, hysteresis is provided to prevent hunting.
Whether the turn is right or left is determined by determining whether the winker is in an operating state. For example, it is determined whether or not the blinker switch 17E is in an ON state.

<10. Control information calculation process>
The control information calculation process is a process of calculating ADB control information based on the recognition / identification results of the preceding vehicle and the oncoming vehicle by the object recognition / identification process and the result of the scene determination process.
Specific processing contents will be described with reference to the flowchart of FIG.
In FIG. 12, the image processing unit 3 determines whether or not the scene is a high beam unnecessary scene in step S701 based on the result of the scene determination process. If a positive result indicating that the scene is not a high beam is obtained, control information indicating high beam OFF is generated in step S705, and the control information is transmitted to the driving support control unit 5 (light distribution control processing unit 5A) in step S706. After the output, the process is terminated.

On the other hand, if a negative result is obtained that the scene is not a high beam unnecessary scene, it is determined in step S702 whether there is a preceding vehicle or an oncoming vehicle. That is, based on the result of the object recognition / identification process described above, it is determined whether there is a preceding vehicle or an oncoming vehicle.
If a negative result that there is no preceding vehicle or oncoming vehicle is obtained, control information indicating full high beam ON is generated in step S703, and after the control information is output to the driving support control unit 5 in step S706, End the process.

If an affirmative result that there is a preceding vehicle or an oncoming vehicle is obtained, control information indicating high beam ON other than the target is generated in step S704. At this time, the image processing unit 3 calculates a range in which the high beam can be irradiated based on the result of the object recognition / identification process (information on the range is hereinafter referred to as “irradiation range information”). The range in which the high beam can be irradiated is calculated based on the left and right coordinate information of the preceding vehicle and the oncoming vehicle.
The image processing unit 3 outputs the control information (including irradiation range information) generated in step S704 to the driving support control unit 5 in step S706, and ends the process.

<11. Light distribution control based on control information>
The driving support control unit 5 (light distribution control processing unit 5A) executes light distribution control based on the control information as light distribution control processing. Specifically, in the light distribution control process, the light control unit 10 is instructed to turn off the high beam in accordance with the control information indicating the high beam off. Further, in response to the control information indicating that the entire high beam is on, the light control unit 10 is instructed to turn on the high beam all over. Further, according to the control information indicating the high beam ON other than the target, the light control unit 10 is instructed so that the high beam is irradiated only in the range according to the irradiation range information included in the control information.

FIG. 13 is an explanatory diagram of a high beam irradiation mode realized in accordance with control information indicating high beam ON other than the target. In the drawing, the upward direction in the drawing represents the forward direction of the host vehicle.
As shown in FIG. 13A, the high beam irradiation in this case is performed on a range other than the range where the preceding vehicle and the oncoming vehicle exist (shaded portion in the figure).

  An ADB having a specification that can shield only one high beam is also conceivable. In that case, when there are a plurality of objects that should not be irradiated with a high beam as in FIG. 13A and a range in which no object exists is formed between them, as shown in FIG. 13B, the high beam should be irradiated. The light distribution control is performed so that the range between the non-target objects (preceding vehicle, oncoming vehicle) is also the high beam non-irradiation range. For this purpose, for example, in the control information calculation process described above, if there are a plurality of targets that should not be irradiated with a high beam, they are grouped, and irradiation range information is calculated based on the maximum horizontal coordinate of the group. Good.

  Here, as an object that should not be irradiated with the high beam, the oncoming vehicle deviates from the headlight detection range At when it comes close to the own vehicle to some extent (see FIG. 14). Therefore, when it is confirmed that the oncoming vehicle has approached to a certain distance (for example, 50 m: provided that it is within the distance that can be detected in the headlight detection range At), the high beam is directed in the direction in which the passing is expected. Control is performed to turn off the irradiation for a certain period (for example, 1 second).

<12. Summary of Embodiment>
As described above, the present embodiment includes the image processing apparatus including the imaging unit 2 and the image processing unit 3 that obtain a captured image obtained by imaging the front of the host vehicle. The image processing unit 3 performs a red region detection process for detecting a red region (element group in the tail lamp detection range As) in the captured image by the tail lamp detection processing unit 3D. Further, a deviation value between a high luminance representative value (maximum luminance value) and a low luminance representative value (minimum luminance value) among luminance values of a plurality of pixels included in the detected red region is obtained, and the deviation value (difference) The target recognition / identification processing unit 3G performs a determination process for determining the magnitude relationship with the threshold value Ts, and the target recognition / identification processing unit 3G indicates the result of the determination process, and whether the red region is the tail lamp image of the preceding vehicle. It is used for the identification processing of whether or not.
Thus, in the detection of the tail lamp of the preceding vehicle, first, the red region of the captured image is extracted. By targeting the red region, the influence of light emitters and reflectors other than red is excluded, and the processing range on the image and the luminance range that is the red detection range are limited.
Furthermore, in order to distinguish between the red light emitter and the reflector, a determination process is performed to determine the magnitude relationship between the difference value (difference) between the high luminance representative value and the low luminance representative value and the threshold Ts. This is because different characteristics appear in the luminance distribution between the light emitter and the reflector. That is, a self-luminous light source such as a tail lamp of a vehicle tends to have a wide luminance distribution because light diffuses. On the other hand, light reflected from a sign or the like by a headlight of the own vehicle tends to have a narrow luminance distribution. It is in. Therefore, the divergence value between the high luminance representative value and the low luminance representative value in the red region is different between the tail lamp light and the reflected light, and the light emitter (tail lamp) and the reflector can be distinguished by comparing the divergence value with the predetermined threshold value Ts.
As described above, in this embodiment, only the red region is first targeted, thereby eliminating the influence of a light emitter or reflector other than red, such as a tail lamp, and the processing range on the image, the red detection range, and the like. Limit the luminance range. For this reason, the determination can be made only with the maximum luminance value, the minimum luminance value, etc. without using the frequency distribution. Therefore, processing can be performed even when the number of target pixels is small. In addition, since it is not necessary to perform frequency distribution processing, processing time is also shortened.
The red sign as a reflector can be distinguished to some extent by its size and height from the road surface. However, when the distance accuracy is not sufficient, such as when the target distance is sufficiently long (for example, 100 m or more), In comparison, erroneous detection (or undetected in an attempt to suppress erroneous detection) often occurs. The same applies to the case where the road surface position is not sufficiently accurate at a distance due to a gradient in the road surface. According to the present embodiment, tail lamp identification with high accuracy can be achieved even when the identification accuracy is insufficient depending on the size and height.
As described above, according to the present embodiment, accurate tail lamp identification can be realized with a small processing load.

  Note that, by using the maximum luminance value and the minimum luminance value as the high luminance representative value and the low luminance representative value, the processing for obtaining each representative value is facilitated and the processing load can be reduced. In addition, the processing load can be reduced by using a difference value as a difference value between the high luminance representative value and the low luminance representative value.

  In the embodiment, the image processing unit 3 changes the threshold value Ts for comparing the difference that is the deviation value according to the value of the high luminance representative value. The deviation value (difference) between the high luminance representative value (maximum luminance value) and the low luminance representative value (minimum luminance value) increases or decreases under the influence of the luminance value. In consideration of this, by changing the threshold value Ts to be compared with the divergence value according to the value of the high luminance representative value, it is possible to improve the identification accuracy of the tail lamp and other reflectors.

Further, the image processing unit 3 performs the identification process using the result of the determination process for determining the magnitude relationship between the difference value between the high luminance representative value and the low luminance representative value and the threshold value Ts, and the headlight of the host vehicle is in the high beam state. Run in the period. This is because reflected light such as a sign appears mainly on the image during the high beam period. When the high beam is turned off, there is almost no influence on the captured image of the red reflector. That is, it can be considered that the element group and the target group as the detected red region are almost tail lamps.
Therefore, it is appropriate to perform the above identification processing at least during the high beam period.

  Further, the image processing unit 3 sets a tail lamp detection range As in the frame of the captured image, and detects an element group and a target group as a red region within the tail lamp detection range As. That is, the red region detection is performed by limiting the detection range that can exist as a tail lamp. As a result, the influence of the surrounding red object is eliminated, the identification accuracy is improved, and the processing load is reduced by targeting a part of the area on the image.

Further, in the identification processing, the image processing unit 3 adds or subtracts the reliability α that the red region is the tail lamp image of the preceding vehicle according to various conditions, and the target group as the red region according to the value of the reliability α. Is a tail lamp image of the preceding vehicle. As described above, the various conditions include the number of element groups, the horizontal size, vertical size, vertical / horizontal size ratio, height from the road surface, horizontal distance, and the like of the target group. Then, the value of the reliability α is subtracted or added according to the result of the determination process for determining the magnitude relationship between the deviation value between the high luminance representative value and the low luminance representative value and the threshold value Ts. Thus, the reliability of tail lamp identification can be improved by performing final identification by adjusting the value of the reliability α according to various conditions including the determination result based on the deviation value.
In the above description, an example in which it is determined whether the divergence value between the high luminance representative value and the low luminance representative value is equal to or greater than the threshold value Ts, and the value of the reliability α is subtracted when the divergence value is equal to or greater than the threshold value Ts. However, conversely, it is possible to determine whether or not the deviation value is equal to or less than the threshold value Ts, and to add the value of the reliability α when the deviation value is equal to or less than the threshold value Ts.

<13. Modification>
Although the embodiments according to the present invention have been described above, the present invention should not be limited to the specific examples illustrated above, and various modifications can be considered.
For example, in the description so far, the example in which the high luminance representative value and the low luminance representative value are the maximum luminance value and the minimum luminance value in the red region has been described, but the present invention is not limited thereto. For example, in all pixels in the red region, the average value of the high luminance pixels with the upper 10% luminance value may be the high luminance representative value, and the average value of the lower luminance pixels with the lower 10% luminance may be the low luminance representative value.
The divergence value is a difference value between the high luminance representative value and the low luminance representative value, but may be a ratio value. In this case, the threshold Ts is also set to a value according to the case where the distribution width is determined by the ratio value.

Further, when performing pre-crash (PCS) or adaptive cruise control (ACC) control as driving support control, the preceding vehicle recognition / identification process for PCS or ACC (higher than the recognition / identification process for light distribution control). (Done with precision). In such a case, in the recognition / identification process of the preceding vehicle for light distribution control described above, an identification result is obtained only for an object existing in a far region (for example, 200 m or more) of a predetermined distance or more, For an area less than the predetermined distance, it is possible to divert (integrate) a highly accurate preceding vehicle recognition / identification result in PCS or ACC.
As described above, in order to obtain an identification result of only an object existing in the far field, as a subtraction condition of the reliability α for the “recognized” target group, “distance Qz is less than a predetermined value” It is sufficient to add a condition of “is”.

  In the description so far, the example in which the object identification process is not executed for the streetlight has been described. However, depending on the control specification, it may be necessary to identify the streetlight from other objects. For example, as a light-emitting body other than the street light detected within the street light detection range Ag, there is a traffic light or the like. When it is necessary to separate the light emitters, a street light / traffic signal identification process is performed. In that case, color information may be given to the target group, and if the color is red, blue, or green, it may be identified as a traffic light, and if it is any other color, it may be identified as a street light.

  In the description so far, the example in which the distance information obtained by the distance measurement by the stereo method is used in the object identification process has been described. However, the distance information can be measured by other methods. For example, a technique using a millimeter wave radar can be employed.

  In the above description, the case where the light distribution control as the ADB is performed based on the result of the object recognition / identification processing is exemplified. However, the light distribution control as the AHB (Auto High Beam) may be performed instead. it can. In that case, control may be performed to turn off the high beam when there is at least one preceding vehicle or oncoming vehicle, and to turn on the entire high beam when there is no preceding vehicle or oncoming vehicle.

  In the above description, the RGB format is exemplified as the color image format, but it is of course possible to use another expression format such as YUV.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle control system, 2 ... Imaging part, 3 ... Image processing part, 3D ... Tail lamp detection processing part, 3G ... Object recognition / identification processing part, As ... Tail lamp detection range

Claims (5)

An imaging unit for obtaining a captured image obtained by imaging the front of the host vehicle;
A red region detection process for detecting a red region in the captured image, and a high luminance representative value and a low luminance representative value among luminance values of a plurality of pixels included in the red region detected by the red region detection process. An image for obtaining a divergence value, performing a determination process for determining a magnitude relationship between the divergence value and a threshold value, and using a result of the determination process for an identification process for determining whether the red region is a tail lamp image of a preceding vehicle. An image processing apparatus comprising: a processing unit. The image processing unit
The image processing apparatus according to claim 1, wherein the threshold value is changed according to a value of the high luminance representative value. The image processing apparatus according to claim 1, wherein the image processing unit executes the identification process during a period in which a headlight of the host vehicle is in a high beam state. The image processing device according to any one of claims 1 to 3, wherein the image processing unit sets a tail lamp detection range in the captured image and detects a red region within the tail lamp detection range. In the identification processing, the image processing unit adds or subtracts the reliability that the red region is the tail lamp image of the preceding vehicle according to various conditions, and the red region corresponds to the tail lamp of the preceding vehicle according to the reliability value. Identify whether it is an image,
The image processing apparatus according to claim 1, wherein the reliability value is subtracted or added according to a result of the determination process.

Images