JP2016110539A - Vehicle-outside environment recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To specify a light emission source accurately and in an early stage.SOLUTION: A vehicle-outside environment recognition device 120 specifies a light emission source candidate as an overexposed white light emission source, and sets a predetermined range in the vicinity of the overexposed white light emission source on the vehicle area, as an overexposed white area when the light emission source candidate on a vehicle area satisfies a predetermined overexposed white condition. Then, to light emission source candidates positioned on a range other than the overexposed white range, it is determined whether or not, each of the light emission source candidates is a lamp which is a lighting state, based on a number of pixels or pixel area satisfying a predetermined color condition. To the light emission source candidate on the overexposed white area, it is determined whether or not the light emission source candidate is a lamp in the lighting state, based on a number of pixels or pixel area satisfying a color condition having higher intensity than that of the color condition of the non-overexposed white range.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a vehicle exterior environment recognition device that identifies a brake lamp that is lit in a preceding vehicle.

  Conventionally, a three-dimensional object such as a preceding vehicle located in front of the host vehicle is detected, and a collision with the preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is kept at a predetermined distance. (Cruise control) technology is known (for example, Patent Document 1). Here, if it is possible to incorporate a process of recognizing whether or not the brake lamp (lamp) of the preceding vehicle is lit (operation state of the brake) and estimating the deceleration operation of the preceding vehicle, smooth cruise control becomes possible. .

  As a technique for detecting whether or not the brake lamp of the preceding vehicle is lit, the brake lamp is configured by changing the threshold value for the size of the light emission source specified from the image captured by the imaging device according to the brightness of the outside environment. A technique (for example, Patent Document 2) for determining whether or not the LED is turned on is disclosed.

Japanese Patent No. 3349060 JP 2013-109391 A

  By the way, for example, when sunlight is reflected on a brake lamp cover, a rear window, or the like of a preceding vehicle, a region where the sunlight is reflected may satisfy the luminance condition of the light source. In such a case, the technique of Patent Document 2 described above does not change the conditions according to the environment outside the vehicle with respect to the luminance of the light source, so the region where sunlight is reflected is specified as the light source. There is a risk of erroneously determining that the light source is a brake lamp in a lit state.

  On the other hand, it may be possible to continuously detect the light source of the specified preceding vehicle over a certain period of time and determine whether or not the brake lamp is lit based on the detected luminance change of the light source. Since information is not sufficiently accumulated immediately after, for example, it is difficult to determine whether or not the brake lamp is lit at an early stage.

  In view of such problems, an object of the present invention is to provide a vehicle environment recognition apparatus that can identify a lamp in a lighting state with high accuracy at an early stage.

  In order to solve the above-described problem, a vehicle environment recognition device according to the present invention includes a vehicle identifying unit that identifies a preceding vehicle and a vehicle area occupied by the preceding vehicle in an image captured by an imaging device. A light source candidate specifying unit that specifies one or a plurality of light source candidates as the light source candidates in the specified vehicle region; and the light source when the light source candidate satisfies a predetermined overexposure condition. A whiteout light source specifying unit for specifying a candidate as a whiteout light source, a whiteout range setting unit for setting a predetermined range in the vicinity of the whiteout light source in the vehicle region as a whiteout range, and the specified Among the light source candidates, the light source candidates located outside the overexposure range are lamps that are lit based on the number of pixels or the pixel area satisfying a color condition of a predetermined intensity. And whether or not the light emitting source candidate is located within the overexposure range is a lit lamp based on the number of pixels or the pixel area satisfying the color condition of higher intensity than the color condition. It functions as a lamp determination unit for determination.

  Further, the computer counts the number of pixels that satisfy the color condition of the predetermined intensity for the light source candidates that are located outside the overexposure range among the identified light source candidates, The number of pixels is converted into a pixel area based on the relative distance from the preceding vehicle, and for a light emitting source candidate located in the overexposure range, a pixel that satisfies a color condition of intensity higher than the color condition of the predetermined intensity It further functions as an area conversion unit that counts the number of pixels and converts the number of pixels into a pixel area based on a relative distance from the preceding vehicle, and the lamp determination unit determines whether the converted pixel area is a lamp of the preceding vehicle. If it is equal to or greater than a predetermined lighting determination threshold value that is lit, the light source candidate may be determined to be a lit lamp.

  In addition, when the area where the light source candidate located outside the overexposure range is equal to or greater than the first lighting determination threshold as the lighting determination threshold, the lamp determination unit determines that the light source candidate is in a lighting state lamp. If the area where the light emission source candidate located in the overexposure range is converted is equal to or higher than the second lighting determination threshold higher than the first lighting determination threshold as the lighting determination threshold, the light emission The source candidate may be determined to be a lit lamp.

  Further, the area conversion unit counts the number of pixels that satisfy color conditions of different intensities according to the relative distance from the preceding vehicle for light emission source candidates located outside the overexposure range, The number may be converted into an area based on the relative distance to the preceding vehicle.

  In the vehicle environment recognition apparatus of the present invention, the computer has a vehicle specifying unit that specifies a preceding vehicle and a vehicle region occupied by the preceding vehicle in an image captured by the imaging device, and the specified vehicle region. A light source candidate specifying unit that specifies one or a plurality of light source candidates as light source candidates, and when the light source candidates satisfy a predetermined overexposure condition, the light source candidates are overexposed. Among the identified light emission source candidates, the overexposed light source specifying unit, the overexposed region setting unit for setting a predetermined range in the vicinity of the overexposed light source in the vehicle area as the overexposed range, For a light emission source candidate located outside the overexposed range, the number of pixels or the pixel area of the light emission source candidate is equal to or greater than a first lighting determination threshold value at which the lamp of the preceding vehicle is turned on. The light emission source candidate is determined to be in the lighting state of the lamp, and for the light source candidate located in the overexposure range, the number of pixels or the pixel area of the light emission source candidate is less than the first lighting determination threshold value. When it is equal to or higher than the high second lighting determination threshold value, the light source candidate functions as a lamp determination unit that determines that the lamp is in a lighting state.

  Further, one of the number of pixels, the pixel area, and the first lighting determination threshold value may be set according to a relative distance from the preceding vehicle.

  According to the present invention, it is possible to specify a light emitting source with high accuracy at an early stage.

It is the block diagram which showed the connection relation of the environment recognition system. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a color image and a distance image. It is explanatory drawing for demonstrating the process of a vehicle specific part. It is explanatory drawing for demonstrating the process of a vehicle specific part. It is explanatory drawing for demonstrating the difference between the imaging by a 1st exposure aspect, and the imaging by a 2nd exposure aspect. It is explanatory drawing which shows a color threshold value. It is a figure explaining an overexposure range. It is a figure which shows the color condition and lighting determination threshold value with respect to the light emission source candidate which exists in a non-overexposed range and an overexposed range. It is explanatory drawing which showed the relationship between the relative distance of the own vehicle and a preceding vehicle, and the number of pixels. It is a figure explaining a statistical lighting determination threshold value. It is a flowchart which shows a vehicle exterior environment recognition process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, an in-vehicle camera mounted on a vehicle images a road environment ahead of the host vehicle, identifies a three-dimensional object such as a preceding vehicle based on color information and position information in the captured image, Vehicles equipped with a so-called collision prevention function that avoids such collisions and keeps the distance between the vehicle and the preceding vehicle at a predetermined distance (ACC: Adaptive Cruise Control) are becoming popular.

  In the ACC and the collision prevention function, for example, a relative distance between the three-dimensional object positioned in front of the host vehicle and the host vehicle is derived, and based on the relative distance, a collision with a three-dimensional object positioned in front of the host vehicle is detected. If it is avoided or the three-dimensional object is a vehicle (preceding vehicle), control is performed so that the relative distance from the preceding vehicle is maintained at a predetermined distance. In addition, it is possible to realize smoother cruise control by incorporating a process for recognizing whether the brake lamp of the preceding vehicle is lit or not and estimating the deceleration operation of the preceding vehicle. Hereinafter, an environment recognition system for achieving such an object will be described, and a vehicle exterior environment recognition apparatus as a specific component thereof will be described in detail.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 is configured to include an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and images the front of the host vehicle 1 to generate a color image represented by a color value. can do. Here, the color value is a numerical group consisting of one luminance (Y) and two color differences (UV), or three hues (R (red), G (green), B (blue)). .

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates image data obtained by imaging the detection area in front of the host vehicle 1 for every 1/20 second frame (20 fps), for example. Here, specific objects recognized from the image data generated by the imaging device 110 include not only solid objects such as vehicles, pedestrians, traffic lights, roads (travel paths), guardrails, and buildings, but also brake lamps. Also included are things that can be specified as part of a three-dimensional object, such as high-mount stop lamps, tail lamps, blinkers, and lighting parts of traffic lights. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  Further, in the present embodiment, the imaging device 110 captures the detection region in the first exposure mode indicating the exposure time and the aperture according to the brightness of the environment outside the vehicle (measurement result of the illuminometer, etc.), and generates the first image. To do. In addition, the imaging device 110 generates a second image that can determine whether a specific light source such as a brake lamp emits light. As a method thereof, an imaging device having a wide dynamic range may be used, and imaging may be performed so that a three-dimensional object that does not emit light is not crushed black and a light emission source is not blown out. The first exposure mode is an exposure mode (exposure). The detection area may be imaged in a second exposure mode with different time and aperture to generate a second image. For example, during the daytime, the second image is generated by shortening the exposure time in the second exposure mode or increasing the aperture in comparison with the exposure time in the first exposure mode according to the bright outside environment. In the present embodiment, the first image and the second image are used as a color image and a distance image, respectively. Moreover, the said 1st exposure aspect and a 2nd exposure aspect are implement | achieved as follows.

  For example, the imaging apparatus 110 generates a first image and a second image sequentially by time-sharing periodic imaging timing and alternately performing imaging according to the first exposure mode and imaging according to the second exposure mode. . Further, the imaging device 110 is provided with two capacitors for each pixel, and in an imaging device capable of charging electric charges in parallel with the two capacitors, two different exposure modes are obtained by changing the charging time in one exposure. Images may be generated in parallel. Furthermore, the imaging device 110 may generate two images having different exposure modes in parallel by reading twice at different times during the charge of one capacitor. The imaging device 110 is configured by two sets of imaging devices with different exposure modes (here, two imaging devices 110 × 2 sets), and each of the two sets of imaging devices 110 may generate an image. . The exposure time that governs the exposure mode is appropriately controlled within a range of 1 to 60 msec, for example.

  The vehicle exterior environment recognition device 120 acquires image data from each of the two imaging devices 110, derives parallax (angle difference) using so-called pattern matching, and derives parallax information (three-dimensional position described later) based on the parallax. (Corresponding to information) is associated with image data to generate a distance image. The color image and the distance image will be described in detail later.

  Further, the outside environment recognition device 120 uses the three-dimensional position information in the real space including the color value based on the color image and the relative distance from the host vehicle 1 based on the distance image, and the three-dimensional position where the color value is equal. Blocks having close information are grouped as a three-dimensional object, and it is specified which specific object (for example, a preceding vehicle) corresponds to the three-dimensional object in the detection area in front of the host vehicle 1. For example, the preceding vehicle is identified by the relative distance or the like, and further, the position of the brake lamp of the preceding vehicle and the presence / absence of lighting are grasped by the color value. By such processing, it is possible to quickly grasp the deceleration of the vehicle due to the lighting of the brake lamp and use it for collision avoidance control and ACC.

  When the three-dimensional object is identified as the preceding vehicle, the outside-vehicle environment recognition device 120 tracks the preceding vehicle, derives the relative speed of the preceding vehicle, the relative distance from the preceding vehicle, and the like, and the preceding vehicle and the host vehicle 1 collide. It is determined whether or not there is a high possibility of being. Here, when it is determined that the possibility of the collision is high, the outside environment recognition device 120 displays a warning (notification) to the driver through the display 122 installed in front of the driver, and the vehicle control device. Information indicating that is output to 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, the procedure for identifying the preceding vehicle and the procedure for identifying the brake lamp that is lit in the preceding vehicle, which are characteristic of the present embodiment, will be described in detail, and the description of the configuration unrelated to the features of the present embodiment will be omitted. .

(Vehicle environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I / F unit 150, a storage unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The storage unit 152 includes a RAM, a flash memory, an HDD, and the like. The storage unit 152 stores various information necessary for processing of each function unit described below, and receives image data (first image and first image) received from the imaging device 110. A color image based on two images and a distance image) are temporarily stored.

  The central control unit 154 is a computer configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM in which programs are stored, a RAM as a work area, and the like. The storage unit 152 and the like are controlled. In the present embodiment, the central control unit 154 includes the image processing unit 160, the position information deriving unit 162, the vehicle specifying unit 164, the light source candidate specifying unit 166, the overexposure source specifying unit 168, and the overexposure range setting unit 170. , Function as an area conversion unit 172 and a lamp determination unit 174. In the following, detailed operations of such functional units will be described in the order of image processing, vehicle identification processing, light emission source candidate identification processing, overexposure range setting processing, and lamp determination processing based on the general purpose.

(Image processing)
The image processing unit 160 acquires image data (first image and second image) from each of the two imaging devices 110, and arbitrarily extracts blocks from one of the first images (for example, an array of horizontal 4 pixels × vertical 4 pixels) The parallax is derived by using so-called pattern matching, in which the block corresponding to) is searched from the other first image. The image processing unit 160 also derives the parallax using pattern matching for the second image. Here, “horizontal” indicates the horizontal direction of the captured color image, and “vertical” indicates the vertical direction of the captured color image.

  As this pattern matching, it is conceivable to compare the luminance (Y color difference signal) in units of blocks indicating an arbitrary image position between two images. For example, SAD (Sum of Absolute Difference) that takes the difference in luminance, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 160 performs such a block-unit parallax derivation process for all blocks displayed in the detection area (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the image processing unit 160 can derive the parallax for each block, which is a detection resolution unit, but cannot recognize what kind of three-dimensional object the block is. Therefore, the parallax information derived based on the parallax is independently derived not in units of solid objects but in units of detection resolution (for example, blocks) in the detection region.

  The position information deriving unit 162 includes a horizontal distance, a height, and a relative distance using a so-called stereo method based on the parallax for each block (each solid part) in the detection region 214 derived by the image processing unit 160. Three-dimensional position information is derived. Here, the stereo method is a method of deriving a relative distance of the three-dimensional part with respect to the imaging device 110 from the parallax of the three-dimensional part by using a triangulation method. At this time, the position information deriving unit 162 determines the road of the three-dimensional part based on the relative distance of the three-dimensional part and the distance on the distance image 212 from the point on the road surface at the same relative distance to the three-dimensional part to the three-dimensional part. Deriving the height from the surface. Note that an image in which the parallax information (three-dimensional position information) derived in this way is associated with image data is referred to as a distance image in distinction from the color image described above.

  FIG. 3 is an explanatory diagram for explaining the color image 210 and the distance image 212. For example, it is assumed that a color image (image data) 210 as shown in FIG. 3A is generated for the detection region 214 through the two imaging devices 110. However, only one of the two color images 210 is schematically shown here for easy understanding. In the present embodiment, the image processing unit 160 obtains the parallax for each three-dimensional part from such a color image 210, and the position information deriving unit 162 derives the three-dimensional position information for each three-dimensional part based on the parallax, A distance image 212 as shown in FIG. 3B is formed. Each three-dimensional part in the distance image 212 is associated with parallax information of the three-dimensional part. Here, for convenience of explanation, the solid part from which the parallax information is derived is represented by black dots. In the present embodiment, such a color image 210 and a distance image 212 are generated based on the first image and the second image, respectively. Therefore, in this embodiment, the color image 210 based on the first image, the distance image 212 based on the first image, the color image 210 based on the second image, and the distance image 212 based on the second image are used.

(Vehicle specific processing)
4 and 5 are explanatory diagrams for explaining the processing of the vehicle specifying unit 164. FIG. First, the vehicle specifying unit 164 divides the detection area 214 of the distance image 212 based on the first image into a plurality of divided areas 216 in the horizontal direction. Then, the divided region 216 has a strip shape as shown in FIG. Such a strip-shaped divided region 216 is originally composed of, for example, 150 columns each having a horizontal width of 4 pixels, but here, for convenience of explanation, the detection region 214 is divided into 16 parts.

  Subsequently, the vehicle specifying unit 164 adds up the relative distances included in each of the predetermined distances divided into a plurality of blocks located above the road surface for each divided region 216 based on the position information. A histogram (indicated by a horizontally long square (bar) in FIG. 4B) is generated. Then, a distance distribution 218 as shown in FIG. 4B is obtained. Here, the vertical direction indicates the divided predetermined distance (distance division), and the horizontal direction indicates the number of blocks (frequency) in which the relative distance is included in each distance division. However, FIG. 4B is a virtual screen for performing the calculation, and actually does not involve generation of a visual screen. Then, the vehicle identification unit 164 refers to the distance distribution 218 derived in this way, and represents a representative distance 220 (indicated by a black square in FIG. 4B) that is a relative distance corresponding to the peak. Identify. Here, “corresponding to a peak” means a peak value or a value that satisfies an arbitrary condition in the vicinity of the peak.

  Next, the vehicle specifying unit 164 compares the adjacent divided areas 216 with each other, and groups the divided areas 216 with the representative distance 220 close to each other (for example, located below 1 m) as shown in FIG. A plurality of divided region groups 222 are generated. At this time, even when the representative distance 220 is close in three or more divided areas 216, all the continuous divided areas 216 are collected as a divided area group 222. By such grouping, the vehicle specifying unit 164 can specify a three-dimensional object located above the road surface.

  Subsequently, the vehicle specifying unit 164 uses a block having a relative distance corresponding to the representative distance 220 in the divided region group 222 as a base point, a difference between the block, a horizontal distance, a height difference, and a relative distance difference in advance. Blocks within a predetermined range (for example, 0.1 m) are grouped on the assumption that they correspond to the same specific object. In this way, a three-dimensional object 224 that is a virtual block group is generated. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. Further, the vehicle identification unit 164 further groups blocks that have a horizontal distance difference, a height difference, and a relative distance difference within a predetermined range with respect to the block newly added by grouping. Turn into. As a result, all blocks that can be assumed to be the same specific object are grouped.

Here, the horizontal distance difference, the height difference, and the relative distance difference are independently determined, and the same group is formed only when all are included in the predetermined range, but other calculations may be used. . For example, the mean square root of the difference in horizontal distance, the difference in height, and the difference in relative distance √ ((difference in horizontal distance) ² + (difference in height) ² + (difference in relative distance) ² ) is included in the predetermined range. May be the same group. With this calculation, an accurate distance between blocks in real space can be derived, so that the grouping accuracy can be improved.

  Next, if the grouped three-dimensional object 224 satisfies a predetermined condition corresponding to a predetermined vehicle, the vehicle specifying unit 164 determines the three-dimensional object 224 as the specific object “vehicle”. For example, when the grouped three-dimensional object 224 is located on a road, the vehicle specifying unit 164 determines whether or not the size of the entire three-dimensional object 224 corresponds to the size of the specific object “vehicle”. If it is determined that the size corresponds to the size of the specific object “vehicle”, the three-dimensional object 224 is specified as the specific object “vehicle”. Here, the vehicle specifying unit 164 specifies a rectangular area that the solid object 224 specified as the specific object “vehicle” occupies on the screen as the vehicle area.

  In this way, the vehicle exterior environment recognition device 120 can extract one or a plurality of three-dimensional objects 224 as specific objects, for example, vehicles (preceding vehicles), from the distance image 212 as the first image. It can be used for control. For example, when an arbitrary three-dimensional object 224 in the detection area 214 is identified as a vehicle, the identified vehicle (preceding vehicle) is tracked to detect a relative distance and a relative acceleration to avoid a collision with the preceding vehicle. Or the distance between the preceding vehicle and the preceding vehicle can be controlled to be kept at a safe distance. In order to more quickly identify the preceding vehicle and the behavior of the preceding vehicle, a lit brake lamp is determined from light emission source candidates located in the vehicle area.

(Light emission source candidate identification process)
By the way, a 2nd image is an image imaged with the 2nd exposure mode which can discriminate | determine a specific light emission source (here, the brake lamp of a lighting state), for example. Here, a self-luminous device such as a brake lamp can acquire a high color value regardless of the brightness of the sun or a streetlight. In particular, since the brightness when the brake lamp is turned on is generally defined by laws and regulations, only pixels corresponding to the brake lamp can be captured by taking an image in an exposure mode (for example, short-time exposure) in which only a predetermined brightness can be exposed. Can be easily extracted.

  FIG. 6 is an explanatory diagram for explaining a difference between imaging according to the first exposure mode and imaging according to the second exposure mode. FIG. 6A shows a color image 210 based on the first image according to the first exposure mode, and in particular, the tail lamp is lit in the left diagram of FIG. 6A, and in the right diagram of FIG. 6A. The brake lamp is lit in addition to the tail lamp. As can be understood with reference to FIG. 6A, in the first exposure mode according to the brightness of the outside environment, the color value of the tail lamp position 230 when the brake lamp is off and the tail lamp is on. The color value difference hardly occurs between the brake lamp position 232 when the brake lamp is lit and the tail lamp is lit. This is because, in the first exposure mode with a long exposure time, the color values of all RGB components of the tail lamp and the brake lamp are saturated.

  FIG. 6B shows a color image 210 based on the second image according to the second exposure mode, and in particular, the tail lamp is lit in the left diagram of FIG. 6B, and in the right diagram of FIG. 6B. The brake lamp is lit in addition to the tail lamp. The second exposure mode is set so that only the color value when the brake lamp is lit can be acquired. Therefore, as can be understood with reference to FIG. 6B, even when the tail lamp is lit, the color value according to the brightness is hardly obtained at the tail lamp position 230, and the brake lamp when the brake lamp is lit is obtained. At the position 232, a clearly high color value can be obtained.

  In the second exposure mode, it is desirable that the R component of the color value of the brake lamp is set to an exposure time such as whether or not the image sensor is saturated. Since the imaging device 110 normally has a dynamic range that is significantly narrower than that of humans, when the first exposure mode is used with a brightness as low as in the evening, the color value of the brake lamp increases relative to the environment outside the vehicle. As a result, not only the R component but also the R component overlap and the G component and the B component are saturated to the maximum value (for example, the color value is 255), and the pixel becomes white. Therefore, by setting the second exposure mode to an exposure time of whether or not the R component is saturated when the brake lamp is lit, the influence on the color values of the G and B components is suppressed regardless of the external environment. However, only the R component is extracted with the maximum value. Thus, for example, it is possible to secure the maximum color value difference from the tail lamp.

  Specifically, if the second exposure state is set so that the lit brake lamp is in the color range (R) “200 or more” when there is a preceding vehicle during night driving, the lit tail lamp is turned on. However, the color range (R) “50”, the color range (G) “50”, and the color range (B) “50” are not displayed in the color image 210 based on the second image. On the other hand, the lit brake lamp is displayed in the second image in, for example, the color range (R) “200 or more”, the color range (G) “50 or less”, and the color range (B) “50 or less”. The In this way, the light emission source candidate specifying unit 166 can specify the brake lamp that is in the lit state through the color image 210 based on the second image. Hereinafter, a light source candidate specifying process for specifying a light source candidate including a lighted brake lamp as a light source candidate from the color image 210 based on the second image will be described.

  FIG. 7 is an explanatory diagram showing the color threshold. In the present embodiment, for example, as illustrated in FIG. 7, “yellow (Y)” and “red (R)” are used as the color threshold values for specifying the lit brake lamp from the color image 210 based on the second image. , “Slightly darker red (WR1) compared to red”, “Further darker red (WR2)”, and “Darkest red (WR3)” are provided in five stages. In order to identify the emitted light source, a color threshold value “whiteout” is provided as a whiteout threshold value. Here, the standard shutter speed in the second exposure mode is 17 msec. In the present embodiment, a plurality of color conditions based on a plurality of color thresholds are employed instead of directly using such a plurality of color thresholds. Here, as a plurality of color conditions, “yellow” (hereinafter simply referred to as “first color condition”), “yellow” + “red” (hereinafter simply referred to as “second color condition”), “yellow” + “ "Red" + "Slightly darker red compared to red" (hereinafter simply referred to as "third color condition"), "Yellow" + "Red" + "Slightly darker red compared to red" + "More dark red" (Hereinafter simply referred to as “fourth color condition”), “yellow” + “red” + “slightly darker red compared to red” + “darker red” + “darkest red” (hereinafter simply referred to as “first color condition”) 5 stages of “5 color conditions” are provided. As described above, the reason why each condition is the sum of each color threshold and another color threshold having a higher intensity (brightness) is to appropriately obtain a region having an intensity higher than the predetermined color threshold.

  The light emission source candidate specifying unit 166 acquires color values of three hues (R, G, B) in units of pixels from the color image 210 based on the second image. Since at least the pixel having the color value of “darkest red” may be an image of the lit brake lamp, the light emission source candidate specifying unit 166 determines that the color value satisfies the fifth color condition. A pixel that satisfies the condition, that is, a pixel that is not less than the color value of “darkest red” is specified. If the detection area 214 is rainy or cloudy, for example, the light emission source candidate specifying unit 166 may acquire the detection area 214 after adjusting the white balance so that the original color value can be acquired.

  If the specified horizontal distance difference, height difference, and relative distance difference between the pixels that satisfy the fifth color condition are within a predetermined range (for example, 0.1 m), the light emission source candidate specifying unit 166 includes a plurality of them. Are grouped as one light source candidate. Thus, even if the pixels that make up the brake lamp span a plurality of times, and even if the left and right brake lights of the vehicle are made up of a plurality of lamps, the brake lights that exist on the left and right It becomes possible to recognize individually as one of these.

  Then, the light source candidate specifying unit 166 associates the vehicle region specified based on the first image with the specified light source candidate based on the parallax information indicated by the distance image 212 based on each image.

(Whiteout range setting process)
However, depending on the external environment such as the type of light source and the sunshine conditions, sunlight is reflected on the brake lamp, rear window, etc., and the color value of the portion where the sunlight is reflected in the second image becomes high, which is erroneous as a light source candidate. May be detected. Therefore, simply comparing the color value of the identified light emitting source candidate with a fixed threshold value may cause erroneous determination of a brake lamp that is in a lighting state. For example, although the brake lamp is not lit, the color value becomes higher than a fixed threshold value due to the reflection of sunlight, and it may be erroneously determined that the brake lamp is lit.

  Accordingly, the whiteout light source specifying unit 168 determines whether or not there is a light source candidate whose color value is higher than the whiteout color threshold among the light source candidates located in the vehicle area specified by the light source candidate specifying unit 166. Is determined, and a light emitting source candidate having a whiteout color threshold or more is specified as a whiteout light source. The overexposure indicates a pixel (a pixel group) that has been saturated in the second image to become white or a color close to white, and the overexposure color threshold is assumed to be saturated in the second image. Set to a color value. That is, it can be said that the light emission source candidate specified as the overexposure light source is a pixel group that is saturated in the second image due to reflection of sunlight. In addition, when an overexposed light source is specified, it is highly likely that pixels near the overexposed light source have high color values due to sunlight reflection.

  Therefore, the overexposure range setting unit 170 sets an overexposure range in which the color value is increased due to the reflection of sunlight with respect to the vicinity of the overexposure light source saturated by the reflection of sunlight.

  FIG. 8 is a diagram for explaining the overexposure range 244. Specifically, as shown in FIG. 8, if four light source candidates 242a, 242b, 242c, and 242d are specified in the vehicle area 240, and the light source candidate 242a is specified as a whiteout light source, a whiteout whiteout occurs. The range setting unit 170 sets a whiteout range 244 of ± 20 cm in the vertical direction in the vehicle area 240 with reference to the vertical center of the light emission source candidate (whiteout light source) 242a. In addition, the whiteout range setting unit 170 sets a range other than the whiteout range 244 in the vehicle region 240 as a non-overout range 246 that is not affected by the reflection of sunlight. Here, the processing load can be reduced by targeting only the vertical direction as the overexposure range 244.

  In the lamp determination process described below, the color condition used to determine whether or not the light source candidate located in the vehicle region is a lit brake lamp, and a lighting determination threshold described later in detail. Are different between the non-whiteout range 246 and the whiteout range 244.

(Ramp judgment processing)
FIG. 9 is a diagram showing color conditions and lighting determination thresholds for light emission source candidates existing in the non-out-of-white range 246 and over-out range 244. As will be described in detail later, the area conversion unit 172 converts the number of pixels of the light emission source candidate into an area using the color condition shown in FIG. 9, and the lamp determination unit 174 uses the lighting determination threshold value shown in FIG. Then, it is determined whether or not the light emission source candidate located in the vehicle area 240 is a brake lamp that is lit.

  As shown in FIG. 9, in the non-whiteout range 246, the color conditions are different depending on the relative distance from the preceding vehicle. This is because the color value of the brake lamp captured in the second image decreases as the relative distance increases, so the color condition is lowered as the relative distance increases, It is possible to prevent erroneous determination that the light is off due to the color condition.

  In the present embodiment, the color condition of the non-whiteout range 246 is set to the second color condition when the relative distance is 0 to 40 m, and is set to the third color condition when the relative distance is 40 to 60 m. When the relative distance is 60 to 80 m, the fourth color condition is set, and when the relative distance is 80 m or more, the fifth color condition is set. The lighting determination threshold value for the non-whiteout range 246 is set to a lighting determination threshold value THL, which will be described later in detail.

  On the other hand, the color condition of the overexposed range 244 is highly likely to have a high color value due to the reflection of sunlight. 2 color condition to 5th color condition) or more. Further, the lighting determination threshold value of the overexposure range 244 is set to a lighting determination threshold value THH that is higher than the lighting determination threshold value THL.

  By the way, if the relative distance between the host vehicle 1 and the preceding vehicle is long, the light emission source candidates that satisfy the color condition are small and the number of pixels is also small. On the other hand, if the relative distance from the preceding vehicle is short, the light source candidate that satisfies the color condition increases and the number of pixels increases. Therefore, even if the brake lamp is kept on, the number of pixels that satisfy the color condition varies according to the change in the relative distance from the preceding vehicle. For example, if the number of pixels that satisfy the color condition differs depending on the positional relationship with the preceding vehicle even though the brake lamp is lit, the brake lamp is lit and there are pixels that satisfy the color condition. However, the relative distance may be too long, and the number may be less than the threshold value. Therefore, in the present embodiment, the number of pixels is converted into an actual area based on the relative distance from the preceding vehicle.

  FIG. 10 is an explanatory diagram showing the relationship between the relative distance between the host vehicle 1 and the preceding vehicle and the number of pixels. In FIG. 10, the horizontal axis indicates the relative distance, and the vertical axis indicates the number of pixels occupied by a three-dimensional object having a predetermined size. As can be understood with reference to FIG. 10, the number of pixels decreases as the relative distance increases even for the same three-dimensional object (the same area). Such a transition can be approximated by a function, and the relative distance from the zero point to the relative distance a point in FIG. 10 is proportional to the relative distance, and after the a point, it is proportional to the third power of the relative distance. Normally, the size of a three-dimensional object in an image is simply proportional to the relative distance, but in the case of a light source, the apparent light emission range is expanded by the influence of light emission. Therefore, as shown in FIG. 10, the relationship between the relative distance and the number of pixels becomes nonlinear.

  Therefore, the area conversion unit 172 counts the number of pixels that satisfy the color condition (FIG. 9) set based on the relative distance from the preceding vehicle, and divides the number by the inverse function of FIG. 10 (divides by the number of pixels of FIG. 10). ), The number of pixels satisfying the color condition is converted into an area. Specifically, the area conversion unit 172 counts the number of pixels satisfying the second color condition if the relative distance is 0 to 40 m for the light emitting source candidates located in the non-out-of-white range 246, and the relative distance is If the relative distance is 60 to 80 m, the number of pixels that satisfy the fourth color condition is counted. If the relative distance is 80 m or more, the fifth color is counted. Count the number of pixels that satisfy the condition. In addition, the area conversion unit 172 counts the number of pixels that satisfy the second color condition, regardless of the relative distance, for the light emission source candidates located in the overexposure range 244.

  Then, the area conversion unit 172 converts the counted number of pixels into an area by the inverse function of FIG. 10 (divide by the number of pixels of FIG. 10). Thus, the variation in the size of the three-dimensional object is reduced, and the lamp determination unit 174 described later can determine the brake lamp in the lighting state with high accuracy by comparing the converted area with the lighting determination threshold value. It becomes.

  FIG. 11 is a histogram for explaining a statistical lighting determination threshold. The area of a region where sunlight is reflected, which is specified as a light source candidate that satisfies a predetermined color condition in the second image, and a brake lamp in a lighting state. The area is taken as a statistic with the horizontal axis as the frequency. As shown in FIG. 11, it is statistically known that the area of the area where sunlight is reflected shows a value that is substantially smaller than the area of the brake lamp in the lit state.

  Therefore, in the present embodiment, a lighting determination threshold value (second lighting determination threshold value) THH for distinguishing between the area of the region where sunlight is reflected and the area of the brake lamp in the lighting state is statistically determined in advance and stored in the storage unit 152. Is remembered. Here, using only the lighting determination threshold THH, it may be possible to determine whether or not all light source candidates are brake lamps in the lighting state, but among the light source candidates that are less than the lighting determination threshold THH. There are not a few brake lights that are lit. Therefore, a lighting determination threshold value (first lighting determination threshold value) THL that is lower than the lighting determination threshold value THH and lower than the area of the brake lamp that is lit is statistically determined in advance and stored in the storage unit 152. Yes. The lighting determination threshold value THH is applied to the light emission source candidate in the overexposure range 244 that is assumed to have a high color value due to the reflection of sunlight, and the reflection of sunlight is not generated. The lighting determination threshold value THL is applied to light emission source candidates in the non-whiteout range 246.

  Specifically, the lamp determination unit 174 tentatively determines that the brake lamp is in the lit state when the area of the light emission source candidate located in the non-out-of-white range 246 is equal to or greater than the lighting determination threshold THL. On the other hand, when the area of the light emitting source candidate located in the overexposure range 244 is equal to or greater than the lighting determination threshold value THH, the lamp determination unit 174 temporarily determines that the brake lamp is in a lighting state.

  Then, the lamp determination unit 174 determines that the light source candidate temporarily determined to be a lit brake lamp is a height range “0.3 to 2.0 m” and a horizontal distance width range “0.05”. It is determined whether the conditions of “˜0.2 m” and the vertical distance width range “0.05-0.2 m” are satisfied. Further, the lamp determination unit 174 has a combination of a pair of brake lamps in which a horizontal distance difference “1.4 to 1.9 m”, a vertical distance difference “0.3 m or less”, and an area ratio “50 to 200%”. It is determined whether or not the above condition is satisfied. And the lamp | ramp determination part 174 determines with a pair of light emission source candidate which satisfy | filled the said conditions being a brake lamp of a lighting state. In this way, the light source candidate that is lit with only one lamp with the same brightness, such as a rear fog lamp, is configured by specifying the brake lamp only when the light source candidate corresponds to an appropriate position of the vehicle. It is possible to prevent erroneous recognition as a brake lamp.

  As described above, when the lamp determination unit 174 determines a brake lamp that is lit, the lamp determination unit 174 associates the brake lamp with the “vehicle” specified by the first image according to the first exposure mode described above.

  As described above, the on-vehicle environment recognition apparatus 120 is capable of emitting light in the whiteout range 244 near the whiteout light source even if the color value of the light emitting source candidate in the whiteout range 244 is high due to sunlight reflection. For the candidate, the number of pixels is counted using a color condition higher than the color condition for the light emitting source candidate in the non-whiteout range 246. Thereby, the area which considered the influence of sunlight reflection can be derived | led-out, and the brake lamp of a lighting state can be determined accurately. It should be noted that the number of pixels may be counted using a color condition higher than the color condition for the light emission source candidates in the non-exposed range 246 for the light emission source candidates in the overexposed range 244.

  Further, the outside environment recognition device 120 sets the light source candidate in the whiteout range 244 as a light source candidate in the whiteout range 244 in the vicinity of the whiteout source even if the color value of the light source candidate in the whiteout range 244 is high due to reflection of sunlight. On the other hand, it is determined whether or not the brake lamp is in a lit state using a high lighting determination threshold THH that is different from the light emission source candidates in the non-whiteout range 246. Thereby, the determination which considered the influence of sunlight reflection can be performed, and the brake lamp of a lighting state can be determined accurately.

  Furthermore, the outside environment recognition device 120 does not need to continuously detect a change in the brightness of the brake lamp after specifying the preceding vehicle, and can determine the lamp that is lit at an early stage by one process.

(External vehicle environment recognition process)
Next, the flow of the outside environment recognition processing including the above-described image processing, vehicle identification processing, light source candidate identification processing, overexposure range setting processing, and lamp determination processing executed by the central control unit 154 will be described.

  FIG. 12 is a flowchart showing the flow of the external environment recognition process. As shown in FIG. 12, first, the image processing unit 160 acquires a first image captured in the first exposure mode and a second image captured in the second exposure mode from the imaging device 110 (S300). ). Then, the image processing unit 160 derives parallax from the acquired image, and the position information deriving unit 162 derives three-dimensional position information for each three-dimensional part based on the derived parallax (S302). Subsequently, the vehicle specifying unit 164 specifies the vehicle and the vehicle area from the three-dimensional objects grouped based on the three-dimensional position information, and specifies the relative position (relative distance) with the preceding vehicle (S304).

  Then, the light emission source candidate specifying unit 166 specifies, as light emission source candidates, a group of pixels in which the color values of the pixels constituting the second image satisfy the fifth color condition (S306). Thereafter, the whiteout light source specifying unit 168 specifies a light source candidate whose color value of the light source candidate located in the vehicle region is equal to or greater than the whiteout threshold as a whiteout light source (S308). Subsequently, the whiteout range setting unit 170 determines whether or not a whiteout light source has been specified (S310). If a whiteout light source has been specified (YES in S310), the whiteout light source is in the vicinity. Is set as a whiteout range 244 (S312). Note that the overexposure range setting unit 170 does not set the overexposure range 244 for the vehicle region 240 unless the overexposure light source is specified (NO in S310).

  Thereafter, the area conversion unit 172 selects one of the identified undetermined light source candidates (S314), and determines whether or not the selected light source candidate is within the overexposure range 244 (S316). If the selected light emission source candidate is in the overexposure range 244 (YES in S316), the area conversion unit 172 counts the number of pixels that satisfy the second color condition, and determines the pixel based on the relative distance from the preceding vehicle. The number is converted into an area (S318). Subsequently, the lamp determination unit 174 tentatively determines whether the brake lamp is in a lighting state by comparing the converted area with the lighting determination threshold value THH (S320).

  On the other hand, if the selected light emission source candidate is not in the overexposure range 244, that is, in the non-excess range 246 (NO in S316), the area conversion unit 172 causes the color condition (second color condition) corresponding to the relative distance. The number of pixels satisfying any one of the fifth color condition) is counted, and the number of pixels is converted into an area based on the relative distance from the preceding vehicle (S322). Subsequently, the lamp determination unit 174 tentatively determines whether or not the brake lamp is in a lighting state by comparing the converted area with the lighting determination threshold value THL (S324).

  Thereafter, the area conversion unit 172 determines whether or not there is an undetermined light source candidate (S326). If there is an undetermined light source (YES in S326), the process returns to S314. On the other hand, if there is no undetermined light source (NO in S326), the lamp determining unit 174 determines that the light source candidate temporarily determined as a lit brake lamp satisfies the conditions such as position and size. Is determined to be a lit brake lamp (S328), and the vehicle exterior environment recognition process is terminated.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  In the above-described embodiment, the light source lamp in the non-exposed range 246 and the light source candidate in the overexposed range 244 are used with the brake lamps in the lit state using different color conditions and lighting determination thresholds. Judgment whether or not there is. However, the present invention is not limited to this, and the lighting determination threshold value is the same for the light emission source candidates in the non-out-of-white range 246 and the light-emitting source candidates in the over-out range 244, and only the color condition is different and the brake lamp is in the lit state. It may be determined whether there is a brake lamp, or the color condition is the same, and only the lighting determination threshold value is varied to determine whether the brake lamp is in the lit state.

  In the above embodiment, the number of pixels satisfying the color condition corresponding to the relative distance is counted for the light emission source candidate, the number of pixels is converted into an area based on the relative distance with the preceding vehicle, and the converted area and The lighting state lamp is determined by comparing the lighting determination threshold value. However, the present invention is not limited to this, and the light-emitting source candidate may be determined by comparing the number of pixels or the pixel area satisfying a predetermined color condition with a lighting determination threshold value.

  Further, in the above embodiment, it is determined whether or not conditions such as position and size are satisfied with respect to a light source candidate that is temporarily determined to be a lit brake lamp, and a light source candidate that satisfies the condition is determined to be a lit brake lamp. It was determined. However, the present invention is not limited to this, and a light source candidate that is temporarily determined to be a lit brake lamp may be determined to be a lit brake lamp as it is.

  In the above embodiment, the brake lamps that are lit are determined. However, the present invention is not limited to this, and other lit lamps may be determined.

  Further, in the above embodiment, when the overexposure light source is specified, a range of ± 20 cm in the vertical direction with respect to the overexposure light source, that is, a constant width in the horizontal direction with respect to the vehicle region 240. The overexposure range 244 is set. However, the present invention is not limited to this, and when an overexposure source is specified, the overexposure range may be set within a predetermined range in the vertical and horizontal directions with respect to the overexposure source.

  In the above embodiment, the central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), ROM, RAM, and the like. However, the present invention is not limited to this, and an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used. Moreover, you may make it comprise with 1 or several central processing unit, FPGA, and ASIC.

  Further, a program for causing a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium such as a flexible disk, a magneto-optical disk, a DRAM, an SRAM, a ROM, an NVRAM, a CD, a DVD, and a BD on which the program is recorded Is also provided. Here, the program refers to data processing means described in an arbitrary language or description method.

  In addition, each step of the environment recognition process outside the vehicle in this specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an external environment recognition device that identifies a brake lamp that is lit in a preceding vehicle.

120 Outside environment recognition device 164 Vehicle identification unit 166 Light emission source candidate identification unit 168 Whiteout light source specification unit 170 Whiteout range setting unit 172 Area conversion unit 174 Lamp determination unit

Claims (6)

Computer
A vehicle identification unit that identifies a preceding vehicle and a vehicle area occupied by the preceding vehicle in an image captured by the imaging device;
A light source candidate identification unit that identifies light source candidates as light source candidates in the identified vehicle region;
When the light emission source candidate satisfies a predetermined overexposure condition, the overexposure light source specifying unit that specifies the light emission source candidate as an overexposure light source,
An overexposure range setting unit for setting a predetermined range near the overexposure light source in the vehicle area as an overexposure range;
Among the identified light emitting source candidates, whether the light emitting source candidate is located outside the overexposure range is a lamp that is lit based on the number of pixels or the pixel area that satisfies a color condition of a predetermined intensity. Whether or not the light emitting source candidate located within the overexposure range is a lit lamp based on the number of pixels or the pixel area satisfying the color condition of higher intensity than the color condition. A vehicle exterior environment recognition device that functions as a lamp determination unit that determines whether or not. The computer counts the number of pixels satisfying the color condition of the predetermined intensity for the light source candidates located outside the overexposure range among the identified light source candidates, and calculates the number of pixels. The area is converted into an area based on the relative distance from the preceding vehicle, and the number of pixels that satisfy the color condition of higher intensity than the color intensity of the predetermined intensity is counted for the light emission source candidates located within the overexposure range. And further function as an area conversion unit that converts the number of pixels into an area based on a relative distance from the preceding vehicle,
The lamp determination unit determines that the light source candidate is a lit lamp when the converted area is equal to or greater than a predetermined lighting determination threshold value that the lamp of the preceding vehicle is lit. The outside environment recognition device according to claim 1 characterized by things. The lamp determination unit
When the area where the light source candidate located outside the overexposure range is equal to or greater than the first lighting determination threshold as the lighting determination threshold, determine that the light source candidate is a lamp in a lighting state;
When the area where the light emission source candidate located within the overexposure range is converted is equal to or higher than the second lighting determination threshold value that is higher than the first lighting determination threshold value as the lighting determination threshold value, the light emission source candidate is turned on. The outside environment recognition device according to claim 2, wherein it is determined that the lamp is a lamp. The area conversion unit is
For light emission source candidates located outside the overexposure range, the number of pixels satisfying color conditions of different intensities according to the relative distance from the preceding vehicle is counted, and the number of pixels is relative to the preceding vehicle. The outside environment recognition device according to claim 2 or 3, wherein the area is converted into an area based on the distance. Computer
A vehicle identification unit that identifies a preceding vehicle and a vehicle area occupied by the preceding vehicle in an image captured by the imaging device;
A light source candidate identification unit that identifies light source candidates as light source candidates in the identified vehicle region;
When the light emission source candidate satisfies a predetermined overexposure condition, the overexposure light source specifying unit that specifies the light emission source candidate as an overexposure light source,
An overexposure range setting unit for setting a predetermined range near the overexposure light source in the vehicle area as an overexposure range;
Among the identified light emission source candidates, for the light emission source candidates located outside the overexposure range, the number of pixels or the pixel area of the light emission source candidates is considered to be lit. If the threshold value is equal to or greater than the first lighting determination threshold value, the light source candidate is determined to be a lamp in a lighting state, and for the light source candidate located in the overexposure range, the number of pixels or pixels of the light source candidate When the area is equal to or greater than a second lighting determination threshold value that is higher than the first lighting determination threshold value, it functions as a lamp determination unit that determines that the light source candidate is a lamp in a lighting state. .   The vehicle exterior environment recognition device according to claim 5, wherein one of the number of pixels, the pixel area, and the first lighting determination threshold is set according to a relative distance from the preceding vehicle.

Images