JP2016038700A - Vehicle external environment recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a vehicle lamp including a prescribed light emission source group.SOLUTION: A vehicle external environment recognition device firstly generates region positional information indicating the position of a lamp region HR on an image. Then, it determines that one lamp region that corresponds to region positional information on a lamp region in a past image, and that is in a lamp region in a current image is a reference region KR. it extracts, as a similar region PR, a lamp region that satisfies a first similar condition set on the basis of the region positional information in the past image, and that is a lamp region other than the reference region in the lamp region in the current image. Next, it generates one integrated region TR by integrating the similar region into the reference region. If the integrated region is generated, it updates the region positional information on the reference region before integration and the region positional information on the similar region to the region positional information on the integrated lamp region.SELECTED DRAWING: Figure 12

Description

  The present invention relates to a vehicle environment recognition apparatus that identifies a vehicle lamp for a preceding vehicle.

  Conventionally, a specific object such as a vehicle positioned in front of the host vehicle is detected, and a collision with a preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is controlled to be a safe distance ( (Cruise control) technology is known (for example, Patent Document 1). Here, as the driver (person) is performing, if the process of recognizing whether the brake lamp of the preceding vehicle is lit (the operation state of the brake) or the like and estimating the deceleration operation of the preceding vehicle can be incorporated. Smoother cruise control is possible.

  As a technique for detecting whether or not the brake lamp of the preceding vehicle is turned on, an image obtained by capturing the road environment ahead of the host vehicle is acquired, and based on the positional relationship between the two regions corresponding to the light emission sources in the acquired image. Thus, a technique for determining whether or not a brake lamp is lit (for example, Patent Document 2) is disclosed.

Japanese Patent No. 3349060 International Publication No. 2014/002413

  By the way, in recent years, in addition to a brake lamp composed of a single light source such as a halogen lamp or an HID lamp (High Intensity Discharge Lamp), a brake lamp incorporating a plurality of LED lamps as a light source is becoming widespread.

  When it is attempted to detect whether or not a brake lamp including a plurality of light emitting sources (hereinafter referred to as “light emitting source group”) is turned on using the technique of Patent Document 2, the distance from the preceding vehicle is long. On the image, pixels indicating a plurality of light sources are connected to each other, and the light source group is recognized as one light source, so that one light source is recognized on each of the left and right sides. Therefore, it is possible to detect whether or not the brake lamp is lit based on the positional relationship between the left and right.

  However, when the distance from the preceding vehicle is short, each light source constituting the light source group is recognized as one light source, or a plurality of light sources constituting the light source group are divided into a plurality of groups. May be recognized. In this case, it may become impossible to detect a pair of light sources that have a positional relationship indicating a brake lamp. In this case, there is a possibility that it may be erroneously determined that the brake lamp is turned off even though the vehicle approaches the preceding vehicle whose brake lamp is turned on.

  Further, high mount stop lamps, lamps such as tail lamps, blinkers, and the like, which are configured by a predetermined light source group, may be erroneously determined as in the case of brake lamps.

  In view of such a problem, an object of the present invention is to provide a vehicle environment recognition device that can identify a vehicle lamp including a predetermined light source group.

  In order to solve the above-described problems, an external environment recognition device according to the present invention includes an image acquisition unit that acquires an image, a vehicle specification unit that specifies a vehicle region in the image, and a lamp region occupied by a lamp in the vehicle region. A lamp specifying unit for specifying a predetermined lamp when the lamp region satisfies a predetermined condition, a region position information generating unit for generating region position information indicating a position of the lamp region on the image, A reference area determining unit that determines, as a reference area, one lamp area corresponding to the area position information of the lamp area in a past image captured before the current image, among the lamp areas in the image; Among the lamp areas, a lamp area other than the reference area, which satisfies the first similarity condition set based on the area position information of the past image A similar region extraction unit that extracts as a similar region, an integrated region generation unit that integrates the similar region into the reference region to generate one integrated region, and when the integrated region is generated, An information update unit that updates the region position information of the reference region and the region position information of the similar region to the region position information of the integrated region, and the lamp specifying unit, when the integrated region is generated, The integrated area is specified as one lamp area.

  Further, the vehicle specifying unit predicts the behavior of the preceding vehicle with respect to the host vehicle, and the first similarity condition is indicated by the region position information of the past image derived according to the predicted behavior of the preceding vehicle. It may be set based on the position in the current image that is in a predetermined positional relationship with the position.

  Further, the vehicle specifying unit predicts the behavior of the preceding vehicle with respect to the host vehicle, and the first similarity condition is indicated by the region position information of the past image derived according to the predicted behavior of the preceding vehicle. It may be set based on a region predicted position that is a position in the current image corresponding to a position.

  The similar region extraction unit extracts a lamp region that satisfies the first similarity condition and satisfies a second similarity condition that is different from the first similarity condition as a similar region, and the second similarity condition includes: It may be set based on the number of times the lamp area is detected in a plurality of images taken at different timings.

  The image acquired by the image acquisition unit is a color image, and the similar region extraction unit selects a lamp region that satisfies the first similarity condition and satisfies a third similarity condition different from the first similarity condition. The similar region may be extracted, and the third similarity condition may be set based on the number of pixels that are color values within a predetermined color range.

  According to the present invention, it is possible to specify a vehicle lamp including a predetermined light source group.

It is the block diagram which showed the connection relation of the environment recognition system. It is explanatory drawing for demonstrating a color image and a distance image. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a specific thing table. It is a flowchart which shows a vehicle exterior environment recognition process. It is a flowchart which shows the vehicle specific process by a vehicle specific part. It is explanatory drawing for demonstrating a vehicle specific process. It is explanatory drawing for demonstrating a vehicle specific process. It is a flowchart which shows a lamp specific process. It is explanatory drawing for demonstrating the difference between the imaging by a 1st exposure aspect, and the imaging by a 2nd exposure aspect. It is a figure for demonstrating the misjudgment of the brake lamp in the vehicle by which the brake lamp which consists of a predetermined light emission source group was arranged. It is a figure for demonstrating an integration process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, an in-vehicle camera mounted on a vehicle images a road environment ahead of the host vehicle, identifies an object such as a preceding vehicle based on color information and position information in the captured image, and the identified object Vehicles equipped with a so-called collision prevention function that avoids such collisions and keeps the distance between the vehicle and the preceding vehicle at a safe distance (ACC: Adaptive Cruise Control) are becoming popular.

  In the ACC and the collision prevention function, for example, the relative distance of the object located in front of the own vehicle with the own vehicle is derived, and based on the relative distance, the collision with the object located in front of the own vehicle is detected. If the target is a vehicle (preceding vehicle), control is performed so that the relative distance from the preceding vehicle is kept at a safe distance. In addition, it is possible to realize smoother cruise control by incorporating a process for recognizing whether the brake lamp of the preceding vehicle is lit or not and estimating the deceleration operation of the preceding vehicle. Hereinafter, an environment recognition system for achieving such an object will be described, and a vehicle exterior environment recognition apparatus as a specific component thereof will be described in detail.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 includes an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), images an environment corresponding to the front of the host vehicle 1, and is represented by a color value. A color image can be generated. Here, the color value is a numerical group consisting of one luminance (Y) and two color differences (UV), or three hues (R (red), G (green), B (blue)). .

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates, for example, image data obtained by capturing an object existing in the detection area in front of the host vehicle 1 every frame (20 fps) for 1/20 second. Here, the objects to be recognized are not only three-dimensional objects such as vehicles, pedestrians, traffic lights, roads (traveling paths), guardrails, and buildings, but also brake lights, high-mount stop lamps, tail lights, blinkers, traffic lights. The thing which can be specified as a part of solid objects, such as each lighting part, is also included. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  Further, in the present embodiment, the imaging device 110 captures the detection region in the first exposure mode indicating the exposure time and the aperture according to the brightness of the environment outside the vehicle (measurement result of the illuminometer, etc.), and generates the first image. To do. In addition, the imaging device 110 generates an image that can determine whether or not a specific lamp such as a brake lamp emits light. As the method, an imaging element having a wide dynamic range may be used to capture an image so that an object that does not emit light is not crushed black and the lamp does not overshoot. The first exposure mode is an exposure mode (exposure time). The detection area may be imaged in a second exposure mode with different apertures to generate a second image. For example, during the daytime, the second image is generated by shortening the exposure time in the second exposure mode or increasing the aperture in comparison with the exposure time in the first exposure mode according to the bright outside environment. In the present embodiment, the first image and the second image are used as a color image and a distance image, respectively. Moreover, the said 1st exposure aspect and a 2nd exposure aspect are implement | achieved as follows.

  For example, the first image and the second image are sequentially generated by time-sharing the periodic image capturing timing of the image capturing apparatus 110 and alternately performing image capturing in the first exposure mode and image capturing in the second exposure mode. Can do. In addition, in an image pickup device in which two capacitors are provided for each pixel and charges can be charged in parallel to the two capacitors, two images having different exposure modes are set in parallel by changing the charging time in one exposure. It can also be generated. Further, the above-described object can be achieved by reading twice at different times and generating two images having different exposure modes in parallel while charging one capacitor. In addition, two sets of imaging devices 110 may be prepared in advance with different exposure modes (here, two imaging devices 110 × 2 sets), and images may be generated from the two sets of imaging devices 110, respectively. Is possible. The exposure time that governs the exposure mode is appropriately controlled within a range of 1 to 60 msec, for example.

  The outside environment recognition device 120 acquires image data from each of the two imaging devices 110, and selects a block corresponding to a block arbitrarily extracted from one image data (for example, an array of 4 horizontal pixels × 4 vertical pixels) on the other image. The parallax information including the parallax and the screen position indicating the position of the arbitrary block in the screen is derived using so-called pattern matching that is searched from the data. Here, the horizontal indicates the horizontal direction of the captured image, and the vertical indicates the vertical direction of the captured image. As this pattern matching, it is conceivable to compare the luminance (Y) in an arbitrary block unit between a pair of images. For example, SAD (Sum of Absolute Difference) that takes the difference in luminance value, SSD (Sum of Squared intensity Difference) that uses the difference squared, and the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel. There are methods such as NCC (Normalized Cross Correlation). The vehicle exterior environment recognition apparatus 120 performs such block-based parallax derivation processing for all blocks displayed in the detection area (for example, 600 pixels × 200 pixels). Here, the block is 4 pixels × 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the vehicle environment recognition apparatus 120 can derive the parallax for each block, which is a detection resolution unit, but cannot recognize what kind of target object the block is. Accordingly, the disparity information is derived independently not in units of objects but in units of detection resolution (for example, blocks) in the detection region. Here, an image in which the parallax information derived in this way is associated with image data is referred to as a distance image in distinction from the color image described above.

  FIG. 2 is an explanatory diagram for explaining the color image 126 and the distance image 128. For example, it is assumed that a color image (image data) 126 as shown in FIG. 2A is generated for the detection region 124 through the two imaging devices 110. However, only one of the two color images 126 is schematically shown here for easy understanding. The vehicle environment recognition apparatus 120 obtains the parallax for each block from the color image 126 and forms a distance image 128 as shown in FIG. Each block in the distance image 128 is associated with the parallax of the block. Here, for convenience of description, blocks from which parallax is derived are represented by black dots. In the present embodiment, such a color image 126 and distance image 128 are generated based on the first image and the second image, respectively. Therefore, in this embodiment, the color image 126 based on the first image, the distance image 128 based on the first image, the color image 126 based on the second image, and the distance image 128 based on the second image are used.

  Further, the outside environment recognition device 120 uses the three-dimensional position information in the real space including the color value based on the color image 126 and the relative distance from the host vehicle 1 based on the distance image 128, and the color values are equal and three-dimensional. Blocks having close positional information are grouped as objects, and the specific object (for example, a preceding vehicle) corresponding to the object in the detection area in front of the host vehicle 1 is specified. For example, the preceding vehicle can be specified by the relative distance or the like, and further, the position of the brake lamp and the presence / absence of lighting of the preceding vehicle can be grasped by the color value. By such processing, it is possible to quickly grasp the deceleration of the vehicle due to the lighting of the brake lamp and use it for collision avoidance control and ACC.

  The relative distance is obtained by converting the disparity information for each block in the distance image 128 into three-dimensional position information using a so-called stereo method. Here, the stereo method is a method of deriving a relative distance of the target object from the imaging device 110 from the parallax of the target object by using a triangulation method.

  The vehicle exterior environment recognition device 120, when specifying an object as an arbitrary specific object, for example, a preceding vehicle, derives a relative distance from the preceding vehicle, a relative speed of the preceding vehicle, and the like while tracking the preceding vehicle. It is determined whether or not there is a high possibility that the vehicle 1 and the host vehicle 1 collide with each other. At this time, if the brake lamp of the preceding vehicle is specified, the deceleration of the preceding vehicle can be recognized early by the lighting of the brake lamp. Here, when it is determined that there is a high possibility of a collision with the preceding vehicle, the outside environment recognition device 120 displays a warning (notification) to the driver through the display 122 installed in front of the driver to that effect, Information indicating that is output to the vehicle control device 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120. For example, when information indicating that there is a high possibility of a collision with a preceding vehicle is input from the outside environment recognition device 120, the vehicle control device 130 supports the driver's braking operation through the braking mechanism 146.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, the preceding vehicle and brake lamp specifying process, which is characteristic of the present embodiment, will be described in detail, and description of the configuration unrelated to the characteristics of the present embodiment will be omitted.

(Vehicle environment recognition device 120)
FIG. 3 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 3, the vehicle exterior environment recognition apparatus 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like, holds a specific object table and various information necessary for processing of each functional unit shown below, and receives image data received from the imaging device 110. (A color image 126 and a distance image 128 based on the first image and the second image) are temporarily stored. Here, the specific object table is defined as follows.

  FIG. 4 is an explanatory diagram for explaining the specific object table 200. In the specific object table 200, for a plurality of specific objects, a color range 202 indicating a range of color values (here, R, G, B), a height range 204 indicating a height range from the road surface, A specific object horizontal distance width range 206, a specific object vertical distance width range 208, a horizontal distance difference 210 with the same specific object, a vertical distance difference 212 with the same specific object, and the same specific object Are associated with each other. Here, as specific items, various items required for identifying a vehicle such as “brake lamp (red)”, “high-mount stop lamp (red)”, “tail lamp (red)”, “winker (orange)”, etc. Things are envisaged. However, it goes without saying that the specific object is not limited to the object shown in FIG. Among the specific objects, for example, the specific object “brake lamp (red)” has a color range (R) “200 or more”, a color range (G) “50 or less”, a color range (B) “50 or less”, a high Range “0.3-2.0 m”, horizontal distance width range “0.05-0.2 m”, vertical distance width range “0.05-0.2 m”, horizontal distance difference “1.4” ˜1.9 m ”, vertical distance difference“ 0.3 m or less ”, and area ratio“ 50 to 200% ”are associated with each other.

  Returning to FIG. 3, the central control unit 154 is composed of a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like. The / F unit 150, the data holding unit 152, and the like are controlled. In the present embodiment, the central control unit 154 includes an image acquisition unit 160, a position information deriving unit 162, an object specifying unit 164, a region position information generating unit 166, a region predicted position deriving unit 168, a reference region determining unit 170, It also functions as a similar region extraction unit 172, an integrated region generation unit 174, an information update unit 176, a position association unit 178, and an arrangement determination unit 180. Hereinafter, the operation of each functional unit will be described, and a vehicle environment recognition process characteristic of the present embodiment will be described in detail.

(External vehicle environment recognition processing)
FIG. 5 is a flowchart showing the external environment recognition process. The image acquisition unit 160 of the outside environment recognition device 120 is different from the first image obtained by capturing the detection region 124 in the first exposure mode according to the brightness of the outside environment from the imaging device 110, and the first exposure mode and the exposure mode are different. A second image obtained by imaging the detection area 124 in the second exposure mode is acquired (S300).

  Subsequently, the position information deriving unit 162 uses the stereo method described above to calculate the disparity information for each block in the detection area 124 in the distance image 128 based on the first image, the horizontal distance x, and the high (from the road surface). The information is converted into three-dimensional position information including the height y and the relative distance z (S302). Here, the parallax information indicates the parallax of each block in the distance image 128, while the three-dimensional position information indicates information on the relative distance of each block in the real space. Further, when the disparity information is derived not in pixel units but in block units, that is, in a plurality of pixel units, the disparity information may be regarded as disparity information of all pixels belonging to the block, and calculation in pixel units may be executed. it can. Regarding the conversion to the three-dimensional position information, since existing techniques such as JP2013-109391A can be referred to, detailed description thereof is omitted here.

  Next, the object specifying unit 164 specifies which specific object the object in the image corresponds to based on the first image and the second image. In the present embodiment, the object specifying unit 164 particularly specifies the preceding vehicle (leading vehicle) and the brake lamp (lamp) (vehicle specifying process S304, lamp specifying process S306). Therefore, in the following, among the object specifying unit 164, the function unit that executes the vehicle specifying process S304 that specifies the preceding vehicle is referred to as the vehicle specifying unit 164a, and the function unit that executes the lamp specifying process S306 that specifies the brake lamp is the lamp. This will be described as the specifying unit 164b. The object specifying unit 164 (the vehicle specifying unit 164a and the lamp specifying unit 164b) tracks (tracks) the object in which the specified object is specified, and the relative distance, relative speed, and relative of the object to the host vehicle 1 are tracked. The absolute speed and absolute acceleration of the preceding vehicle taking into account the acceleration and the traveling state of the host vehicle 1 are also detected.

  FIG. 6 is a flowchart showing the vehicle specifying process S304 by the vehicle specifying unit 164a, and FIGS. 7 and 8 are explanatory diagrams for explaining the vehicle specifying process S304. First, the vehicle identification unit 164a divides the detection area 124 of the distance image 128 based on the first image into a plurality of divided areas 216 in the horizontal direction (S304-1). Then, the divided region 216 has a strip shape as shown in FIG. Such a strip-shaped divided region 216 is originally composed of, for example, 150 columns each having a horizontal width of 4 pixels, but here, for convenience of explanation, the detection region 124 is divided into 16 parts.

  Subsequently, for each divided region 216, the vehicle specifying unit 164a integrates the relative distances included in each of the predetermined distances divided into a plurality of blocks located above the road surface based on the position information. A histogram (indicated by a horizontally long square (bar) in FIG. 7B, hereinafter referred to as “distance histogram”) is generated (S304-2). Then, a distance distribution 218 as shown in FIG. 7B is obtained. Here, the vertical direction indicates the divided predetermined distance (distance division), and the horizontal direction indicates the number of blocks (frequency) in which the relative distance is included in each distance division. However, FIG. 7B is a virtual screen for calculation, and actually does not involve the generation of a visual screen. Then, the vehicle identification unit 164a refers to the distance distribution 218 derived in this way, and represents a representative distance 220 (indicated by a black square in FIG. 7B) that is a relative distance corresponding to the peak. Specify (S304-3). Here, “corresponding to a peak” means a peak value or a value that satisfies an arbitrary condition in the vicinity of the peak.

  Next, the vehicle specifying unit 164a compares adjacent divided areas 216 with each other, and groups the divided areas 216 with a representative distance 220 close (for example, located at 1 m or less) as shown in FIG. A plurality of divided region groups 222 are generated (S304-4). At this time, even when the representative distance 220 is close in three or more divided areas 216, all the continuous divided areas 216 are collected as a divided area group 222. By such grouping, the vehicle specifying unit 164a can specify a three-dimensional object located above the road surface.

  Subsequently, the vehicle specifying unit 164a sets a block whose relative distance z corresponds to the representative distance 220 in the divided region group 222 as a base point, the difference between the block, the horizontal distance x, the difference in the height y, and the relative distance z. Are grouped on the assumption that they correspond to the same specific object (S304-5). Thus, an object 224 that is a virtual block group is generated. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. The vehicle specifying unit 164a also has a block in which the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z are within a predetermined range with respect to the block newly added by grouping. Are further grouped. As a result, all blocks that can be assumed to be the same specific object are grouped.

Here, the horizontal distance x difference, the height y difference, and the relative distance z difference are determined independently, and only when all are included in a predetermined range, the same group is used. You can also. For example, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (difference in relative distance z) ² ) Are included in the predetermined range, the same group may be used. With this calculation, an accurate distance between blocks in real space can be derived, so that the grouping accuracy can be improved.

  Next, if the grouped objects 224 satisfy a predetermined condition corresponding to a predetermined vehicle, the vehicle identification unit 164a determines the object 224 as the specific object “vehicle” (S304−). 6). For example, when the grouped objects 224 are located on a road, the vehicle specifying unit 164a determines whether the size of the entire object 224 corresponds to the size of the specific object “vehicle”. If it is determined that the size corresponds to the size of the specific object “vehicle”, the target object 224 is specified as the specific object “vehicle”. Here, the vehicle specifying unit 164a sets a region corresponding to a circumscribed rectangle of a region occupied on the screen by the object 224 specified as the specific object “vehicle” as a vehicle region. Note that the circumscribed rectangle is a rectangle, and the horizontal direction of the outer edge is parallel to the horizontal direction of the image, and the vertical direction of the outer edge is parallel to the vertical direction of the image.

  Subsequently, based on the position information derived by the position information deriving unit 162, the vehicle identifying unit 164a derives the relative distance z and the horizontal distance x of the object 224 identified as the specific object “vehicle” (S304−). 7). In addition, the vehicle specifying unit 164a acquires the speed and yaw rate of the host vehicle 1 from the vehicle control device 130 through the system bus 156 and the I / F unit 150 (S304-8). Then, the vehicle specifying unit 164a derives a predicted vehicle position based on the relative distance z, the horizontal distance x, the speed of the host vehicle 1, and the yaw rate (S304-9). The predicted vehicle position is a position predicted (estimated) with the relative position of the object 224 (vehicle) in the next frame.

  As described above, the vehicle environment recognition apparatus 120 can extract one or more objects 224 as a specific object, for example, a vehicle (preceding vehicle), from the distance image 128 based on the first image. Information can be used for various controls. For example, when an arbitrary object 224 in the detection area 124 is identified as a vehicle, the identified vehicle (preceding vehicle) is tracked, relative distance and relative acceleration are derived, and a collision with the preceding vehicle is avoided. Or the distance between the preceding vehicle and the preceding vehicle can be controlled to be kept at a safe distance.

  Subsequently, the lamp specifying unit 164b executes a lamp specifying process S306. FIG. 9 is a flowchart showing the lamp specifying process S306. The lamp specifying unit 164b acquires color values of three hues (R, G, B) in units of pixels from the color image 126 based on the second image (S306-1). At this time, if the detection area 124 is rainy or cloudy, for example, the lamp specifying unit 164b may acquire the color after adjusting the white balance so that the original color value can be acquired.

  The lamp specifying unit 164b temporarily specifies a predetermined lamp based on the specified object table 200 held in the data holding unit 152 and the color value of each pixel of the color image 126 based on the second image (S306-2). Specifically, the lamp specifying unit 164b selects and acquires a specific lamp (here, “brake lamp”) associated with the second exposure mode from the specific objects registered in the specific object table 200. It is determined whether or not the color value of the pixel is included in the color range 202 of the selected specific object. If the pixel is included in the target color range 202, the pixel is assumed to be the specific object “brake lamp”.

  As described above, the second image is an image captured in the second exposure mode that can determine whether or not a specific lamp, for example, a specific object “brake lamp” emits light. Here, a specific object such as a “brake lamp” that emits light can acquire a high color value regardless of the brightness of the sun or a streetlight. In particular, since the brightness when the specific object “brake lamp” is lit is generally stipulated by laws and regulations, by capturing an image in an exposure mode (for example, short-time exposure) in which only a predetermined brightness can be exposed, the specific object “ Only pixels corresponding to “brake lamps” can be easily extracted.

  FIG. 10 is an explanatory diagram for explaining a difference between imaging according to the first exposure mode and imaging according to the second exposure mode. FIG. 10A shows a first image according to the first exposure mode. In particular, the tail lamp is lit in the left figure of FIG. 10A, and the brake lamp in addition to the tail lamp is shown in the right figure of FIG. Is lit. As can be understood with reference to FIG. 10A, in the first exposure mode according to the brightness of the outside environment, the color value of the tail lamp position 230 when the brake lamp is not turned on and the tail lamp is turned on, the brake lamp is turned on and the tail lamp is turned on. There is almost no difference in color value between the brake lamp position 232 at the time of lighting. This is because, in the first exposure mode with a long exposure time, the color values of all RGB components of the tail lamp and the brake lamp are saturated.

  FIG. 10B shows a second image according to the second exposure mode. In particular, the tail lamp is lit in the left figure of FIG. 10B, and the brake lamp in addition to the tail lamp is shown in the right figure of FIG. Is lit. The second exposure mode is set so that only the color value when the brake lamp is lit can be acquired. Therefore, as can be understood with reference to FIG. 10B, even when the tail lamp is lit, the tail lamp position 230 hardly obtains a color value according to the brightness, and the brake lamp position 232 when the brake lamp is lit. Then, a clear high color value can be obtained.

  In such a second exposure mode, it is desirable that the color value of the brake lamp is set to an exposure time such as whether or not the R component is saturated in the image sensor. Since the imaging device 110 normally has a dynamic range that is significantly narrower than that of humans, when the first exposure mode is used with a brightness as low as in the evening, the color value of the brake lamp increases relative to the environment outside the vehicle. Then, not only the R component but also the R component overlap, and the G component and the B component are saturated to the maximum value (for example, the color value is 255), and the pixel becomes white. Therefore, by setting the second exposure mode to an exposure time of whether or not the R component is saturated when the brake lamp is lit, the influence on the color values of the G and B components is suppressed regardless of the external environment. However, only the R component is extracted with the maximum value. Thus, for example, it is possible to secure the maximum color value difference from the tail lamp.

  Specifically, when there is a preceding vehicle when traveling at night, the extent to which the tail lamp is lit, for example, color range (R) “50”, color range (G) “50”, color range (B) “ If it is about 50 ", it is not displayed in the second image. In contrast, when the brake lamp is lit, as shown in the specific object table 200 of FIG. 4, the color range 202 has a color range (R) “200 or more” and a color range (G) “50 or less”. The color range (B) is “50 or less”, and even if an image is captured in the second exposure mode, it is displayed on the second image to such an extent that the position can be grasped. In this way, the lamp specifying unit 164b can specify only a predetermined lamp such as a brake lamp through the second image. Although the exposure time according to the second exposure mode is fixed here, it may be adjusted spontaneously according to the environment outside the vehicle or according to the operation of the passenger.

  In addition, the lamp specifying unit 164b, when the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z between pixels that are candidates for the lamp are within a predetermined range (for example, 0.1 m), Are grouped as one lamp candidate (S306-3). Then, the lamp specifying unit 164b specifies a region (lamp region) corresponding to a circumscribed rectangle of the region occupied by the grouped lamp candidates on the screen.

  When the lamp areas HR having similar sizes are extracted from the grouped lamp areas HR, and it is determined whether or not the brake lamp is a brake lamp based on the left and right positional relationship of the extracted lamp areas HR, Actually, there is a possibility that it is erroneously determined that the preceding vehicle is turned off although the brake lamp of the preceding vehicle is turned on. Specifically, if the preceding vehicle is a vehicle provided with a brake lamp composed of a predetermined light source group, it is erroneously determined that the brake lamp is turned off even though the brake lamp is lit. There is a case.

  FIG. 11 is a diagram for explaining an erroneous determination of a brake lamp in a vehicle provided with a brake lamp including a predetermined light source group. For example, as shown in FIG. 11 (a), a vehicle in which brake lamps (having a plurality of LED lamps built in as a light source) are arranged on the left and right sides of the rear portion of the vehicle. To do. In this case, if the relative distance z to the host vehicle 1 is a long distance (for example, 30 m or more), as shown in FIG. 11B, pixels indicating a plurality of light sources on the second image are connected. The light emission source group is detected as one lamp region HR. Further, when the relative distance z with the host vehicle 1 is a medium distance (for example, 20 m to 30 m), as shown in FIG. 11C, a plurality of light emitting sources constituting the light emitting source group are displayed on the second image. In this case, the lamp area HR may be detected as a lamp area HR. Furthermore, if the relative distance z to the host vehicle 1 is a short distance (for example, less than 15 m), depending on the setting of a predetermined range for grouping, as shown in FIG. In some cases, each individual light source constituting the light source group is detected as one lamp region HR. That is, as the preceding vehicle approaches, the number of ramp areas HR derived from brake lamps increases.

  In the case of a long distance shown in FIG. 11B, lighting of the brake lamp can be detected even if it is determined whether or not the grouped lamp area HR is a brake lamp based on the left and right positional relationship. However, in the case of the intermediate distance shown in FIG. 11C, the number of left and right lamp areas HR is different, and there is a possibility that the partner lamp area HR that satisfies the pair condition cannot be detected in each lamp area HR. For example, in FIG. 11C, when the left light source group is detected as three lamp regions HR and the right light source group is detected as one lamp region HR, both of the left lamp regions HR The size of the right lamp area HR is different. For this reason, there is a possibility that the partner lamp area HR that satisfies the pair condition cannot be detected and it is erroneously determined that the brake lamp is turned off.

  In addition, in the case of the short distance shown in FIG. 11 (d), each individual light source constituting one light source group is detected as one lamp region HR, and therefore, the partner lamp region that satisfies the pair condition. There is a possibility that the HR cannot be specified and it is erroneously determined that the brake lamp is turned off.

  Furthermore, there is a problem even when using learning logic that derives a change in the area of the lamp area HR in the vehicle area of the preceding vehicle over a plurality of frames and determines whether or not the lamp is lit based on the change in the area. Arise. For example, when the vehicle is gradually approaching a preceding vehicle whose brake lamp is lit, one light source group is detected as one lamp region HR in the frame (fn), and one in the next frame (fn + 1). It is assumed that the light emission source group is detected as a plurality of lamp regions HR. When the learning logic is executed using the detection result and it is learned that the brake lamp is lit in the frame (fn), the area of the lamp region HR becomes small in the frame (fn + 1) and approaches the preceding vehicle. In spite of this, there is a risk of misrecognizing that the brake lamp is turned off. Note that the learning logic can refer to existing technology such as Japanese Patent Application No. 2014-073374, and a detailed description thereof will be omitted here.

  Therefore, in the present embodiment, an integration process that can improve the accuracy of determination of whether or not a brake lamp that includes a predetermined light-emitting source group is turned on by integrating a plurality of lamp regions HR based on a predetermined similar condition. Run. Hereinafter, the integration process will be described.

(Integration process)
FIG. 12 is a diagram for explaining the integration process. However, FIG. 12 is a virtual screen for calculation, and actually does not involve the generation of a visual screen. Hereinafter, an image (second image) used when executing the current integration process is a current image, and an image (second image) captured before the current image, that is, executed before the current integration process. The image used in the integration process is referred to as a past image. Therefore, the image that is the execution target of the current integration process is a past image in the next integration process.

  In the integration process, first, the region position information generation unit 166 generates region position information indicating the position on the image of the lamp region HR specified in S306-3 (S306-4). Then, the region position information generation unit 166 determines whether or not the region position information of the lamp region HR in the past image is stored in the RAM (storage unit) (S306-5). As will be described in detail later, the region position information is stored in association with each past image acquired by the image acquisition unit 160. Therefore, it is determined that the region position information is not stored as a result of the storage determination process S306-5, which means that the ramp region HR has not been specified in the vehicle region so far. That is, the storage determination process S306-5 is a process of determining whether or not the lamp area HR has been specified in the vehicle area.

  If it is determined that the region position information is not stored (NO in S306-5), the region position information generation unit 166 associates the region position information generated in S306-4 with the current image acquired by the image acquisition unit 160. Is stored in the RAM (S360-6), and the process proceeds to S306-13. For example, as shown in FIGS. 12A and 12B, the lamp region HR was not specified in the previous light emission region specifying process S306-3 (past image, frame (fn-1)). This is a case where the ramp area HR is specified for the first time in the area specifying process S306-3 (current image, frame (fn)). Here, the region position information generation unit 166 sets 1 to the number of detections DN for the ramp region HR specified for the first time.

  On the other hand, when the region position information generating unit 166 determines that the region position information is stored (YES in S306-5), the region predicted position deriving unit 168 derives the region predicted position RE (S306-7). Here, the region predicted position RE is corrected with respect to the region position information (region position information of the past image) stored in the RAM using the vehicle predicted position derived by the vehicle specifying unit 164a in S304-9. Thus, in the frame (frame fn + 1), it is the position on the current image where the ramp region HR is predicted to be located (see FIG. 12C). In other words, the region predicted position RE is a position in the current image corresponding to the position indicated by the region position information of the past image, which is derived according to the behavior of the host vehicle 1 and the preceding vehicle. Further, since the predicted region position RE is based on the ramp region HR detected once in the previous frame fn, the number of detections DN of the predicted region position RE is detected in association with the ramp region HR in the previous frame fn. The number of times DN (DN = 1 in FIG. 12C) is taken over.

  The reference region determination unit 170 selects one ramp corresponding to the region predicted position RE from among the ramp regions HR identified in S306-3 (indicated by HRa, HRb, and HRc in FIGS. 12C to 12I). The region HRa is determined as the reference region KR (see FIG. 12D). In addition, the reference area determination unit 170 sets the detection count DN of the reference area KR to the count obtained by incrementing the detection count DN of the area prediction position RE by 1 (S306-8). More specifically, for example, the reference area determination unit 170 includes, for example, the lamp area HRa whose distance of the center of gravity is closest to the center of gravity (center) of the predicted area RE in the lamp area HR specified by the lamp specifying unit 164b. Is determined as the reference region KR.

  Subsequently, the similar region extraction unit 172 derives an enlarged predicted position KE obtained by enlarging the region predicted position RE. Here, an enlarged predicted position KE indicating the position of a region obtained by enlarging the region indicated by the predicted region position RE 50% vertically and horizontally is derived (see S306-9, FIG. 12E).

  Then, the similar area extraction unit 172 is a lamp area HR other than the reference area KR among the lamp areas HR identified in S306-3, and includes a predetermined first similarity condition, second similarity condition, and The lamp region HR that satisfies the third similarity condition is extracted as the similar region RR (S306-10).

  The first similarity condition is a condition set based on the predicted region position RE, and here, the ramp region HR is included in the enlarged predicted position KE. The second similarity condition is set based on the number of detections, for example, the number of detections is one.

  The third similarity condition is to satisfy a predetermined hue condition. The hue condition is a color value within a predetermined color range (a hue different from the brake lamp, for example, R component color value> 150, G component color value> R × 24/32, B component color value> R × 14). / 32) is set based on the number of pixels (specifically, the number of pixels indicating yellow). Specifically, the third similar condition is that the number of yellow pixels in the lamp region HR is 1.1 times or less the number of yellow pixels in the reference region KR.

  Here, first, the similar region extraction unit 172 extracts the ramp region HR that satisfies the first similarity condition, that is, the ramp region HR included in the predicted expansion position KE, and detects each ramp region HR. The number of times DN is incremented by 1 (see FIG. 12F). Then, the similar area extraction unit 172 extracts a lamp area HR that satisfies the second similarity condition, that is, a lamp area HR with one detection DN, from among the lamp areas HR included in the enlarged predicted position KE. Further, the similar region extraction unit 172 includes a yellow region in the lamp region HR among the lamp regions HR that satisfy the third similarity condition, that is, the lamp region HR that is included in the enlarged predicted position KE and has one detection number DN. Are extracted as the similar region RR (FIG. 12 (g)). The ramp region HRb is 1.1 times the number of yellow pixels in the reference region KR.

  Since the predicted region position RE is predicted based solely on the behavior of the host vehicle 1 and the preceding vehicle, an error occurs between the predicted region position RE and the position of the ramp region HR corresponding to the current brake lamp. Therefore, the similar region extraction unit 172 employs the first similar condition in order to extract the similar region RR that is the ramp region HR similar to the reference region KR, thereby expanding an area that is wider than the region predicted position RE. Similar regions RR can be extracted using the predicted position KE, and errors due to prediction can be absorbed.

  Further, the similar region extraction unit 172 employs the ramp region HR with one detection as the second similarity condition, so that the ramp region HR detected by the previous frame (in the past image) The lamp region HR that is not determined to be a lamp can be excluded from the similar region RR.

  The lamp area HR includes not only the lamp area HR corresponding to the brake lamp but also a lamp area HR corresponding to a lamp having a hue different from that of the brake lamp (for example, a blinker). Therefore, the similar region extraction unit 172 determines, as a third similar condition, the number of pixels having a color value within a predetermined color range (a hue different from the brake lamp) as a predetermined value in the reference region KR associated with the brake lamp. By setting only the lamp region HR that is 1.1 times or less the number of pixels that are the color values in the color range as the similar region RR, the lamp region HR other than the brake lamp can be excluded from the similar region RR.

  Subsequently, the integrated region generation unit 174 generates one integrated region TR corresponding to a circumscribed rectangle by integrating one or a plurality of similar regions RR into the reference region KR, and the reference region updated in S306-8 The number of detections DN of KR is set as the number of detections of the integrated region TR (S306-11, see FIG. 12 (i)). Note that the ramp region HRc that has not been integrated by the integrated region generation unit 174 corresponds to the ramp region HR in FIG. 12B, and therefore the predicted region position RE is derived in the next frame (fn + 2). -4 to S306-11 are executed.

  The information update unit 176 derives the region position information with the integrated region TR as the new ramp region HR, deletes the region position information of each lamp region HR before integration stored in the RAM, and sets the new lamp region HR. The area position information is newly stored (updated) in the RAM in association with the current image (S306-12). Further, the information update unit 176 deletes the area position information stored in the RAM, triggered by the fact that the lamp area HR has not been specified for a predetermined time.

  In addition, when the integrated region TR is generated, the lamp specifying unit 164b sets the integrated region TR as a new lamp region HR, and sets the size of one lamp region HR as a predetermined threshold (for example, horizontal and vertical). The lamp candidates included in one lamp region HR are specified as lamps only when the width is 0.05 m) or more (S306-13). In addition to the size, the lamp specifying unit 164b may also use a candidate lamp shape included in the lamp region HR as a condition. For example, when the brake lamp has a shape that extends in the vertical direction at the left and right ends of the rear portion of the vehicle, it is determined that the brake lamp has a shape that can be regarded as a brake lamp as well as its size. In this way, it is possible to exclude a lamp corresponding to noise that should not be regarded as a predetermined lamp and extract a brake lamp (lamp), so that a specific object can be specified with high accuracy.

  In this way, the brake lamp can be extracted with high accuracy by the lamp specifying unit 164b. However, with only the second image in the second exposure mode, the color value of the entire detection region 124 becomes low (dark) at night or the like, and nothing can be grasped except for a lamp such as a brake lamp. Therefore, the “brake lamp” is associated with the “vehicle” specified by the first image according to the first exposure mode described above.

  The position associating unit 178 associates the vehicle area grouped as the specific object “vehicle” by the vehicle specifying unit 164a with the position of the lamp (brake lamp) specified by the lamp specifying unit 164b (S306-14). Then, the tracking of the specific object “vehicle” by the vehicle specifying unit 164a and the tracking of the specific object “brake lamp” by the lamp specifying unit 164b are supported, and the position information of the other is calibrated. Thus, the positional relationship between the outer edge of the preceding vehicle and the brake lamp of the vehicle can be maintained.

  The arrangement determination unit 180 identifies a pair of brake lamp combinations that are assumed to exist in the same preceding vehicle, and is associated by the position association unit 178 based on the specific object table 200 illustrated in FIG. It is determined whether or not the relative arrangement of the vehicle area and the position of the brake lamp as a lamp is an appropriate arrangement (S306-15). For example, the position determination unit 180 may be configured such that the brake lamp itself has a height range of “0.3 to 2.0 m”, a horizontal distance width range of “0.05 to 0.2 m”, and a vertical distance width range of “0”. .05 to 0.2 m ”. Further, the arrangement determining unit 180 is configured such that a pair of brake lamps has a horizontal distance difference “1.4 to 1.9 m”, a vertical distance difference “0.3 m or less”, and an area ratio “50 to 200%”. It is determined whether or not the above condition is satisfied. By satisfying such a condition, it is specified that the brake lamp of the preceding vehicle is lit. In this way, a configuration that is officially specified as a brake lamp only when the lamp that is assumed to be a brake lamp corresponds to the appropriate position of the vehicle, only one lamp with the same brightness, such as a rear fog lamp, is lit. It is possible to prevent erroneous recognition of the lamps that are present.

  As described above, according to the external environment recognition device 120 according to the present embodiment, it is possible to specify a vehicle lamp composed of a predetermined light source group.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the lamp specifying unit 164b determines whether the brake lamp is lit based on the color image 126 generated by the imaging device 110, but determines whether the brake lamp is lit based on the monochrome image. You may do that.

  In the above-described embodiment, the lamp identification unit 164b derives the relative distance z and the horizontal distance x of the object 224 identified as the specific object “vehicle”, but in addition to this, derives the vertical distance. Thus, the predicted vehicle position may be derived.

  In the above embodiment, the reference region determination unit 170 determines one ramp region HRa corresponding to the region predicted position RE of the current image as the reference region KR, so that the reference region KR uses the region predicted position RE as the reference region KR. Indirectly associated with the area position information of the past image. However, the reference area determination unit 170 may determine one lamp area HRa corresponding to the area position information of the past image as the reference area KR, and the area position information of the past image and the reference area KR may be directly associated with each other. .

  In the above embodiment, the first similarity condition is that the ramp region HR is included in the enlarged predicted position KE. However, the first similarity condition may be a condition set based on the predicted region position RE. For example, the ramp region HR may be included in the region predicted position RE. In any case, the first similarity condition is set based on the position in the current image that has a predetermined positional relationship with the position indicated by the area position information of the past image, which is derived according to the predicted behavior of the preceding vehicle. That's fine.

  Further, the first similarity condition may be a condition set based on the area position information of the past image instead of the condition set based on the area predicted position RE. For example, the lamp area HR may be included in the position indicated by the area position information of the past image.

  In the above embodiment, the similar region extracting unit 172 extracts the ramp region HR that satisfies the first similar condition and further satisfies the second similar condition and the third similar condition as a similar region RR. Explained. However, the similar region extraction unit 172 may extract a ramp region HR that satisfies the first similarity condition and satisfies either the second similarity condition or the third similarity condition as the similar region RR. The similar region extraction unit 172 may extract at least the lamp region HR that satisfies the first similarity condition as the similar region RR.

  Moreover, in the said embodiment, the vehicle environment recognition apparatus 120 demonstrated and demonstrated the brake lamp as an example as a vehicle lamp which consists of a predetermined light emission source group. However, the vehicle exterior environment recognition device 120 can identify a blinker, a rear fog lamp, or the like as long as it is a vehicle lamp composed of a predetermined light source group.

  It should be noted that each step of the vehicle environment recognition processing in the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an external environment recognition device that identifies a vehicle lamp for a preceding vehicle.

120 External environment recognition device 160 Image acquisition unit 164a Vehicle identification unit 164b Lamp identification unit 166 Region position information generation unit 170 Reference region determination unit 172 Similar region extraction unit 174 Integrated region generation unit 176 Information update unit

Claims (5)

An image acquisition unit for acquiring images;
A vehicle specifying unit for specifying a vehicle region in the image;
In the vehicle area, a lamp area that a lamp occupies is specified, and a lamp specifying unit that specifies a predetermined lamp when the lamp area satisfies a predetermined condition;
An area position information generating unit that generates area position information indicating the position of the lamp area on the image;
A reference area determination unit that determines, as a reference area, one lamp area corresponding to the area position information of the lamp area in a past image captured before the current image, among the lamp areas in the current image;
Similar region extraction that extracts, as a similar region, a lamp region other than the reference region that satisfies the first similarity condition set based on the region position information of the past image among the ramp regions in the current image. And
An integrated region generating unit that integrates the similar region into the reference region to generate one integrated region;
When the integrated area is generated, an information update unit that updates the area position information of the reference area before the integration and the area position information of the similar area to the area position information of the integrated area;
With
When the integrated area is generated, the lamp specifying unit specifies the integrated area as one lamp area. The vehicle specifying unit predicts the behavior of a preceding vehicle relative to the host vehicle,
The first similarity condition is set based on a position in the current image that is derived according to the predicted behavior of the preceding vehicle and has a predetermined positional relationship with the position indicated by the region position information of the past image. The outside environment recognition device according to claim 1 characterized by things. The vehicle specifying unit predicts the behavior of a preceding vehicle relative to the host vehicle,
The first similarity condition is set based on a predicted area position that is a position in the current image corresponding to a position indicated by area position information of the past image, which is derived according to the predicted behavior of the preceding vehicle. The outside environment recognition device according to claim 1 characterized by things. The similar region extraction unit extracts a lamp region that satisfies the first similarity condition and satisfies a second similarity condition different from the first similarity condition as a similar region,
The vehicle exterior environment recognition according to any one of claims 1 to 3, wherein the second similarity condition is set based on the number of times of detection of a lamp area in a plurality of images taken at different timings. apparatus. The image acquired by the image acquisition unit is a color image,
The similar region extraction unit extracts a lamp region that satisfies the first similarity condition and satisfies a third similarity condition different from the first similarity condition as a similar region;
The external environment recognition device according to any one of claims 1 to 3, wherein the third similarity condition is set based on the number of pixels that are color values within a predetermined color range.

Images