JP6378547B2 - Outside environment recognition device - Google Patents

Description

  The present invention relates to a vehicle environment recognition apparatus that identifies a specific object existing around a host vehicle.

  Conventionally, a specific object such as a vehicle positioned in front of the host vehicle is specified, and a collision with a preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is controlled to be kept at a safe distance ( (Cruise Control) technology is known. Such forward monitoring technology of the own vehicle is expected to be effective in avoiding and reducing contact accidents with specific objects such as preceding vehicles and pedestrians.

  As a specific object identification technique, for example, a technique is known in which an obstacle is detected by a radar device and whether or not the obstacle is a stationary object is determined based on the relative speed between the vehicle and the obstacle ( For example, Patent Document 1). In addition, a technique for determining the type of a three-dimensional object based on the edge direction and edge strength distribution of an edge component in a determination target region where a three-dimensional object exists is disclosed (for example, Patent Document 2).

JP 2007-278892 A JP 2007-156626 A

  However, in the technique of Patent Document 1 described above, since the solid object is determined only by the radar device, the shape of the solid object cannot be recognized, and the specific object is inferior in accuracy. Moreover, in the technique of Patent Document 2, although the accuracy of specifying a specific object increases according to the edge direction and the distribution of edge strength, the change in the environment outside the vehicle such as day and night, and the change in the irradiation range of the light source to the determination target associated therewith Not adapted to. Therefore, for example, at night, the irradiation range of the preceding vehicle when the headlight is turned on varies depending on the relative distance from the preceding vehicle, and the edge characteristics fluctuate above and below the preceding vehicle. However, in the technique of Patent Document 2, since the influence of the irradiation range of the preceding vehicle by the headlight is not taken into consideration, the edge component of the road surface is included in the edge component of the determination target region by the headlight from the own vehicle at night. There is a problem that the detection of the edge component is not sufficient, and the preceding vehicle cannot be specified appropriately.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a vehicle environment recognition apparatus that can improve the accuracy of identifying a specific object regardless of changes in the vehicle environment or irradiation range.

  In order to solve the above-described problem, the vehicle environment recognition apparatus according to the present invention has a boundary that divides a stereoscopic image in the vertical direction based on the vertical luminance distribution of the stereoscopic image in the image captured by the imaging device. A boundary deriving unit for deriving, and a specific object specifying unit that determines whether a stereoscopic image is a predetermined specific object based on an edge component using an image having a higher luminance in the vertical direction above or below the boundary. It is characterized by.

  The boundary deriving unit may generate a plurality of first divided regions extending in the horizontal direction by dividing the stereoscopic image in the vertical direction, and deriving the boundary based on a difference in luminance between the plurality of first divided regions. .

  A light / dark determination unit that determines whether the image is in a bright state or a dark state. A plurality of second divided regions extending in the vertical direction, and the number of vertical edges extending in the vertical direction in the second divided region located in the center among the plurality of second divided regions; An edge counting unit that counts the number of vertical edges in the second divided region located on the left and right, and the specific object specifying unit includes the number of vertical edges in the second divided region located in the center, It may be determined whether or not the stereoscopic image is a predetermined specific object based on the number of vertical edges in the second divided region.

  The specific object specifying unit is configured so that the number of vertical edges in the second divided region located at the center is less than a predetermined threshold value, and the vertical edges in the second divided regions located on the left and right are equal to or larger than the predetermined threshold value. The stereoscopic image may be determined to be a vehicle.

  According to the present invention, it is possible to improve the accuracy of specifying a specific object regardless of the environment outside the vehicle and the irradiation range.

It is the block diagram which showed the connection relation of the environment recognition system. It is explanatory drawing for demonstrating the change of the irradiation range of the light source to the three-dimensional image by the change of a vehicle exterior environment. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is a flowchart which shows the flow of a vehicle exterior environment recognition process. It is explanatory drawing for demonstrating a luminance image and a distance image. It is explanatory drawing which illustrated the grouping process. It is explanatory drawing which showed the other example of the grouping process. It is explanatory drawing which showed the other example of the grouping process. It is explanatory drawing for demonstrating the restriction | limiting of a three-dimensional image. It is explanatory drawing for demonstrating the process of a brightness determination part. It is explanatory drawing for demonstrating operation | movement of a boundary derivation | leading-out part. It is explanatory drawing for demonstrating operation | movement of an edge counting part. It is explanatory drawing for demonstrating an edge direction. It is explanatory drawing for demonstrating the appearance aspect of a vertical edge.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 is configured to include an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). The imaging device 110 captures an environment corresponding to the front of the host vehicle 1 and brightness by luminance (Y). An image (monochrome image) or a color image with color values can be generated. Here, the color value is from a YUV format color space consisting of one luminance (Y) and two color differences (U, V), and three hues (R (red), G (green), B (blue)). A numerical value group represented by any one of the RGB color space or the HSB color space consisting of hue (H), saturation (S), and brightness (B). In the present embodiment, it is sufficient that at least the luminance (Y) is provided, and any one of the luminance image and the color image is sufficient. Therefore, the imaging apparatus 110 will be described as generating a luminance image.

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates a luminance image obtained by imaging a three-dimensional object existing in the detection area in front of the host vehicle 1, for example, every 1/60 second frame (60 fps). Here, the three-dimensional object to be recognized and the specific object to be specified in association with the three-dimensional object are three-dimensionally independent such as vehicles, pedestrians, bicycles, traffic lights, roads, guardrails, buildings, road side walls, and steep slopes. In addition to existing objects, there are also those that can be specified as part of specific objects such as taillights, blinkers, and lighting parts of traffic lights. One of the objects of the present embodiment is to specify a three-dimensional image (three-dimensional object image) included in the luminance image as a specific object, for example, a vehicle. Each functional unit in the following embodiment performs each process for each frame triggered by such acquisition (update) of the luminance image.

  The vehicle environment recognition device 120 acquires luminance images from the two imaging devices 110, derives parallax using so-called pattern matching, and associates the derived parallax information (corresponding to a relative distance described later) with the luminance image. Generate a distance image. The luminance image and the distance image will be described in detail later. Further, the vehicle exterior environment recognition device 120 extracts a stereoscopic image in a detection area in front of the host vehicle 1 using the luminance based on the luminance image and the relative distance z to the host vehicle 1 based on the distance image, and The specific object to be controlled is specified from the inside.

  Further, when the outside environment recognition device 120 identifies a specific object, the specific object and the host vehicle 1 may collide with each other by deriving the relative speed of the specific object while tracking the specific object (for example, a preceding vehicle). Judgment is made whether or not the property is high. Here, when it is determined that the possibility of the collision is high, the outside environment recognition device 120 displays a warning (notification) to the driver through the display 122 installed in front of the driver, and the vehicle control device. Information indicating that is output to 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120.

  As described above, in the environment recognition system 100, for example, a preceding vehicle existing in front of the host vehicle 1 is identified from the three-dimensional images included in the luminance image, and follow-up control to the preceding vehicle, The collision avoidance control is executed. Therefore, in the environment recognition system 100, it is necessary to accurately determine whether the three-dimensional object existing in front of the host vehicle 1 is a moving object such as a preceding vehicle or a stationary object such as a road side wall or a steep slope. . Therefore, for example, based on the edge component (edge direction or edge strength) of the extracted stereoscopic image, it can be determined whether or not the stereoscopic image is a vehicle.

  However, if you try to identify a specific object (for example, a preceding vehicle) simply based on the edge component without considering changes in the outside environment such as day and night and the corresponding change in the light source irradiation range to the stereoscopic image, In some cases, the preceding vehicle cannot be accurately identified.

  FIG. 2 is an explanatory diagram for explaining a change in the irradiation range of the light source onto the stereoscopic image due to a change in the environment outside the vehicle. For example, in contrast to a stereoscopic image (the back of the preceding vehicle) as shown in FIG. 2A, the sun located above the preceding vehicle is often the main light source during the day, as shown in FIG. The irradiation range of the light source to the stereoscopic image is limited to the upper portion of the stereoscopic image, and the luminance may be lowered at the lower portion of the stereoscopic image. Even in such a state, if the edge component is simply determined for the entire stereoscopic image, the characteristics of the edge component differ between the top and bottom of the stereoscopic image. In particular, the edge direction at the bottom of the stereoscopic image is not the intended edge direction, and the leading edge There may be a case where the vehicle is erroneously determined as another specific object, for example, a road side wall or a steep slope. Further, during the daytime, a vehicle such as a truck with a cargo compartment located relatively upward has a shadow in the lower part of the back of the vehicle under a backlit condition, and appears darker and has no edge component in the first place. Further, under the condition of normal light, light hits the space below it and the road surface display is included as a three-dimensional image, and the pattern may become noise.

  On the other hand, at night, the headlight of the host vehicle 1 that irradiates from behind the preceding vehicle often serves as the light source, and as shown in FIG. 2 (c), the irradiation range of the light source on the rear surface (rear surface) of the preceding vehicle is large. The luminance is limited to the lower part, and the luminance is lower in the upper part of the stereoscopic image. Even in such a state, if the edge component is simply determined for the entire stereoscopic image, the characteristics of the edge components differ between the top and bottom of the stereoscopic image. In particular, the edge direction at the top of the stereoscopic image is not the intended edge direction, and the leading edge There may be a case where the vehicle is erroneously determined as another specific object, for example, a road side wall or a steep slope.

  Therefore, an object of the present embodiment is to improve the accuracy of specifying a specific object by appropriately determining a stereoscopic image in consideration of changes in the outside environment and irradiation range by determining day and night (brightness and darkness).

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, a procedure for specifying a stereoscopic image characteristic of the present embodiment as one of a preceding vehicle, a side wall of a road, or a steep slope is described in detail, and a configuration unrelated to the features of the present embodiment is described. Is omitted.

(Vehicle environment recognition device 120)
FIG. 3 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 3, the vehicle exterior environment recognition device 120 includes an I / F unit 160, a data holding unit 162, and a central control unit 164.

  The I / F unit 160 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 162 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 162 holds various information necessary for processing of each function unit described below, and temporarily holds a luminance image received from the imaging device 110. To do.

  The central control unit 164 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like, and through the system bus 166, an I / F unit 160, a data holding unit 162 and the like are controlled. In the present embodiment, the central control unit 164 includes the image processing unit 170, the three-dimensional position deriving unit 172, the grouping unit 174, the stereoscopic image tracking unit 176, the brightness determination unit 178, the boundary deriving unit 180, and the edge counting unit 182. Also, it functions as the specific object specifying unit 184.

(External vehicle environment recognition processing)
FIG. 4 is a flowchart showing the flow of the external environment recognition process. In the outside environment recognition processing, the image processing unit 170 processes the image acquired from the imaging device 110 (S200), the three-dimensional position deriving unit 172 derives the three-dimensional position from the image (S202), and the grouping unit 174 Generates a three-dimensional image grouped based on the three-dimensional position (S204). In addition, the stereoscopic image tracking unit 176 tracks the stereoscopic image (S206), the light / dark determination unit 178 determines whether the environment outside the vehicle is bright or dark (S208), and the boundary deriving unit 180 vertically moves the stereoscopic image vertically. The boundary that is divided into two is derived (S210), and the edge counting unit 182 narrows down the region to be determined for the stereoscopic image, and counts the number of vertical edges in the narrowed region (S212). And the specific thing specific | specification part 184 specifies a specific thing according to the result (S214). Here, “vertical” indicates the vertical direction of the screen, and “horizontal” described later indicates the horizontal direction of the screen. Hereinafter, each process is explained in full detail.

(Image processing S200)
The image processing unit 170 acquires a luminance image from each of the two imaging devices 110, and selects a block corresponding to a block arbitrarily extracted from one luminance image (for example, an array of horizontal 4 pixels × vertical 4 pixels) as the other luminance image. The parallax is derived using so-called pattern matching that is searched from the above.

  As this pattern matching, it is conceivable to compare the luminance (Y color difference signal) in units of blocks indicating an arbitrary image position between two luminance images. For example, SAD (Sum of Absolute Difference) that takes a luminance difference, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of a variance value obtained by subtracting an average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 170 performs such block-unit parallax derivation processing for all blocks displayed in the detection area (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set. Hereinafter, a block that is a unit for deriving such parallax information is referred to as a parallax block.

  However, the image processing unit 170 can derive the parallax for each parallax block that is a unit of detection resolution, but cannot recognize what kind of stereoscopic image the parallax block is. Therefore, the parallax information is independently derived not in units of stereoscopic images but in units of detection resolution in the detection region (for example, units of parallax blocks). Here, an image in which the parallax information derived in this way (corresponding to a relative distance z described later) is associated with a luminance image is referred to as a distance image.

  FIG. 5 is an explanatory diagram for explaining the luminance image 212 and the distance image 214. For example, it is assumed that a luminance image 212 as shown in FIG. 5A is generated for the detection region 216 through the two imaging devices 110. However, here, for easy understanding, only one of the two luminance images 212 generated by each of the imaging devices 110 is schematically illustrated. In the present embodiment, the image processing unit 170 obtains the parallax for each parallax block from such a luminance image 212 and generates a distance image 214 as shown in FIG. Each parallax block in the distance image 214 is associated with the parallax of the parallax block. Here, for convenience of explanation, the parallax block from which the parallax is derived is represented by black dots.

(Three-dimensional position derivation process S202)
Returning to FIG. 3, the three-dimensional position deriving unit 172 performs disparity information for each disparity block in the detection region 216 based on the distance image 214 generated by the image processing unit 170 (all the pixels in the disparity block are disparity). A parallax information equal to that of a block) is converted into a three-dimensional position in a real space including a horizontal distance x, a height y, and a relative distance z using a so-called stereo method. Here, the stereo method is a method of deriving the relative distance z of the pixel with respect to the imaging device 110 from the parallax in the distance image 214 of the pixel by using a triangulation method. In addition to such pixels, a block made up of a plurality of pixels can also be used as the image portion that is the minimum unit of such processing. At this time, the three-dimensional position deriving unit 172 determines the pixel road based on the relative distance z of the pixel and the detected distance on the distance image 214 between the point on the road surface and the pixel at the same relative distance z to the pixel. Deriving the height y from the surface. Then, the derived three-dimensional position is associated with the distance image 214 again. Since various known techniques can be applied to the relative distance z deriving process and the three-dimensional position specifying process, the description thereof is omitted here.

(Grouping process S204)
The grouping unit 174 groups pixels having a three-dimensional position difference within a predetermined range to form a stereoscopic image. Specifically, the grouping unit 174 selects pixels in the distance image 214 that are within a predetermined range (for example, 0.1 m) in the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z. , Grouping on the assumption that they correspond to the same specific thing. In this way, a stereoscopic image that is a virtual pixel group is generated. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. The grouping unit 174 also applies to pixels newly added as a result of grouping, with the pixel being the base point, the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z are within a predetermined range. Are further grouped. As a result, all pixels that can be assumed to be the same specific object are grouped.

Here, the horizontal distance x difference, the height y difference, and the relative distance z difference are determined independently, and only when all are included in a predetermined range, the same group is used. You can also. For example, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (difference in relative distance z) ² ) Are included in the predetermined range, the same group may be used. With this calculation, an accurate distance between pixels in real space can be derived, so that the grouping accuracy can be improved.

  FIG. 6 is an explanatory diagram illustrating the grouping process S204. For example, in the luminance image 212 as shown in FIG. 6A, the grouping unit 174 uses the three-dimensional position based on the distance image 214 as shown in FIG. , A difference in height y, and a difference in relative distance z are grouped together in a predetermined range. At this time, the grouping unit 174 sets a rectangular window (hereinafter simply referred to as a rectangular shape) whose outline is composed of a horizontal line and a vertical line, including a stereoscopic image (an aggregate of pixels) and a stereoscopic image. In FIG. 6B, for convenience of explanation, only the relationship between the horizontal distance x and the relative distance z is shown, and the height y is omitted.

  Specifically, the grouping unit 174 groups the pixels of the preceding vehicle 220a shown in FIG. 6A based on the three-dimensional position as shown by a one-dot chain line in FIG. A rectangular window 222a is generated as shown in FIG. Similarly, the grouping unit 174 groups the pixels of the person 220b shown in FIG. 6A based on the three-dimensional positions as shown by the one-dot chain line in FIG. The rectangular window 222b is generated as shown in FIG.

  In the present embodiment, among the plurality of windows 222 positioned in front of the host vehicle 1, in particular, the window 222 including a stereoscopic image of a preceding vehicle that can be a target of follow-up control or collision avoidance control is specified. To do. However, a window that is clearly not a preceding vehicle may be determined as a window that may be a preceding vehicle, or a window that should be determined as a preceding vehicle may be excluded from windows that may be a preceding vehicle. . In the following, we will give specific examples of how this situation occurs and describe how to deal with it.

  First, examples of determining a window that is clearly not a preceding vehicle as a window that may be a preceding vehicle include trees, guardrails, walls, and steep slopes located on the side of the road. For example, it is determined that a part of the window 222 may be a preceding vehicle. When the edge component is determined for these stereoscopic images, the edge direction of the stereoscopic image appears randomly, and the histogram distribution regarding the edge components may be similar to the vehicle distribution. In this case, the stereoscopic image may be specified as a vehicle (preceding vehicle) even though it is not a vehicle.

  Moreover, as an example of excluding the window that should be determined as the preceding vehicle from the window that may be the preceding vehicle, there is a case where the preceding vehicle is imaged under the condition of backlight. In this case, in the luminance image 212, the background of the preceding vehicle (the scenery in front) is close to white and the luminance is high. On the other hand, the preceding vehicle itself is close to black and the luminance is low. Then, the pattern on the back of the preceding vehicle cannot be recognized, the edge component cannot be extracted, and cannot be specified as the preceding vehicle. Further, when the preceding vehicle is a special vehicle such as a freezer car, there are cases where there are few patterns and edges on the back surface. Also in this case, the pattern of the preceding vehicle cannot be recognized, the edge component cannot be extracted, and cannot be specified as the preceding vehicle.

  Therefore, the grouping unit 174 according to the present embodiment derives a stereoscopic image when the change of the relative distance from the stereoscopic image in both the left and right neighboring regions of the window 222 (stereoscopic image) is not continuous. .

  FIG. 7 is an explanatory diagram showing another example of the grouping process S204. The wall (fence) located on the side of the road shown in FIG. 7A is erected in the vertical direction along the road. Therefore, when the road is curved (curved), as shown in FIG. 7A, the wall is positioned in front of the host vehicle 1, so that the grouping unit 174 has a plurality of walls. As shown in FIG. 7B, the portions may be individually grouped and recognized as a window 222 that may be a preceding vehicle.

  Here, referring to FIG. 7 (b), the stereoscopic image in each window 222 in FIG. 7 (a) is originally a continuous wall, so that the relative distance change mode is continuous. That is, it can be understood that the variation of the change amount of the relative distance with respect to the horizontal distance is small. In view of this, the grouping unit 174 determines that the window 222 (a stereoscopic image) has a possibility that the preceding vehicle may be a preceding vehicle 222 when the change of the relative distance of the stereoscopic image in both the left and right neighboring regions of the window 222 (stereoscopic image) continues. Only when the relative distance change mode is not continuous, the window 222 is set as a window 222 that may be a preceding vehicle. In this way, it is possible to exclude the window 222 that is clearly not in the preceding vehicle from the window 222 that may be the preceding vehicle.

  Moreover, in FIG.7 (c), since the preceding vehicle is imaged on the conditions which become backlight, the preceding vehicle itself is near black and the brightness | luminance is low. However, referring to FIG. 7D, the stereoscopic image in each window 222 in FIG. 7C is different from the stereoscopic image of both the left and right neighboring regions in the relative distance and the change mode of the relative distance. That is, the change of the relative distance with respect to the horizontal distance is not continuous. Accordingly, the grouping unit 174 determines that the window 222 (a three-dimensional image) has a possibility of being a preceding vehicle when the change of the relative distance of the three-dimensional image is not continuous in the left and right neighboring regions of the window 222 (three-dimensional image). 222. In this way, it is possible to appropriately set the determination target without excluding the window 222 to be determined as the preceding vehicle.

  FIG. 8 is an explanatory diagram showing another example of the grouping process S204. In the present embodiment, in order to determine the likelihood of the preceding vehicle, as described later, only the back surface of the preceding vehicle is set as the determination target. Therefore, it is desirable that the grouping unit 174 groups only the back surface of the preceding vehicle as a stereoscopic image. In this regard, usually, the preceding vehicle and the host vehicle 1 are often traveling in the same direction, and therefore only the rear surface of the preceding vehicle should be acquired as a three-dimensional image. However, as shown in FIG. 8A, when the road on which the host vehicle 1 is traveling is curved, the preceding vehicle is directed to the left or right, and the traveling direction of the preceding vehicle and the traveling direction of the host vehicle 1 are different. It has a significant angle, and also when the preceding vehicle turns left or right at the intersection, the traveling direction of the preceding vehicle and the traveling direction of the host vehicle 1 have a significant angle. In this case, not only the back surface of the preceding vehicle but also a part of the side surface of the preceding vehicle may be acquired as a stereoscopic image.

  Then, the grouping unit 174 groups the stereoscopic images including a part of the side surface as shown in FIG. 8B, and the possibility of the preceding vehicle with respect to the stereoscopic image as shown in FIG. 8A. A certain window 222 is generated. In this case, when the back surface of the preceding vehicle is determined, the edge components of the side surfaces are mixed as noise, which may hinder the identification of the preceding vehicle.

  Here, referring to FIG. 8B, in the stereoscopic image in the window 222 of FIG. 8A, in the one of the left and right neighboring regions 224 of the stereoscopic image, here, in the left neighboring region 224, Only a part of the vehicle side surface close to the back surface may be grouped together with the back surface. Accordingly, when the traveling direction of the preceding vehicle and the traveling direction of the host vehicle 1 have a significant angle, the relative positions of the regions corresponding to the left and right neighboring regions 224 of the stereoscopic image, here the left neighboring region 224, are compared. If the distance is inclined as indicated by a two-dot chain line in FIG. 8B, there is a high possibility that the side surface of the preceding vehicle is acquired in the right direction (the left part of the preceding vehicle) of the vicinity region 224. It will be.

  Therefore, if the relative distance is inclined in one of the left and right neighboring areas 224 of the stereoscopic image, the grouping unit 174 continuously displays the relative distance from the neighboring area 224 in the stereoscopic image as shown in FIG. Is excluded from the window 222 that may be a preceding vehicle. Here, the region 226 in which the relative distance continuously inclines is indicated by a solid line and an inclination of the relative distance of the left neighboring region 224 indicated by a two-dot chain line in FIG. 8B from the left end of the window 222. And the intersection with the slope of the back of the stereoscopic image.

  Further, when the width of the vehicle is roughly determined to be, for example, 1.5 m, for example, the horizontal distance from the end of the window 222 opposite to the region where the relative distance is inclined (the right end of the window 222 in this case). Up to 1.5 m, and the left area may be fixedly excluded from that part. For example, in the example of FIG. 8A, the left region is excluded from a position 1.5 m from the right end. Thus, as shown in FIG. 8C, the horizontal width of the window 222 is reduced, and only the back surface of the preceding vehicle can be acquired as a stereoscopic image.

(Stereoscopic image tracking process S206)
The stereoscopic image tracking unit 176 tracks the stereoscopic images (window 222) grouped by the grouping unit 174. Since various existing techniques can be applied to the procedure for tracking such a stereoscopic image, detailed description thereof is omitted here.

  By the way, in the environment recognition system 100, it is set as one of the objectives to determine whether the three-dimensional image located in the detection area ahead of the own vehicle 1 is a preceding vehicle. However, since the preceding vehicle is determined using various methods such as comparison with a template, Hough conversion, and acquisition of color information, the processing load becomes heavy. In particular, since the number of stereoscopic images located in the detection area in front of the host vehicle 1 is not limited, if determination is performed for all the stereoscopic images, the processing load may increase. Therefore, in the present embodiment, the stereoscopic image to be determined is limited based on the traveling path corresponding to the traveling direction of the host vehicle 1 and the stereoscopic image tracked by the stereoscopic image tracking unit 176.

  FIG. 9 is an explanatory diagram for explaining the limitation of a stereoscopic image. For example, as illustrated in FIG. 9, when the stereoscopic image 232 a is located on the traveling path 230 indicated by a one-dot chain line in FIG. 9 corresponding to the traveling direction of the host vehicle 1, the stereoscopic image 232 a is determined as a determination target. To do. When the stereoscopic image 232a does not exist (no stereoscopic image is located on the traveling path 230), the stereoscopic image having the shortest horizontal distance from the traveling path 230, for example, the stereoscopic image 232b is determined.

  Further, instead of or in addition to such a condition, as shown in hatching in FIG. 9, the three-dimensional image 232a tracked by the three-dimensional image tracking unit 176 has a difference in the horizontal distance from the traveling path 230 within a predetermined range. The three-dimensional image 232a can be determined as being determined (contained). In this way, the boundary deriving unit 180 described later determines a stereoscopic image having the shortest horizontal distance difference from the traveling path 230 or a stereoscopic image in which the horizontal distance difference from the traveling path 230 is maintained within a predetermined range. A predetermined process is executed as a target. With this configuration, it is possible to reduce the processing load even when a plurality of stereoscopic images are located in the detection area in front of the host vehicle 1.

(Brightness determination process S208)
The light / dark determination unit 178 determines whether the light state or the dark state according to the luminance of the luminance image 212. Such a bright state mainly indicates a daytime when the brightness of the outside environment is high, and a dark state mainly indicates a nighttime when the brightness of the outside environment is low, but this is not the case, for example, it is dark when traveling in a tunnel even in the daytime. It is good also as a state.

  FIG. 10 is an explanatory diagram for explaining the processing of the light / dark determination unit 178. First, the brightness determination unit 178 extracts the luminance of a predetermined number of pixels included in a predetermined block at three target points 240 corresponding to a predetermined position in the luminance image 212, and calculates the average luminance for each target point 240. To derive. Further, the light / dark determination unit 178 further averages the derived three average values, and derives an average value (overall average value) of the luminances of all the three target points 240. Then, the light / dark determination unit 178 compares the overall average value with a predetermined threshold value, determines that the overall average value is equal to or greater than the threshold value, determines the bright state, and if the overall average value is less than the threshold value, judge.

  In the above, an example in which a bright state or a dark state is determined based on the luminance image 212 has been described. However, the present invention is not limited to this, and the existing opening / closing time and gain of the imaging device 110 are compared with respective threshold values. Whether the light state or the dark state can be determined by various procedures. In the above description, an example in which a bright state or a dark state is determined by one-time comparison between the overall average value and the threshold value is described. However, the present invention is not limited to this. Points may be added to or subtracted from the state and the dark state, respectively, and the light state or the dark state may be determined according to the total number.

(Boundary derivation process S210)
The boundary deriving unit 180 derives a boundary that divides the window 222 in the vertical direction based on the vertical luminance distribution of the window 222 including the stereoscopic image generated by the grouping unit 174. Hereinafter, a specific operation of the boundary deriving unit 180 will be described.

  FIG. 11 is an explanatory diagram for explaining the operation of the boundary deriving unit 180. As described with reference to FIG. 2, in the bright state, as shown on the left side of FIG. 11A, the illumination range of the light source to the stereoscopic image is limited to the upper part of the stereoscopic image, and the luminance is below the stereoscopic image. Becomes lower. On the other hand, in the dark state, as shown on the left side of FIG. 11B, the light source irradiation range to the back surface (rear surface) of the preceding vehicle is limited to the lower part, and the luminance is lower in the upper part of the stereoscopic image. Therefore, the boundary deriving unit 180 derives the boundary (boundary) of the change in luminance.

  Specifically, the boundary deriving unit 180 first divides the window 222 in the vertical direction and generates a plurality (here, 10) of first divided regions 250 extending in the horizontal direction. Subsequently, the boundary deriving unit 180, as shown on the right in FIG. 11 (a) and FIG. 11 (b), the luminance in all pixels in the first divided region 250 for each of the plurality of divided first divided regions 250. The average value of is derived. Here, in order to facilitate understanding, an example of dividing into 10 in the vertical direction is given, but the number of divisions is not limited to this, and can be arbitrarily determined according to the required resolution.

  Next, based on the result determined by the light / dark determination unit 178, the boundary deriving unit 180 searches from above in the vertical direction if it is bright, derives a change point where the luminance decreases by a predetermined threshold or more, and if it is dark, Similarly, a search is performed from vertically above, and a change point at which the luminance increases by a predetermined threshold value or more is derived. The change point becomes a boundary 252 in the stereoscopic image extending in the horizontal direction. Referring to FIG. 11, it can be understood that the irradiation mode is inverted above and below the boundary 252, and the boundary becomes the boundary 252.

(Edge Counting Process S212)
Subsequently, the edge counting unit 182 and the specific object specifying unit 184 use a higher brightness image in the vertical direction of the boundary 252 derived by the boundary deriving unit 180, and a stereoscopic image is predetermined based on the edge component. It is determined whether it is a specific item. Specifically, first, the edge counting unit 182 specifies the appearance mode of the edge component on the back surface of the preceding vehicle based on the boundary 252 derived by the boundary deriving unit 180.

  FIG. 12 is an explanatory diagram for explaining the operation of the edge counting unit 182. First, the edge counting unit 182 narrows down the window 222 based on the boundary 252 derived by the boundary deriving unit 180. Specifically, as shown in FIG. 12A, the edge counting unit 182 excludes the lower side of the boundary 252 of the window 222 (stereoscopic image) and leaves the upper side as shown in FIG. As shown in the figure, if it is a dark state, the upper part of the boundary 252 of the window 222 is excluded and the lower part is left, and this is used as an edge component determination target.

  Next, as shown in FIGS. 12A and 12B, the edge counting unit 182 divides the portion left in this way in the horizontal direction and extends a plurality of (here 3). ) Second divided region 260 is generated. Then, the edge counting unit 182 derives an edge direction for each pixel included in each of the plurality of second divided regions 260. Here, the edge direction indicates the extending direction of the edge.

  FIG. 13 is an explanatory diagram for explaining the edge direction. As shown in FIG. 13A, when an arbitrary block 262 composed of four pixels in the second divided region 260 is enlarged, the luminance distribution as shown in FIG. 13B is obtained. Further, the luminance range is 0 to 255, and in FIG. 13B, it is assumed that the white fill is luminance “200” and the black fill is luminance “0”. Here, it is assumed that the luminance of the upper left pixel in the block 262 is A, the luminance of the upper right pixel is B, the luminance of the lower left pixel is C, the luminance of the lower right pixel is D, and the horizontal component in the edge direction is (A + B). -(C + D), and the vertical component of the edge direction is defined as (B + D)-(A + C).

  Then, the horizontal component in the edge direction of the block 262 shown in FIG. 13B is (A + B) − (C + D) = (200 + 0) − (200 + 0) = 0, and the vertical component in the edge direction is (B + D). − (A + C) = (0 + 0) − (200 + 200) = − 400. Therefore, the horizontal direction component is “0” and the vertical direction component is “−400”, and the edge direction is indicated by a vertically downward (negative) arrow as shown in FIG. However, as shown in FIG. 13D, the horizontal component is positive in the screen right direction, and the vertical component is positive in the screen upward direction. The edge direction thus derived is associated with all four pixels in the block. That is, all four pixels in the block have the same edge direction.

  As described above, the configuration in which the other half area is subtracted from the half area in the block can remove the luminance offset and noise included in the entire block, and can appropriately extract the edge. . In addition, since the edge direction can be derived by simple calculation with only addition and subtraction, the calculation load can be reduced.

  The purpose of this embodiment is to accumulate the edge directions derived in this way and derive the ratio. However, if the values derived from the horizontal direction component and the vertical direction component are used as they are and the edge direction is simply used, there are infinite variations in the edge direction. Then, the edge direction range that can be regarded as the same for the infinite variation must be set.

  Therefore, in this embodiment, both the horizontal direction component and the vertical direction component are defined by unit length, and the variation in the edge direction is simplified. That is, both the horizontal direction component and the vertical direction component are assumed to be either -1, 0, or +1. Then, the edge directions can be limited to eight directions each having an angle of 45 degrees as shown in FIG. For example, in the example of FIG. 13B described above, the edge direction is the direction of “7” in FIG. In this way, the calculation load on the edge counting unit 182 can be greatly reduced. However, the edge direction deriving unit is not limited to such a case, and various existing deriving units that can determine the appearance mode of the edge can be applied.

  The edge counting unit 182 derives such edge directions for blocks corresponding to all the pixels in each of the plurality of second divided areas 260, and accumulates the number of edge directions for each second divided area 260. . At this time, the edge counting unit 182 counts only the number of vertical edges in the second divided region 260. The vertical edge refers to an edge in a vertical direction and an edge in a diagonal direction within a predetermined angle range from the vertical direction. For example, in the example of FIG. 13E, in the edge directions “3” and “7” which are vertical edges, and the edge directions “2”, “4”, “6” and “8” which are diagonal directions. is there. Therefore, here, the angle range of the vertical edge is 22.5 ° to 157.5 ° and −22.5 ° to −157.5 ° when the horizontal left direction is zero. However, the angle range of the vertical edge is not limited to this, and can be arbitrarily determined.

  In addition, here, an example in which the edge direction is derived with respect to the blocks corresponding to all the pixels in the second divided region 260 has been described. However, the present invention is not limited to this, and the block from which the edge direction is derived is described. You may restrict | limit to a predetermined number. For example, 50 pixels are extracted at a predetermined interval in the horizontal direction of the second divided region 260 and 50 pixels are extracted at a predetermined interval in the vertical direction, and the edge direction is derived for the block corresponding to the pixel. Vertical edges may be counted for pixels × 50 pixels.

  In this way, the number of vertical edges for each second divided area 260 is counted on the vehicle, the side wall of the road, and the steep slope, and the number of vertical edges for each second divided area 260 differs, and the vehicle, This is because the side walls of the road and steep slopes can be identified.

  FIG. 14 is an explanatory diagram for explaining an appearance mode of a vertical edge. In FIG. 14, the edge direction is shown, but the resolution is shown low for easy understanding. As shown in FIG. 14, since the vehicle includes many vertical edges at the left and right ends of the outer shape, the number of vertical edges in the second divided region 260 located in the center of the second divided regions 260 is small, and left and right The number of vertical edges in the second divided region 260 located in the region increases. Further, since the edge of the road is randomly generated and includes many vertical edges as a whole, the vertical edges of the second divided regions 260 are relatively uniformly increased. Furthermore, since the steep slope generally has a large amount of edge components in the horizontal direction, the vertical edges of the second divided regions 260 are relatively uniformly reduced.

(Specific object specifying process S214)
The specific object specifying unit 184 determines whether or not the stereoscopic image is a predetermined specific object based on the number of vertical edges in the second divided area located at the center and the number of vertical edges in the second divided area positioned on the left and right. judge.

  Specifically, the specific object specifying unit 184 counts the vertical edges for each of the plurality of second divided regions 260, compares the number of each vertical edge with a predetermined threshold value, and in the window 222 according to the result. The three-dimensional image is specified as follows. For example, the specific object specifying unit 184 has the number of vertical edges in the second divided region 260 located at the center being less than a predetermined threshold and the number of vertical edges in the second divided region 260 located on the left and right. If it is equal to or greater than a predetermined threshold, it is temporarily determined that the vehicle is a vehicle. Further, the specific object specifying unit 184 has the number of vertical edges in the second divided region 260 positioned at the center equal to or greater than a predetermined threshold and the number of vertical edges in the second divided region 260 positioned on the left and right. If it is equal to or greater than a predetermined threshold value, it is temporarily determined that the road side wall. The specific object specifying unit 184 has a number of vertical edges in the second divided region 260 located at the center being less than a predetermined threshold value, and a number of vertical edges in the second divided region 260 located on the left and right is a predetermined value. If it is less than the threshold value, it is temporarily determined that the slope is steep.

  Here, the predetermined threshold is a predetermined value obtained statistically from a plurality of trials. Further, the predetermined threshold is changed according to the relative distance of the preceding vehicle, and is smaller when the relative distance is long and larger when the relative distance is short. In addition, according to the absolute size of the stereoscopic image in the window 222, the size is changed if the stereoscopic image such as a track is large, and smaller if the stereoscopic image is small.

  Subsequently, the specific object specifying unit 184 adds points to the window 222. Specifically, if the specific object specifying unit 184 tentatively determines that the vehicle is a vehicle, the vehicle point is added by 1 for example, and the other points are subtracted or reset by 1 for example. Also, if the specific object specifying unit 184 temporarily determines that the road is a side wall of the road, the specific object specifying unit 184 adds 1 to the point on the side wall of the road and subtracts or resets the other point by 1. When the specific object specifying unit 184 temporarily determines that the slope is a steep slope, the specific object specifying unit 184 adds 1 to the steep slope and subtracts or resets another point. Then, when any point of the vehicle, the side wall of the road, or the steep slope becomes equal to or higher than a predetermined point as time elapses, the window 222 displays the three-dimensional image that is equal to or higher than the point, for example, Identify the vehicle. Such point counting may be weighted for each vehicle, road side wall, or steep slope.

  Thus, in the vehicle environment recognition apparatus 120 according to the present embodiment, the range to be determined is appropriately determined regardless of the environment outside the vehicle and the irradiation range, and the specific object is determined based on the distribution of the edge components therein. Thus, it is possible to improve the identification accuracy of the specific object. In addition, by appropriately determining the specific object, it is possible to appropriately determine the control target of the collision avoidance control and the cruise control, and it is possible to improve the running stability of the vehicle.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the window 222 including a stereoscopic image is set as the determination target. However, the present invention is not limited to this, and the stereoscopic image itself can be determined as the determination target.

  It should be noted that each step of the vehicle environment recognition processing in the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an outside vehicle environment recognition device that identifies a three-dimensional image existing around a host vehicle.

DESCRIPTION OF SYMBOLS 1 Own vehicle 100 Environment recognition system 110 Imaging device 120 Outside vehicle environment recognition apparatus 170 Image processing part 172 Three-dimensional position deriving part 174 Grouping part 176 Three-dimensional image tracking part 178 Brightness determination part 180 Boundary deriving part 182 Edge counting part 184 Specific object specification Part 250 first divided area 260 second divided area

Claims (4)

A boundary deriving unit for deriving a boundary for vertically dividing the stereoscopic image based on a vertical luminance distribution of the stereoscopic image in an image captured by the imaging device;
A specific object specifying unit that determines whether the three-dimensional image is a predetermined specific object based on an edge component using an image having a higher luminance in the vertical direction of the boundary;
A vehicle exterior environment recognition device comprising:   The boundary deriving unit generates a plurality of first divided regions extending in the horizontal direction by dividing the stereoscopic image in the vertical direction, and deriving the boundary based on a luminance difference between the plurality of first divided regions. The external environment recognition device according to claim 1, wherein: A light / dark determination unit for determining whether the image is in a light state or a dark state;
If the light is bright, the upper part of the stereoscopic image is left above, and if it is dark, the lower part of the stereoscopic image is left below. Of the plurality of second divided areas, the number of vertical edges extending in the vertical direction in the second divided area located in the center, and the second divided area located on the left and right An edge counter for counting the number of vertical edges of
Further comprising
The specific object specifying unit is configured to specify the stereoscopic image based on the number of vertical edges in the second divided region located at the center and the number of vertical edges in the second divided region located on the left and right. It is determined whether it is a thing, The exterior environment recognition apparatus of Claim 1 or 2 characterized by the above-mentioned.   The specific object specifying unit is configured such that the number of vertical edges in the second divided region located at the center is less than a predetermined threshold, and the vertical edges in the second divided regions located on the left and right are equal to or larger than a predetermined threshold. If it is, it will determine with the said three-dimensional image being a vehicle, The exterior environment recognition apparatus of Claim 3 characterized by the above-mentioned.

Images