JP2013109457A - Device and method for recognizing vehicle exterior environment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve identification accuracy of an object.SOLUTION: A vehicle exterior environment recognizing device 130 retains a plurality of color identifiers and luminance ranges associated with each other, acquires an image obtained by capturing vehicle exterior environment, divides a detection area of the image into a plurality of specific areas according to the positional relations relative to a self vehicle (S306), resets a luminance range associated with the color identifier for every specific area (S308), sets the color identifier to an object site on the basis of the luminance of a plurality of object sites in the detection area, and the luminance range reset for a specific area where the object site is located (310), and groups one or a plurality of object sites in which a difference in a horizontal distance and a difference in height are present in the predetermined range (S312).

Description

  The present invention relates to a vehicle environment recognition apparatus and a vehicle environment recognition method for recognizing an environment outside a host vehicle.

  Conventionally, an object such as a vehicle or an obstacle located in front of the host vehicle is detected, and a collision with the detected object is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is kept at a safe distance. Such a control (cruise control) technique is known (for example, Patent Document 1).

  Many of these technologies are configured to track the preceding vehicle, update the position information, and calculate the moving speed of the preceding vehicle, and the result is used, for example, for collision avoidance control or cruise control. In addition, in order to smoothly perform such control, the vehicle environment other than the preceding vehicle, for example, an object such as a traffic light or a road sign, is recognized to support the traveling of the host vehicle, or the brake lamp of the preceding vehicle is recognized. In preparation for sudden deceleration of the preceding vehicle.

  The above-described object is specified based on an image obtained by capturing an environment outside the vehicle ahead of the host vehicle. As a technology for recognizing such an environment outside the vehicle, for example, a color image outside the vehicle is captured, adjacent pixels having the same luminance (color) are grouped, and a light source such as a red light of a traffic light is recognized as an object. The technique to do is known (for example, patent document 2).

Japanese Patent No. 3349060 JP 2010-224924 A

  In order to smoothly perform the collision avoidance control and the cruise control described above, it is necessary to grasp the behavior of the preceding vehicle in more detail through recognition of, for example, a brake lamp and a blinker. It is also necessary to acquire traffic information necessary for driving the vehicle more accurately through the environment outside the vehicle such as red, yellow, and blue lights.

  For example, since the luminance of the brake lamp and the red light of the traffic light are similar as red light sources, they are extracted using the same reference (luminance range). This is because if the detection target is excessively subdivided, the subsequent calculation load increases and the reaction time is delayed, which may adversely affect smooth collision avoidance control and cruise control.

  However, if both the brake lamp and the red light of the traffic light are extracted based on the same standard, for example, the luminance range as the judgment standard must be widened. There is a possibility that an object that should not be erroneously determined. If the luminance range is narrowed to avoid such an erroneous determination, an object that cannot be extracted among a plurality of types of objects to be extracted is generated.

  In view of such a problem, the present invention provides a vehicle exterior environment recognition apparatus and vehicle exterior environment recognition capable of accurately identifying a plurality of types of objects whose brightness approximates while avoiding wasteful subdivision of detection targets. It aims to provide a method.

  In order to solve the above problem, an external environment recognition apparatus of the present invention includes a data storage unit that stores a plurality of color identifiers and luminance ranges in association with each other, an image acquisition unit that acquires an image of the external environment, A specific area dividing unit that divides the detection area of the image into a plurality of specific areas according to a relative positional relationship with the host vehicle, and a luminance range reset that resets the luminance range associated with the color identifier for each specific area A color identifier setting unit for setting a color identifier for the target part based on the setting part, the luminance of the plurality of target parts in the detection region, and the luminance range reset to the specific region where the target part is located, and the horizontal distance And a grouping unit for grouping one or a plurality of target portions whose height difference and height difference are within a predetermined range.

  The luminance range resetting unit may correct the luminance range of the color identifier according to the imaging state or the environment outside the vehicle.

  The specific area includes a vehicle, a road surface, a road sign, a pedestrian, a moving solid object, a predetermined height range, a predetermined relative distance range, a predetermined horizontal distance range, a horizontal direction of an image, or It may be an area occupied by one or more specific objects selected from a group of division ranges divided in the vertical direction.

  In order to solve the above-described problem, the vehicle exterior environment recognition method according to the present invention stores a plurality of color identifiers and luminance ranges in association with each other, acquires an image of the vehicle exterior environment, and is relative to the host vehicle. The detection area of the image is divided into a plurality of specific areas according to a specific positional relationship, the luminance range associated with the color identifier is reset for each specific area, and the luminance and target areas of the plurality of target parts in the detection area are reset. Based on the luminance range reset to the specific area where the is located, set a color identifier for the target part, and group one or more target parts whose horizontal distance difference and height difference are within the predetermined range It is characterized by doing.

  According to the present invention, it is possible to accurately identify a plurality of types of objects with similar brightness while avoiding unnecessary increase in processing load by avoiding unnecessary subdivision of detection targets.

It is the block diagram which showed the connection relation of the environment recognition system. It is explanatory drawing for demonstrating a luminance image and a distance image. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a color identifier table. It is explanatory drawing for demonstrating the conversion to the three-dimensional positional information by a positional information acquisition part. It is explanatory drawing for demonstrating a division area and a representative distance. It is explanatory drawing for demonstrating a division area group. It is explanatory drawing for demonstrating correction | amendment of a luminance range. It is explanatory drawing for demonstrating a color identifier map. It is the flowchart which showed the rough flow of the process of the environment recognition method outside a vehicle. It is the flowchart which showed the flow of the positional information acquisition process. It is the flowchart which showed the flow of the representative distance derivation processing. It is the flowchart which showed the flow of the division area group production | generation process. It is the flowchart which showed the flow of the specific area division process. It is the flowchart which showed the flow of the brightness | luminance range reset process. It is the flowchart which showed the flow of the color identifier setting process. It is the flowchart which showed the flow of the grouping process. It is the flowchart which showed the flow of the specific object determination process. It is explanatory drawing for demonstrating an effect.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes a plurality of (two in the present embodiment) imaging devices 110, an image processing device 120, an outside vehicle environment recognition device 130, and a vehicle control device 140 provided in the host vehicle 1. Consists of.

  The imaging device 110 includes an imaging device such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and is a color image, that is, three hues (R (red), G (pixel-by-pixel)). The brightness of green) and B (blue) can be acquired. In the present embodiment, color and luminance are treated equally, and when both words are included in the same sentence, each other can be read as luminance constituting the color or a color having luminance. Here, a color image picked up by the image pickup apparatus 110 is called a luminance image, and is distinguished from a distance image described later.

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates, for example, every 1/60 seconds (60 fps), image data obtained by imaging an object existing in the detection area in front of the host vehicle 1. Each functional unit in the following embodiment performs each process triggered by such update of image data. Here, the object is not only a three-dimensional object that exists independently such as a vehicle, a traffic light, a road, a guardrail, but also an object that can be specified as a part of a three-dimensional object such as a brake lamp (tail lamp), a blinker, or a lighting part of a traffic light. Including.

  The image processing device 120 acquires image data from each of the two imaging devices 110, and based on the two image data, the parallax of an arbitrary block (collected a predetermined number of pixels) in the image, and an arbitrary Disparity information including a screen position indicating a position of the block in the screen is derived. The image processing apparatus 120 derives the parallax using so-called pattern matching in which a block corresponding to a block arbitrarily extracted from one image data (for example, an array of 4 horizontal pixels × 4 vertical pixels) is searched from the other image data. . Here, “horizontal” used in the description of the block indicates the horizontal direction of the captured image and corresponds to the horizontal direction in real space. “Vertical” indicates the vertical direction of the captured image and corresponds to the vertical direction in real space.

  As this pattern matching, it is conceivable to compare luminance values (Y color difference signals) in units of blocks indicating an arbitrary image position between two pieces of image data. For example, the SAD (Sum of Absolute Difference) that takes the difference in luminance value, the SSD (Sum of Squared intensity Difference) that uses the difference squared, and the similarity of the variance value obtained by subtracting the average value from the luminance value of each pixel. There are methods such as NCC (Normalized Cross Correlation). The image processing apparatus 120 performs such a block-based parallax derivation process for all blocks displayed in the detection area (for example, 600 pixels × 200 pixels). Here, the block is 4 pixels × 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the image processing apparatus 120 can derive the parallax for each block, which is a unit of detection resolution, but cannot recognize what kind of target object the block is. Accordingly, the disparity information is derived independently not in units of objects but in units of detection resolution (for example, blocks) in the detection region. Here, an image in which the parallax information derived in this way (corresponding to a relative distance described later) is associated with image data is referred to as a distance image.

  FIG. 2 is an explanatory diagram for explaining the luminance image 124 and the distance image 126. For example, it is assumed that a luminance image (image data) 124 as illustrated in FIG. 2A is generated for the detection region 122 through the two imaging devices 110. However, only one of the two luminance images 124 is schematically shown here for easy understanding. The image processing apparatus 120 obtains the parallax for each block from the luminance image 124 and forms a distance image 126 as shown in FIG. Each block in the distance image 126 is associated with the parallax of the block. Here, for convenience of description, blocks from which parallax is derived are represented by black dots.

  Since the parallax is easily specified at the edge portion of the image (the portion where the difference in brightness between adjacent pixels is large), the block to which the black dot is attached in the distance image 126 and from which the parallax is derived is the luminance image 124. Are often edges. Accordingly, the luminance image 124 shown in FIG. 2A and the distance image 126 shown in FIG. 2B are similar in outline of each object.

  The vehicle exterior environment recognition device 130 acquires the luminance image 124 and the distance image 126 from the image processing device 120, and uses the luminance based on the luminance image 124 and the relative distance between the host vehicle 1 based on the distance image 126 and the detection region 122. It is specified which specific object the object in corresponds to. For example, by specifying the preceding vehicle by the relative distance or the like, and by specifying the brake lamp of the preceding vehicle by the brightness, it is possible to more accurately grasp the vehicle in which the brake lamp is lit. By doing so, it is possible to quickly grasp the deceleration of the vehicle by the brake lamp and use it for collision avoidance control and cruise control.

  The relative distance is obtained by converting the disparity information for each block in the distance image 126 into three-dimensional position information using a so-called stereo method. Here, the stereo method is a method of deriving a relative distance of the target object from the imaging device 110 from the parallax of the target object by using a triangulation method. The processing of the outside environment recognition device 130 will be described in detail later.

  The vehicle control device 140 executes control for avoiding a collision with the object specified by the outside environment recognition device 130 and for keeping the distance between the vehicle and the preceding vehicle at a safe distance. Specifically, the vehicle control device 140 acquires the current traveling state of the host vehicle 1 through the steering angle sensor 142 that detects the steering angle, the vehicle speed sensor 144 that detects the speed of the host vehicle 1, and the like, and controls the actuator 146. The distance between the vehicle and the preceding vehicle is kept safe. Here, the actuator 146 is an actuator for vehicle control used for controlling a brake, a throttle valve, a steering angle, and the like. In addition, when a collision with an object is assumed, the vehicle control device 140 displays a warning (notification) on the display 148 installed in front of the driver and controls the actuator 146 to control the host vehicle 1. Brake automatically. Such a vehicle control device 140 can also be formed integrally with the outside environment recognition device 130.

  In the present embodiment, a specific area (a part of the detection area according to the specific object) occupied by the vehicle is specified through the environment recognition system 100, and the specific object “brake” is used by using the specific area. An example will be given in which a “lamp” is distinguished from a specific item “red light of a traffic light”. However, the “brake lamp” in the present embodiment intends the lighting state of the brake lamp. Needless to say, the specific area or specific object is not limited to a vehicle, a brake lamp, or a red light of a traffic light.

(Exterior environment recognition device 130)
FIG. 3 is a functional block diagram showing a schematic function of the outside environment recognition device 130. As shown in FIG. 3, the vehicle exterior environment recognition device 130 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the image processing device 120 and the vehicle control device 140. The I / F unit 150 functions as an image acquisition unit. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 152 holds a color identifier table (association) and various information necessary for processing of each functional unit described below. The luminance image 124 and the distance image 126 received from are temporarily stored. Here, the color identifier table is used as follows.

  FIG. 4 is an explanatory diagram for explaining the color identifier table 200. In the color identifier table 200, a luminance range 202 representing a predetermined number of colors that are determined in advance is associated with the color identifier 204. For example, the color identifier “1” includes a luminance range corresponding to a red specific object, for example, a luminance range (R) “80 or more”, a luminance range (G) “40 or less”, and a luminance range (B) “40 or less. "Is associated. Similarly, for the color identifiers “2”, “3”, “4”, “5”, “6”, “7”, “8”, yellow, blue-green, magenta, orange, vermilion, blue, green A luminance range corresponding to the specific object is associated. However, it goes without saying that the luminance range is not limited to the luminance range shown in FIG. 4, and the number thereof is not limited.

  Although not shown, each color identifier in the color identifier table 200 is associated with one or more specific objects. For example, the color identifier “1” is associated with a red specific object “brake lamp” and a specific object “red light of a traffic light”. This is for avoiding an increase in calculation load by extracting specific objects having similar brightness with the same reference (brightness range).

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program and the like, a RAM as a work area, and the like, and through the system bus 156, the I / F unit 150 and the data holding unit. 152 is controlled. In the present embodiment, the central control unit 154 includes the position information acquisition unit 160, the representative distance derivation unit 162, the divided region group generation unit 164, the specific region division unit 166, the luminance range resetting unit 168, and the color identifier setting unit 170. , And also functions as a grouping unit 172 and a specific object determining unit 174.

  The position information acquisition unit 160 uses the stereo method described above for the disparity information for each block in the detection area 122 in the distance image 126 acquired via the I / F unit 150, and the horizontal distance x, height y, and relative Conversion into three-dimensional position information including the distance z. Here, the parallax information indicates the parallax of each block in the distance image 126, while the three-dimensional position information indicates information on the relative distance of each block in the real space. Further, when the disparity information is derived not in pixel units but in block units, that is, in a plurality of pixel units, the disparity information may be regarded as disparity information of all pixels belonging to the block, and calculation in pixel units may be executed. it can.

  FIG. 5 is an explanatory diagram for explaining the conversion into the three-dimensional position information by the position information acquisition unit 160. First, the position information acquisition unit 160 recognizes the distance image 126 as a pixel unit coordinate system as shown in FIG. In FIG. 5, the lower left corner is the origin (0, 0), the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. Therefore, a pixel having the parallax dp can be expressed as (i, j, dp) by the pixel position i, j and the parallax dp.

The three-dimensional coordinate system in real space in the present embodiment is considered as a relative coordinate system with the host vehicle 1 as the center. Here, the right side of the traveling direction of the host vehicle 1 is the positive direction of the X axis, the upper direction of the host vehicle 1 is the positive direction of the Y axis, and the traveling direction (front) of the host vehicle 1 is the positive direction of the Z axis. The intersection of the vertical line passing through the center of 110 and the road surface is defined as the origin (0, 0, 0). At this time, assuming that the road is a plane, the road surface coincides with the XZ plane (y = 0). The position information acquisition unit 160 converts the block (i, j, dp) on the distance image 126 into a three-dimensional point (x, y, z) in the real space by the following (Formula 1) to (Formula 3). Convert coordinates.
x = CD / 2 + z · PW · (i-IV) (Equation 1)
y = CH + z · PW · (j−JV) (Formula 2)
z = KS / dp (Formula 3)
Here, CD is the interval (baseline length) between the imaging devices 110, PW is the viewing angle per pixel, CH is the height of the imaging device 110 from the road surface, and IV and JV are The coordinates (pixels) on the image of the point at infinity in front of the vehicle 1 and KS is a distance coefficient (KS = CD / PW).

  Therefore, the position information acquisition unit 160 determines the road surface based on the relative distance between the blocks and the detected distance (for example, the number of pixels) on the distance image 126 between the points on the road surface and the blocks at the same relative distance as the blocks. The height from is derived.

  First, the representative distance deriving unit 162 divides the detection area 122 of the distance image 126 into a plurality of divided areas 216 in the horizontal direction. Subsequently, the representative distance deriving unit 162 adds up the relative distances included in each of the predetermined distances divided into a plurality of divided regions for each divided region based on the position information for the block located above the road surface. Generate. Then, the representative distance deriving unit 162 derives a representative distance corresponding to the peak of the accumulated distance distribution. Here, “corresponding to a peak” means a peak value or a value that satisfies an arbitrary condition in the vicinity of the peak.

  FIG. 6 is an explanatory diagram for explaining the divided region 216 and the representative distance 220. As shown in FIG. 5, when the distance image 126 is divided into a plurality of parts in the horizontal direction, the divided region 216 has a strip shape as shown in FIG. Such a strip-shaped divided region 216 is originally composed of, for example, 150 columns each having a horizontal width of 4 pixels, but here, for convenience of explanation, the detection region 122 is divided into 16 parts.

  Subsequently, the representative distance deriving unit 162 refers to the relative distances of all the blocks in each divided region 216 and creates a histogram (indicated by a horizontally long square (bar) in FIG. 6B). A distance distribution 218 as shown in (b) is obtained. Here, the vertical direction indicates the divided predetermined distance, and the horizontal direction indicates the number of blocks in which the relative distance is included in each of the divided predetermined distances. However, FIG. 6B is a virtual screen for performing the calculation, and actually does not involve generation of a visual screen. Then, the representative distance deriving unit 162 refers to the distance distribution 218 derived in this way, and represents a representative distance (indicated by a square filled in black in FIG. 6B) that is a relative distance corresponding to a peak. Is identified.

  The divided region group generation unit 164 sequentially compares the representative distances 220 between the adjacent divided regions 216, and groups the divided regions 216 that are close to each other (for example, located at 1 m or less) into one or more divided regions. A region group is generated. At this time, even when the representative distance 220 is close in three or more divided areas 216, all the continuous divided areas 216 are collected as a divided area group.

  FIG. 7 is an explanatory diagram for explaining the divided region group 222. The divided area group generation unit 164 compares the divided areas 216 and groups the representative distances 220 as shown in FIG. 7 (virtual group 224 after grouping). By such grouping, the divided region group generation unit 164 can specify a three-dimensional object located above the road surface. Further, the divided region group generation unit 164 follows a road such as a rear portion, a side portion, or a guardrail of the preceding vehicle based on the transition of the relative distance in the horizontal direction and the vertical direction in the grouped divided region group 222. It is possible to recognize which one is a structure or the like.

  The specific area dividing unit 166 specifies a specific object such as a preceding vehicle according to a relative positional relationship with the host vehicle 1, and divides the detection area of the image into one or more specific areas according to the outer edge of the specific object. To do. The specific region can be represented by various predetermined shapes that accommodate the outer edge, but here it is represented by a rectangular frame. Specifically, first, the specific area dividing unit 166 uses the block whose relative distance z corresponds to the representative distance 220 in the divided area group 222 generated by the divided area group generating unit 164 as a base point, and the horizontal distance from the block. Blocks in which a difference in x, a difference in height y, and a difference in relative distance z are within a predetermined range (for example, 0.1 m) are grouped on the assumption that they correspond to the same specific object. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. In addition, the specific area dividing unit 166 also has a difference in the horizontal distance x, a difference in the height y, and a difference in the relative distance z within a predetermined range with respect to a block newly added by grouping, with the block as a base point. Group the blocks further. As a result, all blocks that can be assumed to be the same specific object are grouped.

Here, the horizontal distance x difference, the height y difference, and the relative distance z difference are determined independently, and only when all are included in a predetermined range, the same group is used. You can also. For example, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (difference in relative distance z) ² ) Are included in the predetermined range, the same group may be used. With this calculation, an accurate distance between blocks in real space can be derived, so that the grouping accuracy can be improved.

  Subsequently, the specific area dividing unit 166 determines the target object as the specific object if the grouped block group satisfies a predetermined condition. For example, when the grouped block group is located on the road, the specific area dividing unit 166 determines whether the size of the entire block group corresponds to the size of the specific object “vehicle”. If it is determined that it corresponds to the size of the object “vehicle”, the block group is specified as the specific object “vehicle”. Then, the specific area dividing unit 166 distinguishes the area occupied on the screen by the block group specified as the specific object “vehicle” from the other detection areas 122 as the specific area of the vehicle.

  Thus, in the vehicle environment recognition apparatus 130, the detection area 122 can be divided into, for example, specific areas occupied by the vehicle, and the information can be used for various processes. For example, since objects such as brake lamps, tail lamps, blinkers, etc. are attached only to the vehicle, their detection range can be narrowed down only to a specific area. In the following, a process for distinguishing a specific object “brake lamp” from a specific object “red light of a traffic light” by determining the luminance for each specific area will be described.

  The luminance range resetting unit 168 resets the luminance range 202 associated with the color identifier in the color identifier table 200 for each specific region. However, the luminance range 202 that does not need to be changed, i.e., that is not reset is treated as if the value has been reset.

  For example, the color identifier “1” is associated with the luminance range (R) “80 or more”, the luminance range (G) “40 or less”, and the luminance range (B) “40 or less”. By using this luminance range 202, both the specific item “brake lamp” and the specific item “red light of traffic light” can be extracted at a time. However, if the luminance range 202 is simply used, a red lamp cover when the brake lamp is turned off or a vehicle with a red vehicle body may be extracted as an object having the color identifier “1”. . Therefore, regarding the specific area occupied by the vehicle, the luminance is determined after resetting the luminance range 202 of the color identifier 204 associated in the color identifier table 200.

  For example, the brightness of the specific object “brake lamp” is generally defined by laws and regulations, and the brightness range (R) can be increased. Therefore, the brightness range resetting unit 168 sets the brightness range 202 of the color identifier “1” to, for example, the brightness range (R) “160 or more” and the brightness range (G) “80 or less” for the specific area occupied by the vehicle. The luminance range (B) is reset to “80 or less”. Then, the red lamp cover when the brake lamp is turned off and the red color of the vehicle body are excluded from the target of the color identifier “1”, and only the specific object “brake lamp” can be appropriately extracted.

  In addition, the luminance range resetting unit 168 may correct the luminance range 202 of each color identifier according to the imaging state and the environment outside the vehicle. Here, the imaging state includes an exposure mode and a relative distance to the object, and the environment outside the vehicle includes luminance around the vehicle and a color temperature (color balance) of illumination that illuminates the inside of the detection region 122.

  The exposure mode indicates the exposure time and aperture of the imaging apparatus 110. For example, if the exposure time is shortened, the brightness range resetting unit 168 multiplies the brightness range 202 by the reduced ratio to lower the brightness range 202. . Similarly, if the exposure time becomes longer, the luminance range 202 is raised by multiplying the luminance range 202 by the ratio. Further, for example, when the relative distance becomes shorter, the luminance range resetting unit 168 increases the luminance range 202 by removing the reduced ratio from the luminance range 202, and when the relative distance becomes longer, the luminance range resetting unit 168 changes the ratio to the luminance range. The brightness range 202 is lowered by dividing from 202.

  FIG. 8 is an explanatory diagram for explaining correction of the luminance range 202. The luminance of the object is affected by the luminance around the outside of the vehicle and the color temperature of the illumination that illuminates the detection area 122. For example, as shown in FIG. 8A, the brightness of the shade is lower than that of the sun, and the brightness of the object is also lowered accordingly. Therefore, the luminance range resetting unit 168 decreases the luminance range 202 by the reduced amount when the luminance around the outside of the vehicle (for example, the road surface) becomes low, and decreases the luminance when the luminance around the outside of the vehicle becomes high. Increase range 202. In addition, the color balance of the object is affected by illumination that illuminates the detection area 122. For example, as shown in FIG. 8B, when receiving light from a sodium lamp or a halogen lamp, the color temperature is relatively high, and when it is cloudy, the color temperature is relatively low. Therefore, for example, the luminance range resetting unit 168 decreases the luminance range 202 when the illumination color temperature decreases, and increases the luminance range 202 when the illumination color temperature increases.

  Here, the correction of the luminance range by the luminance range resetting unit 168 has been individually described. However, the correction is performed based on one or a plurality of parameters selected from the group of exposure time, relative distance, luminance outside the vehicle, and color temperature. It is also possible to perform correction. In this way, the luminance range resetting unit 168 can cause the luminance range that is the determination criterion to follow the change in luminance of the object, so that the object can be specified as a specific object according to the original luminance. Become.

  The color identifier setting unit 170 sets a color identifier for each pixel based on the luminance of a plurality of pixels (target parts) in the detection region and the luminance range reset in the specific region where the pixel is located.

  First, the color identifier setting unit 170 acquires the luminance (the luminance of three hues (R, G, B) in units of pixels) from the luminance image 124 acquired through the I / F unit 150 in units of pixels.

  Then, the color identifier setting unit 170 sequentially selects the color identifiers 204 registered in the color identifier table 200, and whether or not the acquired luminance of one pixel is included in the luminance range 202 of the sequentially selected color identifier 204. To determine. If the luminance of the target part is included in the luminance range 202 of any one of the color identifiers 204, the color identifier 204 is set for the pixel, and a color identifier map is created.

  However, in this embodiment, the luminance range 202 of the color identifier 204 registered in the color identifier table 200 may not be used as it is. The color identifier setting unit 170 determines whether the luminance range 202 is set by the luminance range resetting unit 168 according to the position of the pixel (whether it is included in the specific region or which specific region), the imaging state, and the environment outside the vehicle. If it has been reset, the luminance of the pixel is determined based on the reset luminance range 202.

  The color identifier setting unit 170 sequentially performs a series of comparisons between the luminance of each target portion and the luminance range 202 of the plurality of color identifiers 204 registered in the color identifier table 200 for each of a plurality of pixels. Here, the selection order of the color identifier 204 is ascending order such as “1”, “2”. When it is determined that the luminance of the pixel is included in the luminance range 202 of the color identifier 204 with a low ordinal as a result of comparison in accordance with such a selection order, the comparison process for the color identifier 204 with a higher ordinal is performed earlier. Absent. Therefore, two or more color identifiers 204 are not attached to one pixel. This is because a plurality of specific objects do not overlap in space, so whether or not an object for which an arbitrary color identifier 204 is once set by the color identifier setting unit 170 is another color identifier 204 is determined. Based on the reason that there is no need to judge. By exclusively handling pixels in this way, it is possible to avoid duplicate setting processing of pixels for which the color identifier 204 has already been set, and to reduce the processing load.

  FIG. 9 is an explanatory diagram for explaining the color identifier map 210. The color identifier map 210 is obtained by superimposing the color identifier 204 on the luminance image 124. Therefore, the color identifier 204 is collectively set at the position of the object corresponding to the specific object.

  Attention is paid to the partial map 210 a in the color identifier map 210. The position of the partial map 210a is not included in the specific area 214 of the vehicle. Therefore, the luminance of the pixel in the partial map 210a is compared with the luminance range 202 of the color identifier “1” in the color identifier table 200, and the color identifier “1” corresponds to the plurality of pixel groups 212 corresponding to the red light of the traffic light. Attached.

  Subsequently, attention is focused on the partial map 210b in the color identifier map 210. The position of the partial map 210b is included in the specific area 214 of the vehicle. Therefore, the luminance of the pixel in the partial map 210b is compared with the luminance range 202 reset based on the specific region 214, and the color identifier “1” is associated with the plurality of pixel groups 212 corresponding to the brake lamps of the vehicle. It is done.

  Next, attention is focused on the partial map 210 c in the color identifier map 210. The position of the partial map 210c is also included in the specific area 214 of the vehicle, like the partial map 210b. However, in the partial map 210c, unlike the partial map 210b, it is assumed that the brake lamp of the vehicle is not lit. Again, the pixels in the partial map 210 c are compared with the brightness range 202 reset based on the specific area 214. However, since the reset luminance range 202 is higher in luminance than the luminance range 202 registered in the color identifier table 200, it is not determined that the pixel in the partial map 210c is included in the luminance range 202. For example, a color identifier “6” corresponding to a vermilion specific object is associated with the plurality of pixel groups 212 corresponding to the brake lamps of the vehicle, instead of the color identifier “1”. In FIG. 9, a diagram in which color identifiers are attached to a plurality of pixels of the luminance image 124 is presented. However, such an expression is conceptual for ease of understanding. As a color identifier 204 is registered.

  The grouping unit 172 uses an arbitrary pixel (target part) as a base point, a range in which the difference between the pixel, the horizontal distance x, the height y, and the relative distance z is predetermined (for example, 0.1 m). The pixels inside are grouped on the assumption that they correspond to the same specific object. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. Further, the grouping unit 172 also applies to a pixel newly added as a result of grouping, with the pixel being the base point, the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z are within a predetermined range. Are further grouped. As a result, all pixels that can be assumed to be the same specific object are grouped.

Also here, the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z are independently determined, and the same group is formed only when all are included in the predetermined range. You can also. For example, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (difference in relative distance z) ² ) Are included in the predetermined range, the same group may be used.

  The specific object determining unit 174 determines the target object as the specific object if the target object (pixel group) grouped by the grouping unit 172 satisfies a predetermined condition. For example, the specific object determining unit 174 determines that the size of the target object (both the width of the horizontal distance x and the width of the height y of the target object) is included in the width range associated with the specific object. The target object is determined as the target specific object. Moreover, you may set the width range about each of the width | variety of the horizontal distance x of the target object, and the width | variety of height y. Here, it is confirmed that the object has a size that is appropriate to be regarded as a specific object. Therefore, when it is not included in the width range, it can be excluded as information unnecessary for the environment recognition process. For example, since the size of the target object (pixel group 212) in the partial map 210b of FIG. 9 is included in the width range of the specific object “brake lamp”, for example, “0.05 to 0.2 m”, the specific object is appropriately selected. Identified as “brake lamp”.

  Thus, the vehicle exterior environment recognition device 130 can extract one or more objects as specific objects from the luminance image 124, and can use the information for various controls. For example, by extracting the specific item “brake lamp”, it is possible to grasp that there is a preceding vehicle traveling with the host vehicle 1 and that the preceding vehicle is decelerating. In addition, by extracting the specific object “red light of the traffic light”, it is understood that the target object is a fixed object that does not move, and if the target object is a traffic signal related to the own lane, The own vehicle 1 can grasp that it should stop or decelerate.

(External vehicle environment recognition method)
Hereinafter, specific processing of the vehicle environment recognition apparatus 130 will be described based on the flowcharts of FIGS. FIG. 10 shows an overall flow relating to the interrupt processing when the luminance image 124 and the distance image 126 are transmitted from the image processing apparatus 120, and FIGS. 11 to 18 show individual subroutines therein. . Further, here, a block or a pixel is cited as a processing target part, and the lower left corner of the luminance image 124 or the distance image 126 is the origin, and in the block, 1 to 150 blocks in the horizontal direction of the image and 1 to 50 in the vertical direction. In the range of the block, in the pixels, the processing by the vehicle environment recognition method is performed in the range of 1 to 600 pixels in the horizontal direction of the image and 1 to 200 pixels in the vertical direction. Here, it is assumed that the number of target color identifiers 204 is eight.

  As shown in FIG. 10, when an interruption by the outside environment recognition method occurs, disparity information for each block in the detection area 122 is acquired as three-dimensional position information (S300), and the representative distance 220 for each divided area 216 is obtained. Is derived (S302). Subsequently, the divided area 216 is grouped to generate a divided area group 222 (S304), the blocks are grouped in the divided area group 222, and the detection area is divided into specific areas according to the grouped specific objects. (S306). The luminance range is reset for the specific area thus divided (S308), and a color identifier is set for the pixel of the luminance image 124 based on the reset luminance range (S310). Then, the color identifiers are grouped (S312), and when the grouped objects satisfy a predetermined condition, they are identified as specific objects (S314). The above processing will be specifically described below.

(Position information acquisition process S300)
Referring to FIG. 11, the position information acquisition unit 160 initializes a vertical variable j for specifying a block (substitutes “0”) (S400). Subsequently, the position information acquisition unit 160 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S402). Next, the position information acquisition unit 160 adds “1” to the horizontal variable i (S404).

  The position information acquisition unit 160 acquires the parallax information dp from the block (i, j, dp) of the distance image 126 (S406). Then, the position information acquisition unit 160 performs coordinate conversion of the block (i, j, dp) including the parallax information dp to a point (x, y, z) in the real space using the above Equations 1 to 3. It is assumed that the block is (i, j, dp, x, y, z) (S408).

  Subsequently, the position information acquisition unit 160 determines whether or not the horizontal variable i exceeds 150 which is the maximum value of the horizontal block (S410), and if the horizontal variable i does not exceed the maximum value (NO in S410), The process from the step S404 of incrementing the horizontal variable i is repeated. If the horizontal variable i exceeds the maximum value (YES in S410), the position information acquisition unit 160 determines whether or not the vertical variable j exceeds 50, which is the maximum value of the vertical block (S412). If the vertical variable j does not exceed the maximum value (NO in S412), the process from the vertical variable j increment process in step S402 is repeated. If the vertical variable j exceeds the maximum value (YES in S412), the position information acquisition process S300 is terminated. Thus, the parallax information dp of the distance image 126 is converted into three-dimensional position information.

(Representative distance derivation process S302)
Referring to FIG. 12, the representative distance deriving unit 162 reads road shape parameters (S450), and divides the detection area 122 into 150 divided areas 216, for example, in units of 4 pixels in the horizontal direction (S452). Next, the representative distance deriving unit 162 sequentially extracts one divided region 216 from the 150 divided regions 216, for example, from the left side in the horizontal direction, and arbitrarily blocks (i, j, dp, x, y, z) are set (S454).

  The representative distance deriving unit 162 calculates the height yr of the road surface at the coordinate z in the real space of the block (S456), and the block whose coordinate y in the real space of the block is equal to or higher than the height yr of the road surface. For example, the relative distance is added (voted) to the histogram divided at predetermined distance intervals (S458). Here, even if the coordinate y in the real space of the block is equal to or higher than the height yr of the road surface, the block having a height of 0.1 m or less from the road surface is white line, dirt, shadow, etc. on the road. Consider and exclude from processing. Also, blocks positioned above the height of the host vehicle 1 are regarded as pedestrian bridges, signs, etc., and are excluded from processing targets.

  The representative distance deriving unit 162 determines whether or not the integration process to the histogram has been performed for all the blocks in one extracted divided region 216 (S460). Here, if all the blocks are not completed (NO in S460), the setting process S454 is repeated for the blocks that have not been subjected to the histogram integration process.

  If all the blocks are completed (YES in S460), the representative distance deriving unit 162 refers to the histogram generated in this way, and the frequency of the histogram (the number of relative distances) is set to a predetermined threshold value (appropriately set). If there is a section that becomes the above, it is determined that a three-dimensional object exists in the divided area 216. The representative distance deriving unit 162 sets the relative distance corresponding to the peak as the representative distance 220 (S462).

  Then, the representative distance deriving unit 162 determines whether the representative distance 220 has been derived for all of the plurality of divided regions 216 (S464). If it is not determined that all the divided areas 216 are completed (NO in S464), a new divided area 216 is set (S466), and the block setting process S454 is repeated for the new divided area 216. On the other hand, if the derivation process for the representative distance 220 has been completed (YES in S464), the representative distance derivation process S302 ends.

(Division area group generation processing S304)
Referring to FIG. 13, the divided region group generation unit 164 sequentially identifies an arbitrary divided region 216 from a plurality of divided regions 216, for example, from the left side in the horizontal direction, and divides the arbitrary divided region 216 adjacent to the right side in the horizontal direction. The area 216 is also specified (S500). Then, the divided region group generation unit 164 determines whether or not the representative distance 220 exists in both the divided regions 216 (S502). If the representative distance 220 does not exist in both the divided areas 216 (NO in S502), the process proceeds to the divided area completion determination step S508. On the other hand, if the representative distance 220 exists in both divided areas 216 (YES in S502), the representative distances 220 of both divided areas 216 are compared (S504).

  Here, if the difference between the two representative distances 220 is equal to or less than a predetermined threshold (a value that can be regarded as the same three-dimensional object), the two representative distances 220 are regarded as close to each other, and the divided region group generation unit 164 The divided areas 216 are grouped to form a divided area group 222 (S506). At this time, when one divided region 216 is already set as the divided region group 222, the other divided region 216 is integrated into the divided region group 222.

  Then, the divided region group generation unit 164 determines whether the generation processing S502, S504, and S506 of the divided region group 222 has been performed for all of the plurality of divided regions 216 (S508). Here, if not all is completed (NO in S508), a new divided area 216 is set (S510), and the processing from the specific process S500 is repeated for the new divided area 216. On the other hand, if all the generation processes of the divided area group 222 have been completed (YES in S508), the divided area group generation process S304 ends.

(Specific area division processing S306)
Referring to FIG. 14, the specific area dividing unit 166 sequentially extracts one divided area group 222 from the grouped divided area groups 222 from the left side in the horizontal direction, for example, and exists in the divided area group 222. Arbitrary blocks (i, j, dp, x, y, z) to be set are set (S550).

  The specific area dividing unit 166 compares the set block (i, j, dp, x, y, z) with a block in the divided area group 222 whose relative distance z corresponds to the representative distance 220, and performs horizontal comparison. It is determined whether the difference in distance x, the difference in height y, and the difference in relative distance z are within a predetermined range (for example, 0.1 m) (S552). If it is within the predetermined range (YES in S552), the blocks are grouped on the assumption that they correspond to the same specific object (S554). If not within the predetermined range (NO in S552), the process proceeds to block completion determination step S556.

  The specific area dividing unit 166 determines whether or not the grouping process has been performed for all the blocks in the extracted one divided area group 222 (S556). Here, if the grouping process for all the blocks has not been completed (NO in S556), the setting process from S550 is repeated for the blocks that have not been subjected to the grouping process.

  If the grouping process for all the blocks has been completed (YES in S556), the specific object determining unit 174 determines that the size of the block group thus grouped is the size of the specific object “vehicle”. It is determined whether it corresponds (S558). If it is determined that it corresponds to the size of the specific object “vehicle” (YES in S558), the block group is specified as the specific object “vehicle” (S560). If it is not determined that it corresponds to the size of the specific object “vehicle” (NO in S558), the process proceeds to the completion determination step S562 of the divided region group 222.

  The specific area dividing unit 166 determines whether or not the specific object determination determinations S558 and S560 have been performed for all of the plurality of divided area groups 222 (S562). If it is not determined that all of the divided region groups 222 have been completed (NO in S562), a new divided region group 222 is set (S564), and the block setting process S550 for the new divided region group 222 is performed. repeat. On the other hand, if the specific object determination determinations S558 and S560 are all completed (YES in S562), the specific area dividing process S306 is terminated.

  In the grouping, the mutual positional relationship is further determined for a plurality of groups. For example, when the position of the end point is close between groups of the same kind of three-dimensional object and the transition of the relative distance in the horizontal direction and the vertical direction in the three-dimensional object is substantially equal (continuous), It is determined that they are the same surface, and these groups are integrated into one group. At this time, the transition of the relative distance in the horizontal direction and the vertical direction in the three-dimensional object can be specified by an approximate straight line by the Hough transform or the least square method. In the case of a preceding vehicle, a plurality of groups can be integrated into one group even if the relative movement speed with respect to the z coordinate is equal.

  In addition, when the processing so far is performed in units of blocks, the same information is set in all the pixels in the block, thereby changing the units of pixels.

(Luminance range resetting process S308)
Referring to FIG. 15, the luminance range resetting unit 168 determines whether or not a specific area is included in the detection area 122 (S600). If the specific area is included (YES in S600), the luminance range resetting unit 168 resets the luminance range 202 based on the specific object indicated by the specific area (S602). Then, the brightness range resetting unit 168 determines whether or not the resetting process S602 has been performed for all the specific areas (S604). If it is not determined that all the specific areas have been completed (NO in S604), a new specific area is set (S606), and the reset process S602 is repeated for the new specific area. On the other hand, if all the specific areas are completed (YES in S604), the process proceeds to exposure time determination step S608. Also, when the specific area is not included in the detection area 122 (NO in S600), the process proceeds to the exposure time determination step S608.

  Subsequently, the luminance range resetting unit 168 determines whether or not the exposure time of the imaging device 110 has been changed (S608). If the exposure time has been changed (YES in S608), the luminance range 202 is corrected based on the ratio (S610). If the exposure time has not been changed (NO in S608), the process proceeds to the relative distance determination step S612. Move.

  Next, the luminance range resetting unit 168 determines whether or not the relative distance of the object has been changed (S612). If the relative distance has been changed (YES in S612), the luminance range 202 is corrected based on the ratio (S614). If the relative distance has not been changed (NO in S612), the process proceeds to the peripheral luminance determination step S616. Move.

  Subsequently, the luminance range resetting unit 168 determines whether the luminance around the outside of the vehicle has changed (S616). If the luminance has changed (YES in S616), the luminance range 202 is corrected based on the luminance after the change (S618), and if the luminance has not changed (NO in S616), the process proceeds to the illumination color temperature determination step S620. Move.

  Next, the brightness range resetting unit 168 determines whether or not the color temperature of the illumination has changed (S620). If the color temperature has changed (YES in S620), the luminance range 202 is corrected based on the changed color temperature (S622). If the color temperature has not changed (NO in S620), the luminance range resetting process is performed. S308 ends.

(Color identifier setting process S310)
Referring to FIG. 16, the color identifier setting unit 170 initializes (substitutes “0”) a vertical variable j for specifying a pixel (S650). Subsequently, the color identifier setting unit 170 adds (increments) “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S652). Next, the color identifier setting unit 170 adds “1” to the horizontal variable i, and initializes the color identifier variable m (substitutes “0”) (S654). Here, the horizontal variable i and the vertical variable j are provided to execute the identifier setting process for all 600 × 200 pixels, and the color identifier variable m is provided for the pixels. This is because the eight color identifiers 204 are sequentially compared every time.

  The color identifier setting unit 170 acquires the luminance br of the pixel (i, j, br) as the target part from the luminance image 124 (S656), adds “1” to the color identifier variable m (S658), and the pixel ( The luminance range 202 of the color identifier (m) or the reset luminance range 202 is acquired according to the position of i, j, br) (S660), and the luminance br of the pixel (i, j, br) is obtained. It is determined whether it is included in the luminance range 202 of the color identifier (m) (S662).

  If the luminance br of the pixel is included in the luminance range 202 of the color identifier (m) (YES in S662), the color identifier setting unit 170 associates the identification number p indicating the color identifier (m) with the pixel. , Pixel (i, j, br, p) (S664). In this way, the color identifier map 210 in which the identification number is assigned to each pixel in the luminance image 124 is generated. If the luminance br of the pixel (i, j, br) is not included in the luminance range 202 of the color identifier (m) (NO in S662), whether or not the color identifier variable m has exceeded the maximum number of eight. Determination is made (S666). If the color identifier variable m does not exceed the maximum value (NO in S666), the process of incrementing the color identifier variable m in step S658 is repeated. If the color identifier variable m exceeds the maximum value (YES in S666), it is determined that there is no color identifier 204 corresponding to the pixel (i, j, br), and the process is moved to the horizontal pixel determination step S668. .

  Subsequently, the color identifier setting unit 170 determines whether or not the horizontal variable i exceeds 600, which is the maximum value of the horizontal pixel (S668), and if the horizontal variable i does not exceed the maximum value (NO in S668), The process from the step S654 of incrementing the horizontal variable i is repeated. If the horizontal variable i exceeds the maximum value (YES in S668), the color identifier setting unit 170 determines whether or not the vertical variable j exceeds 200, which is the maximum value of the vertical pixel (S670). If the vertical variable j does not exceed the maximum value (NO in S670), the process of incrementing the vertical variable j in step S652 is repeated. If the vertical variable j exceeds the maximum value (YES in S670), the color identifier setting process S310 ends.

(Grouping process S312)
Referring to FIG. 17, the grouping unit 172 refers to a predetermined range (for example, 0.1 m) for grouping pixels (S700), and initializes a vertical variable j for identifying the pixels (“0”). Is substituted) (S702). Subsequently, the grouping unit 172 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S704). Next, the grouping unit 172 adds “1” to the horizontal variable i (S706).

  The grouping unit 172 acquires a pixel (i, j, br, p, dp) including the parallax information dp from the luminance image 124, and executes the pixel (i, j, br, p, dp) including the parallax information dp. The coordinates are converted to a point (x, y, z) in space to obtain a pixel (i, j, br, p, dp, x, y, z) (S708). Whether a valid (non-zero) color identifier p exists for the pixel (i, j, br, p, dp, x, y, z) and the group number g is not yet assigned. Determination is made (S710). Here, if there is a valid color identifier p and the group number g has not yet been assigned (YES in S710), the grouping unit 172 determines the coordinates (x, y, z) of the pixel in the real space. ) In the predetermined range, it is determined whether or not there is one or more other pixels for which the color identifier p is set and the group number g has not been assigned to the pixel (S712).

  If one or a plurality of other pixels (i, j, br, p, dp, x, y, z) for which the color identifier p is set exist and the group number g is not yet attached to the pixel (S712) YES), the smallest value among the numbers not yet used as the group number is newly assigned to all pixels within a predetermined range including itself (S714). Here, pixels that have already been grouped as other objects are excluded from the grouping process and are not grouped.

  Thus, when there are a plurality of pixels having the same color identifier within a predetermined range, grouping is performed by assigning a group number g of 1. At this time, the reason why the smallest value among the numbers not yet used as the group number is employed is to prevent missing numbers as much as possible in group numbering. By doing so, the maximum value of the group number g is not increased unnecessarily, and the processing load can be reduced.

  If the color identifier p is not a valid value (0), or if it is a valid value but the group number g has already been assigned (NO in S710), there are no other pixels with the same color identifier, Alternatively, if it exists but the group number g has already been assigned to all the pixels (NO in S712), the process proceeds to the horizontal variable determination step S716.

  Subsequently, the grouping unit 172 determines whether or not the horizontal variable i exceeds 600, which is the maximum value of the horizontal pixel (S716), and if the horizontal variable i does not exceed the maximum value (NO in S716), step The process from step S706 for incrementing the horizontal variable i is repeated. If the horizontal variable i exceeds the maximum value (YES in S716), the grouping unit 172 determines whether the vertical variable j exceeds 200, which is the maximum value of the vertical pixels (S718). If the vertical variable j does not exceed the maximum value (NO in S718), the process of incrementing the vertical variable j in step S704 is repeated. If the vertical variable j exceeds the maximum value (YES in S718), the grouping process S312 ends.

(Specific object determination process S314)
Referring to FIG. 18, the specific object determining unit 174 initializes (assigns “0”) a group variable k for specifying a group (S752). Subsequently, the specific object determining unit 174 adds “1” to the group variable k (S754).

  The specific object determining unit 174 determines whether or not there is an object whose group number g is the group variable k from the luminance image 124 (S756), and if it exists (YES in S756), the group number g is attached. The size of the target object is calculated (S758). At this time, the size of the object is determined by the horizontal direction component (difference) between the pixel located at the left edge of the object and the pixel located at the right edge of the object, the pixel located at the upper edge of the object, and the lower edge of the screen. It is specified by the vertical direction component which is the height (difference) between the pixels located at. Then, it is determined whether or not the calculated size is included in the width range of the specific object indicated by the color identifier p associated with the object whose group number g is the group variable k (S760). For example, the horizontal component of the size of the object is within the width range of the specific object indicated by the color identifier p, and the vertical component of the size of the object is the specific object indicated by the color identifier p. If it is within the width range, it can be determined that the object is included in the width range of the specific object indicated by the color identifier p.

  If the size is included in the width range of the specific object indicated by the color identifier p (YES in S760), the specific object determining unit 174 determines the target object as the specific object (S762). If the size is not included in the width range of the specific object indicated by the color identifier p (NO in S760) or there is no object whose group number g is the group variable k (NO in S756), the group variable Processing proceeds to decision step S764.

  Subsequently, the specific object determining unit 174 determines whether or not the group variable k has exceeded the maximum value of the group numbers set in the grouping process (S764). If the group variable k does not exceed the maximum value (NO in S764), the process of incrementing the group variable k in step S754 is repeated. If the group variable k exceeds the maximum value (YES in S764), the specific object determination process S314 is terminated. In this way, the grouped objects are formally determined as specific objects.

  As described above, according to the outside environment recognition device 130 and the outside environment recognition method described above, a plurality of types of brightness approximated while avoiding unnecessary increase in processing load by avoiding unnecessary subdivision of detection targets. The object can be identified with high accuracy.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 130 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

(Effect of this embodiment)
FIG. 19 is an explanatory diagram for explaining the effect of the present embodiment. Here, according to FIG. 19, the effect by the vehicle exterior environment recognition apparatus 130 and vehicle exterior environment recognition method mentioned above is confirmed. Assume that a luminance image 124 as shown in FIG. 19A is acquired while the host vehicle 1 is traveling. Here, when the specific object “brake lamp” and the specific object “red light of the traffic light” are tried to be specified without applying the present embodiment, as shown in FIG. Pixels satisfying the luminance range 202 of “1” were extracted. However, in FIG. 19B, in addition to the emission source 230 of the specific item “brake lamp” and the specific item “red light of the traffic light”, the lamp cover 232 having a red color although the brake lamp is not lit is erroneously displayed. It will be extracted.

  At this time, if the exposure time is shortened uniformly, the luminance of the lamp cover 232 of the brake lamp that is not lit does not satisfy the luminance range 202 of the color identifier “1”, so the lamp cover 232 of the brake lamp that is not lit is excluded. At the same time, the red light of the traffic light will be eliminated. This is due to the fact that the red light of the traffic light is located far away, so that the brightness is low and cannot be distinguished from the brake lamp that is not lit.

  However, on the specific area 234 extracted as the vehicle shown in FIG. 19 (c), it is possible to clearly distinguish between lighting and non-lighting of the brake lamp. Criteria are set so that the brake lamps that are lit are clearly distinguished from others. Then, as shown in FIG. 19D, a specific object satisfying the luminance range 202 of the color identifier “1” is narrowed down to the light emission source 230, and the brake lamp and the red light of the traffic light are appropriately used for collision avoidance control and cruise control. It will be.

(Modification)
In the embodiment described above, the area occupied by the preceding vehicle is set as the specific area. Therefore, in addition to the brake lamp described above, a determination different from normal can be realized through the reset brightness range for the tail lamp, winker, license plate, and the like attached to the vehicle.

Further, the specific area is not limited to the area occupied by the vehicle, and the following area can also be set as the specific area as in the case of the vehicle, and the following specific items (1) to (9) in the specific area are determined. It can be carried out.
(1) Specific area: an area occupied by an XZ (y = 0) plane (road surface) specified based on the three-dimensional coordinates of the lane extracted from the luminance change with the road, specific object: white line on the road surface , Crosswalks, road markings.
(2) Specific area: an area occupied by a road sign, specific object: characters, numerical values, marks, and other indicators in the road sign.
(3) Specific area: an area occupied by a pedestrian (person), specific object: an object worn by the pedestrian.
(4) Specific area: an area occupied by a moving three-dimensional object, specific object: an object for which it is previously known that the three-dimensional object is provided.
(5) Specific area: Area occupied by a predetermined height range from the road surface in real space, specific object: traffic light, road sign, pedestrian bridge.
(6) Specific area: Area occupied by a predetermined relative distance range in real space, specific object: preceding vehicle.
(7) Specific area: Area occupied by a predetermined horizontal distance range in real space, specific object: road sign, guardrail, obstacle.
(8) Specific area: a divided range divided in the horizontal or vertical direction of the image with reference to the infinity point (disappearance point) on the image (for example, an area above or below the infinity point on the image) Area), specific objects: traffic lights, road signs, pedestrian bridges if they are above the infinity point, leading vehicles, road signs, guardrails, obstacles if they are below the infinity point.
(9) Specific area: When tracking a specific object as described above through image processing, if the specific object is lost, the horizontal or vertical direction of the image with reference to the position on the lost image Area occupied by a divided range (a range centered on the reference or a range above, below, left and right from the reference), a specific object: a specific object lost in tracking.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the three-dimensional position of the object is derived based on the parallax between the image data using the plurality of imaging devices 110. However, the present invention is not limited to this. For example, a laser radar ranging device Various known distance measuring devices can be used. Here, the laser radar distance measuring device projects a laser beam onto the detection region 122, receives light reflected by the laser beam upon the object, and measures the distance from the required time to the object.

  In the embodiment described above, the relative distance of the block is obtained using the distance image 126. However, when the specific object can be specified to some extent by the arrangement and size of the object on the screen, the monocular imaging device The present embodiment can be realized by 110. Also, a specific object can be specified by deriving a motion vector by optical flow.

  In the above-described embodiment, it is assumed that the imaging apparatus 110 acquires a color image. However, the present embodiment is not limited to this, and the present embodiment can also be performed by acquiring a monochrome image. In this case, the color identifier table 200 is defined with the luminance of a single color.

  In the above-described embodiment, an example is given in which the position information acquisition unit 160 receives the distance image (parallax information) 126 from the image processing apparatus 120 and generates three-dimensional position information. However, the present invention is not limited to this, and the image processing device 120 may generate three-dimensional position information in advance, and the position information acquisition unit 160 may acquire the generated three-dimensional position information. In this way, by distributing functions, it is possible to reduce the processing load on the outside environment recognition device 130.

  In the above-described embodiment, the position information acquisition unit 160, the representative distance derivation unit 162, the divided region group generation unit 164, the specific region division unit 166, the luminance range resetting unit 168, the color identifier setting unit 170, the grouping unit 172, the specific object determining unit 174 is configured to operate by software by the central control unit 154. However, the functional unit described above can be configured by hardware.

  Note that each step of the vehicle environment recognition method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a vehicle environment recognition apparatus and a vehicle environment recognition method that recognize an environment outside the host vehicle.

DESCRIPTION OF SYMBOLS 1 ... Own vehicle 110 ... Imaging device 122 ... Detection area 124 ... Luminance image 126 ... Distance image 130 ... Outside-vehicle environment recognition device 160 ... Position information acquisition part 162 ... Representative distance deriving part 164 ... Division area group generation part 166 ... Specific area division Unit 168 ... luminance range resetting unit 170 ... color identifier setting unit 172 ... grouping unit 174 ... specific object determining unit

Claims (4)

A data holding unit that holds a plurality of color identifiers and luminance ranges in association with each other;
An image acquisition unit that acquires an image of the environment outside the vehicle;
A specific area dividing unit that divides the detection area of the image into a plurality of specific areas according to a relative positional relationship with the host vehicle;
A luminance range resetting unit that resets the luminance range associated with the color identifier for each specific region;
A color identifier setting unit configured to set the color identifier in the target portion based on the luminance of the plurality of target portions in the detection region and the luminance range reset in the specific region where the target portion is located;
A grouping unit for grouping one or a plurality of target parts whose horizontal distance difference and height difference are within a predetermined range;
A vehicle exterior environment recognition device comprising:   The vehicle exterior environment recognition apparatus according to claim 1, wherein the brightness range resetting unit corrects the brightness range of the color identifier according to an imaging state or a vehicle exterior environment.   The specific area includes a vehicle, a road surface, a road sign, a pedestrian, a moving three-dimensional object, a predetermined height range, a predetermined relative distance range, a predetermined horizontal distance range, and a horizontal direction of an image. The outside environment recognition device according to claim 1, wherein the vehicle is an area occupied by one or more specific objects selected from a group of division ranges divided in the vertical direction. A plurality of color identifiers and luminance ranges are stored in association with each other.
Get an image of the environment outside the car,
Dividing the detection area of the image into a plurality of specific areas according to the relative positional relationship with the host vehicle,
Resetting the luminance range associated with the color identifier for each specific area;
Based on the luminance of a plurality of target parts in the detection area and the luminance range reset to the specific area where the target part is located, the color identifier is set to the target part,
A method for recognizing an environment outside a vehicle, wherein one or a plurality of target parts whose horizontal distance difference and height difference are within a predetermined range are grouped.