JP6313667B2 - Outside environment recognition device - Google Patents

Description

  The present invention relates to a vehicle environment recognition apparatus that identifies which specific object corresponds to a three-dimensional object existing in a detection area.

  Conventionally, a specific object such as a vehicle positioned in front of the host vehicle is detected, and a collision with a preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is controlled to be a safe distance ( (Cruise control) technology is known (for example, Patent Document 1). Such forward monitoring technology of the host vehicle is expected to be effective in avoiding and reducing contact accidents with objects such as preceding vehicles and pedestrians.

  In Patent Document 1, as a technique for identifying an object, a three-dimensional position for each block composed of a predetermined pixel group is derived from an image obtained by capturing an environment outside the vehicle in front of the host vehicle, and blocks having similar three-dimensional positions are derived from each other. A technique is described in which a three-dimensional object is specified by grouping, and a solid object corresponds to the specific object based on the size, shape, and the like of the solid object.

Japanese Patent No. 3349060

  By the way, when the environment outside the vehicle is imaged under an environment such as rainfall or snowfall, an image of fallen objects such as raindrops or snowdrops that fall during the exposure time is captured in a line shape on the image. Here, when the object identification technique as described above is used in an environment such as rainfall or snowfall, a plurality of blocks based on a plurality of falling objects captured in a line shape occupy a certain area on the image. It is necessary to prevent them from being identified as specific objects such as vehicles and pedestrians because they are grouped as objects.

  In view of such a problem, an object of the present invention is to provide a vehicle environment recognition apparatus that can improve the accuracy of specifying a specific object.

  In order to solve the above problems, an external environment recognition apparatus according to the present invention divides an acquired image into a plurality of blocks each composed of one or a plurality of pixels, and acquires the image. A position information deriving unit for deriving a three-dimensional position of the image, a cluster generating unit for generating a cluster by clustering a plurality of blocks having the three-dimensional position within a predetermined first range, and the position information deriving unit. A cluster position deriving unit that derives, for each cluster, a cluster three-dimensional position, which is a three-dimensional position of the cluster, based on the three-dimensional position of the block, and the cluster three-dimensional position is larger than the first range. A group generation unit configured to generate a group by grouping a plurality of clusters within two ranges, and the cluster based on an occupation area of the cluster in the image. A cluster width deriving unit for deriving a horizontal width of each group for each cluster, a group width deriving unit for deriving a horizontal width of the group based on the horizontal width of the cluster, and the group The total value of the horizontal widths of the plurality of clusters is less than a first threshold set based on the horizontal width of the group, and the horizontal direction of all of the plurality of clusters constituting the group Is less than a second threshold set based on a horizontal width to be specified as at least part of the predetermined specific object, the group is ungrouped and returned to the plurality of clusters. The group cancellation unit, the group, and the ungrouped cluster are defined as one solid object, and when the solid object satisfies a predetermined condition, the solid object is the predetermined specific object. Characterized in that and a specific object identifying unit that identifies.

  In addition, when the group generation unit groups three or more clusters to generate one group, the group cancellation unit has a total value of horizontal widths of the plurality of clusters constituting the group as the first value. A horizontal direction between two adjacent clusters in the group, wherein the horizontal width of all of the plurality of clusters constituting the group is less than the second threshold value. If any one of the separation distances is equal to or greater than a third threshold set based on the separation distance to be the one three-dimensional object, the grouping may be canceled and returned to the plurality of clusters. .

  In order to solve the above-described problem, another external environment recognition device of the present invention divides an image acquisition unit that acquires an image and the acquired image into a plurality of blocks including one or a plurality of pixels. A position information deriving unit for deriving a three-dimensional position for each block; a cluster generating unit for generating a cluster by clustering a plurality of blocks having the three-dimensional position within a predetermined first range; and the position information deriving unit. A cluster position deriving unit that derives, for each cluster, a cluster three-dimensional position that is a three-dimensional position of the cluster based on the derived three-dimensional position of the block; and the cluster based on the occupied area of the cluster in the image A horizontal width of each of the clusters, and a total value of the horizontal widths of the plurality of clusters is equal to or greater than a first threshold value. Alternatively, when the horizontal widths of all of the plurality of clusters are equal to or greater than a second threshold set based on the horizontal width to be specified as at least a part of the predetermined specific object, the plurality of clusters A group generation unit that generates a group by grouping the group and the cluster that is not grouped into one solid object, and when the solid object satisfies a predetermined condition, the solid object is A specific object specifying unit that specifies a specific object, wherein the first threshold value is set based on a horizontal width of the group derived according to a horizontal width of the cluster. To do.

  In addition, when there are three or more clusters in the second range, the group generation unit is configured such that a total value of horizontal widths of the plurality of clusters is not equal to or more than the first threshold value, and the plurality of clusters When all horizontal widths are not equal to or greater than the second threshold value, any one of the horizontal widths between two adjacent clusters among the plurality of clusters is the separation distance to be the three-dimensional object. If it is less than the third threshold value set based on, the plurality of clusters may be grouped.

  Further, when the group generation unit generates two groups by grouping two clusters, the first threshold is based on a relative distance between the plurality of clusters in addition to the horizontal width of the group. It may be set.

  According to the present invention, it is possible to improve the accuracy of specifying a specific object.

It is the block diagram which showed the connection relation of the environment recognition system. It is a functional block diagram showing the schematic function of the outside environment recognition device concerning a 1st embodiment. It is a flowchart which shows the external environment recognition process concerning 1st Embodiment. It is explanatory drawing for demonstrating a luminance image and a distance image. It is a figure for demonstrating the process of a cluster production | generation part. It is a figure for demonstrating the process of a cluster production | generation part. It is a figure for demonstrating the grouping process by a group production | generation part. It is a figure for demonstrating the grouping process by a group production | generation part. It is a figure for demonstrating the process of a cluster width deriving part, a group width deriving part, and a group cancellation | release part. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus concerning 2nd Embodiment. It is a flowchart which shows the external environment recognition process concerning 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, an in-vehicle camera mounted on a vehicle images an environment outside the vehicle ahead, identifies a three-dimensional object such as a preceding vehicle based on color information and position information in the captured image, Vehicles equipped with a so-called collision prevention function that avoids such collisions and keeps the distance between the vehicle and the preceding vehicle at a safe distance (ACC: Adaptive Cruise Control) are becoming popular.

  In the ACC and the collision prevention function, for example, a relative distance between a three-dimensional object located in front of the own vehicle and the own vehicle is derived, and based on the relative distance, a collision with a three-dimensional object located in front of the own vehicle is detected. If it is avoided or the three-dimensional object is a vehicle (preceding vehicle), control is performed so that the relative distance to the preceding vehicle is kept at a safe distance. Hereinafter, an environment recognition system for achieving such an object will be described, and a vehicle exterior environment recognition apparatus as a specific component thereof will be described in detail.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, vehicle exterior environment recognition devices 120 and 420, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 includes an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), captures an environment corresponding to the front of the host vehicle 1, and captures three hues (R ( Brightness images and monochrome image data consisting of red), G (green), and B (blue)) are generated. Here, an image picked up by the image pickup device 110 is adopted as a luminance image, and is distinguished from a distance image described later.

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates image data obtained by imaging a three-dimensional object existing in the detection area in front of the host vehicle 1, for example, every 1/60 second frame (60 fps). Here, solid objects to be recognized in the image are not only solid objects such as vehicles, pedestrians, traffic lights, roads (traveling paths), guardrails, buildings, but also brake lights, high-mount stop lamps, tail lamps, The thing which can be specified as a part of solid objects, such as each lighting part of a blinker and a traffic light, is also included. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  The vehicle exterior environment recognition devices 120 and 420 acquire image data from each of the two imaging devices 110, derive parallax using so-called pattern matching, and correspond the derived parallax information (corresponding to a depth distance described later) to the image data. Add a distance image. The luminance image and the distance image will be described in detail later. Further, the outside environment recognition devices 120 and 420 use the three-dimensional position information in the real space including the relative distance to the host vehicle 1 to determine which three-dimensional object in the detection area in front of the host vehicle 1 is (for example, Specify whether it corresponds to the preceding vehicle).

  The vehicle exterior environment recognition devices 120 and 420, when specifying a specific object as a three-dimensional object, for example, a preceding vehicle, derives a relative distance from the preceding vehicle, a relative speed of the preceding vehicle, and the like while tracking the preceding vehicle. It is determined whether or not there is a high possibility that the preceding vehicle and the host vehicle 1 collide. When it is determined that there is a high possibility of a collision with the preceding vehicle, the outside environment recognition devices 120 and 420 display a warning (notification) to the driver through the display 122 installed in front of the driver. Then, information indicating that is output to the vehicle control device 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition devices 120 and 420. For example, when information indicating that there is a high possibility of a collision with a preceding vehicle is input from the outside environment recognition devices 120 and 420, the vehicle control device 130 supports the driver's braking operation through the braking mechanism 146.

  Hereinafter, the configuration of the outside environment recognition devices 120 and 420 will be described in detail. Here, the process for specifying a predetermined specific object, which is characteristic of the present embodiment, will be described in detail, and the description of the configuration unrelated to the characteristics of the present embodiment will be omitted.

(First Embodiment: Outside environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the vehicle exterior environment recognition apparatus 120 according to the first embodiment. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 152 holds various pieces of information necessary for the processing of each function unit described below, and temporarily holds image data received from the imaging device 110. To do.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like, and through the system bus 156, an I / F unit 150, a data holding unit 152 and the like are controlled. In the present embodiment, the central control unit 154 includes the image acquisition unit 160, the image processing unit 162, the position information deriving unit 164, the cluster generation unit 166, the cluster position deriving unit 168, the group generation unit 170, and the cluster width deriving unit 172. , And also functions as a group width deriving unit 174, a group releasing unit 176, and a specific object specifying unit 178. Hereinafter, the operation of each functional unit will be described, and a vehicle environment recognition process characteristic of the present embodiment will be described in detail.

(External vehicle environment recognition processing)
FIG. 3 is a flowchart showing an environment recognition process outside the vehicle according to the first embodiment. The image acquisition unit 160 of the outside environment recognition device 120 acquires image data (luminance image) from each of the two imaging devices 110 (S300).

  The image processing unit 162 searches for a block corresponding to a block arbitrarily extracted from one image data (for example, an array of horizontal 4 pixels × vertical 4 pixels) from the other image data, so-called parallax using so-called pattern matching, and The parallax information including the screen position indicating the position of the arbitrary block in the screen is derived (S302). Here, the horizontal indicates the horizontal direction of the captured image, and the vertical indicates the vertical direction of the captured image. As this pattern matching, it is conceivable to compare the luminance (Y) in an arbitrary block unit between a pair of images. For example, SAD (Sum of Absolute Difference) that takes the difference in luminance value, SSD (Sum of Squared intensity Difference) that uses the difference squared, and the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel. There are methods such as NCC (Normalized Cross Correlation). The image processing unit 162 performs such a block-unit parallax derivation process for all blocks displayed in the detection area (for example, 600 pixels × 200 pixels). Here, the block is 4 pixels × 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the image processing unit 162 can derive the parallax for each block, which is a unit of detection resolution, but cannot recognize what kind of three-dimensional object the block is. Therefore, the parallax information is independently derived not in units of solid objects but in units of detection resolution (for example, blocks) in the detection region. Here, an image in which the parallax information derived in this way is associated with image data is referred to as a distance image in distinction from the above-described luminance image.

  FIG. 4 is an explanatory diagram for explaining the luminance image 126 and the distance image 128. For example, assume that a luminance image (image data) 126 as shown in FIG. 4A is generated for the detection region 124 through the two imaging devices 110. The image processing unit 162 obtains the parallax for each block from the luminance image 126 and forms a distance image 128 as shown in FIG. Each block in the distance image 128 is associated with the parallax of the block. Here, for convenience of description, blocks from which parallax is derived are represented by black dots.

  Returning to FIG. 3, the position information deriving unit 164 uses the stereo method to calculate disparity information for each block in the detection area 124 in the distance image 128, the horizontal distance x, and the height y (from the road surface). And converted into three-dimensional position information including the relative distance z (S304). The stereo method is a method of deriving a relative distance of the three-dimensional object from the imaging device 110 from the parallax of the three-dimensional object by using a triangulation method. Here, the parallax information indicates the parallax of each block in the distance image 128, while the three-dimensional position information indicates information on the relative distance of each block in the real space. Further, when the disparity information is derived not in pixel units but in block units, that is, in a plurality of pixel units, the disparity information may be regarded as disparity information of all pixels belonging to the block, and calculation in pixel units may be executed. it can. Regarding the conversion to the three-dimensional position information, since existing techniques such as JP2013-109391A can be referred to, detailed description thereof is omitted here.

  Subsequently, the cluster generation unit 166 generates a cluster by clustering a plurality of blocks whose three-dimensional positions are within a predetermined first range.

  5 and 6 are diagrams for explaining the processing of the cluster generation unit 166. FIG. The cluster generation unit 166 first divides the detection area 124 of the distance image 128 into a plurality of divided areas 216 in the horizontal direction (S306). Then, the divided area 216 has a strip shape as shown in FIG. Such a strip-shaped divided region 216 is originally composed of, for example, 150 columns each having a horizontal width of 4 pixels, but here, for convenience of explanation, the detection region 124 is divided into 16 parts.

  Subsequently, for each divided region 216, the cluster generation unit 166 adds up the relative distances included in each of the predetermined distances divided into a plurality of blocks located above the road surface based on the position information. A histogram (indicated by a horizontally long square (bar) in FIG. 5B) is generated (S308). Then, a distance distribution 218 as shown in FIG. 5B is obtained. Here, the vertical direction indicates the divided predetermined distance (distance division), and the horizontal direction indicates the number of blocks (frequency) in which the relative distance is included in each distance division. However, FIG. 5B is a virtual screen for calculation, and actually does not involve generation of a visual screen. Then, the cluster generation unit 166 refers to the distance distribution 218 derived in this way, and represents a representative distance 220 (shown by a black square in FIG. 5B) that is a relative distance corresponding to the peak. Specify (S310). Here, “corresponding to a peak” means a peak value or a value that satisfies an arbitrary condition in the vicinity of the peak.

  Next, the cluster generation unit 166 compares the adjacent divided areas 216 with each other, and groups the divided areas 216 whose representative distance 220 is close (for example, located at 0.1 m or less) as shown in FIG. One or a plurality of divided region groups 222 are generated (S312). At this time, even when the representative distance 220 is close in three or more divided areas 216, all the continuous divided areas 216 are collected as a divided area group 222. Thereby, the cluster generation unit 166 can select one or more solid objects positioned above the road surface as candidates for the specific object.

  Subsequently, the cluster generation unit 166 uses the block in the divided region group 222 whose relative distance z corresponds to the representative distance 220 as a base point, and the block and the three-dimensional position are within a predetermined first range (with a horizontal distance x). Blocks in which the difference, the difference in height y and the difference in relative distance z are within a predetermined range (for example, within 0.1 m) are clustered on the assumption that they correspond to the same three-dimensional object (S314).

  Thus, a cluster 224 that is a virtual block group is generated. Said 1st range is represented by the distance on real space, and can be set to arbitrary values by a manufacturer or a passenger. Also, the cluster generation unit 166 further clusters the blocks within the first range with the block newly added by clustering as a base point. As a result, all blocks that can be assumed to be the same three-dimensional object are clustered.

Here, the difference in horizontal distance x, the difference in height y, and the difference in relative distance z are determined independently, and only when all are included in the first range, the same cluster 224 is used. It can also be calculated. For example, as the first range, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (relative distance) The same cluster 224 may be used when the difference z) ² ) is included in the predetermined range. With this calculation, an accurate distance between blocks in real space can be derived, so that the accuracy of clustering can be improved.

  Returning to FIG. 3, the cluster position deriving unit 168 determines the cluster three-dimensional position, which is the three-dimensional position of the cluster 224, for each cluster 224 based on the three-dimensional position of the block derived by the position information deriving unit 164. Derived (S316). For example, the cluster position deriving unit 168 derives the barycentric position of the blocks constituting the cluster 224 as the cluster three-dimensional position, or derives the average value of the three-dimensional positions of the blocks constituting the cluster 224 as the cluster three-dimensional position. Alternatively, the three-dimensional position where the frequency of appearance of the blocks constituting the cluster 224 is highest can be set as the cluster three-dimensional position.

  The group generation unit 170 has a cluster three-dimensional position within a second range that is larger than the first range (a range in which a difference in horizontal distance x, a difference in height y, and a difference in relative distance z are predetermined (for example, 0 A group is generated by grouping a plurality of clusters 224 within (5m)) (S318).

  7 and 8 are diagrams for explaining the grouping process S318 performed by the group generation unit 170. FIGS. 7A, 8A, and 8D show the luminance image 126, and FIG. FIGS. 7B, 8B, and 8E show conceptual diagrams of the cluster 224 generated by the cluster generation unit 166. FIGS. 8C and 8F show the group generation unit 170. The conceptual diagram of the group 226 to produce | generate is shown. However, FIG. 7 (b), FIG. 8 (b), FIG. 8 (c), FIG. 8 (e), and FIG. 8 (f) are virtual in which information regarding the height of the block is omitted in the calculation. This is a typical screen and does not actually involve the generation of a visual screen.

  For example, as illustrated in FIG. 7A, when an environment outside the vehicle including a preceding vehicle is imaged under a clear sky environment, the entire preceding vehicle is reflected in the luminance image 126. When the cluster generation unit 166 generates the cluster 224 based on the distance image 128 formed from the luminance image 126, one preceding vehicle is generated as one cluster 224 as shown in FIG. It becomes.

  On the other hand, as shown in FIG. 8A, when an environment outside the vehicle including a preceding vehicle is imaged under an environment such as rain or snow, a falling object is superimposed on the preceding vehicle, and the preceding vehicle is divided (divided) by the falling object. ) And may appear in the luminance image 126. When the cluster generation unit 166 generates the cluster 224 based on the distance image 128 formed from the luminance image 126, a plurality of (for example, three) clusters from one preceding vehicle as shown in FIG. 8B. 224 will be generated. In this case, as illustrated in FIG. 8C, the group generation unit 170 generates a group 226 by grouping a plurality of clusters 224 whose cluster three-dimensional positions are within the second range that is larger than the first range. By doing so, one solid object (preceding vehicle) can be handled as one group 226 (block group).

  Further, as shown in FIG. 8A, the luminance image 126 includes an image of falling objects (raindrops, snowdrops, etc.) in a linear shape. Therefore, a cluster 224 is also generated for the falling object. However, if the cluster three-dimensional position is outside the second range, the cluster 224 based on the falling object is shown in FIG. 8C. Are not grouped.

  However, as shown in FIGS. 8D and 8E, when the three-dimensional positions of the plurality of clusters 224 based on the fallout are within the second range, the plurality of clusters 224 based on the fallout are erroneously Grouped (see FIG. 8 (f)), and there is a possibility that the specific object specifying unit 178, which will be described later, may erroneously determine that the specific object is a preceding vehicle or the like.

  Therefore, in the present embodiment, focusing on the fact that the horizontal width of the clusters 224 based on the falling objects is short and the horizontal separation distance between the clusters 224 based on the falling objects is large, the clusters 224 constituting the group 226 are focused on. The grouping of the clusters 224 based on the fallout is canceled by determining the characteristics of Hereinafter, the cluster width deriving unit 172, the group width deriving unit 174, and the group releasing unit 176 that execute such processing will be described in detail.

  FIG. 9 is a diagram for explaining the processing of the cluster width deriving unit 172, the group width deriving unit 174, and the group releasing unit 176. The cluster width deriving unit 172 derives the horizontal width of the cluster 224 (hereinafter referred to as “cluster width CW”) for each cluster 224 based on the cluster three-dimensional position and the area occupied by the cluster 224 in the luminance image 126 (hereinafter referred to as “cluster width CW”). S320). Here, the cluster width deriving unit 172 derives the horizontal width of the circumscribed rectangle of the area occupied by the cluster 224 as the cluster width CW. Note that the circumscribed rectangle is a rectangle, and the horizontal direction of the outer edge is parallel to the horizontal direction of the image, and the vertical direction of the outer edge is parallel to the vertical direction of the image.

  The group width deriving unit 174 derives the horizontal width of the group 226 (hereinafter referred to as “group width GW”) based on the cluster three-dimensional position and the cluster width CW (S322). For example, as shown in FIG. 9, the group width deriving unit 174 in the horizontal direction from the left end of the leftmost cluster 224A to the right end of the rightmost cluster 224C among the clusters 224 constituting the group 226 Is the group width GW.

  The group cancellation unit 176 determines whether or not the group 226 satisfies all the cancellation conditions shown in the following (1) to (3) (S324). If all the cancellation conditions are satisfied (YES in S324), the group 226 Is released and returned to the cluster 224 (S326). On the other hand, when all the release conditions are not satisfied (NO in S324), the grouping of the group 226 is not released and the process proceeds to a specific object specifying process S328 described later.

  The cancellation condition (1) is that the total value of the cluster widths CW of the plurality of clusters 224 constituting the group 226 is less than a first threshold value determined based on the group width GW. The first threshold is a value obtained by multiplying the group width GW by a predetermined coefficient (less than 1). Here, the first threshold value is, for example, the cluster width CW of the plurality of clusters 224 constituting the group 226 when the clusters 224 generated when the preceding vehicle is imaged in an environment such as rain or snow are grouped. The total value is a value set so as to be equal to or greater than the first threshold value.

  The cluster 224 based on the falling object is clustered based on the image of the falling object that has fallen during the exposure time of the imaging apparatus 110. Since the falling object hardly moves in the horizontal direction during the exposure time of the imaging device 110, the cluster width CW of the cluster 224 based on the falling object is small. Therefore, when the clusters 224 based on the fallout are grouped, the ratio of the total value of the cluster widths CW of the plurality of clusters 224 in the group width GW is small.

  On the other hand, the fact that a plurality of clusters 224 are generated from a single three-dimensional object (for example, a preceding vehicle) whose horizontal width is larger than the falling object means that, for example, the falling object is superimposed on the three-dimensional object and captured. Is the case. That is, the total value of the cluster widths CW of the clusters 224 when the plurality of clusters 224 are generated from one solid object whose horizontal width is larger than the falling object is the cluster width of the plurality of clusters 224 based on the falling object. It is considered that it will not become smaller than the total value of CW. Therefore, when a plurality of clusters 224 generated from one solid object are grouped, the ratio of the total value of the cluster widths CW of the plurality of clusters 224 in the group width GW increases.

  Therefore, by determining whether or not the release condition (1) is satisfied, it is possible to determine whether the group 226 can be regarded as one solid object.

  The release condition (2) is that in the group 226, the cluster width CW of all the clusters 224 is less than the second threshold value. The second threshold value is set based on a horizontal width to be specified as at least a part of a predetermined specific object (for example, a preceding vehicle), for example, the horizontal width of a fallen object such as raindrops or snowdrops. This is the horizontal width of the license plate, which is larger.

  When a plurality of clusters 224 are generated from one solid object whose horizontal width is larger than the falling object, even if the cluster width CW of the cluster 224 has the smallest horizontal width among the plurality of clusters 224, It is unlikely that the cluster width CW of the cluster 224 based on the fallout will be smaller. In addition, the cluster width CW of the cluster 224 having the smallest horizontal width among the plurality of clusters 224 is likely to be at least larger than the horizontal width of the license plate.

  Therefore, by determining whether or not the release condition (2) is satisfied, it is determined whether all the clusters 224 constituting the group 226 have a horizontal width that should be specified as one solid object. be able to.

  The cancellation condition (3) is that there is a cluster 224 in which any of the horizontal separation distances between the two adjacent clusters 224 in the group 226 (hereinafter referred to as “separation distance RK”) is equal to or greater than the third threshold value. It is. The third threshold is set based on the separation distance that should be one solid object. Here, the third threshold value is, for example, a separation between two adjacent clusters 224 in the group 226 when a plurality of clusters 224 generated when the preceding vehicle is imaged in an environment such as rainfall or snowfall are grouped. This is a value set so that all of the distances RK are less than the third threshold value.

  In an environment such as rainfall or snowfall, it is unlikely that the separation distance between adjacent falling objects will be smaller than the horizontal width of the falling objects. Therefore, in the group 226 in which the clusters 224 based on the fallout object are grouped, the separation distance RK between two adjacent clusters 224 is greater than the horizontal width of the fallout (the cluster width CW of the cluster 224 based on the fallout object). growing. On the other hand, the separation distance RK between two adjacent clusters 224 when a plurality of clusters 224 are generated from one solid object whose horizontal width is larger than that of the falling object is the horizontal width ( The cluster width CW of the cluster 224 based on the fallout is as small as about.

  Therefore, by determining whether or not the release condition (3) is satisfied, it is possible to determine whether or not the cluster 224 based on the fallout is included in the clusters 224 constituting the group 226.

  When the two clusters 224 are grouped to generate the group 226, the release condition (1) and the release condition (3) can be regarded as substantially equal conditions. Specifically, the cancellation condition (1) is a condition for determining the ratio of the total value of the cluster widths CWa and CWb of the two clusters 224 to the group width GW, in other words, (CWa + CWb) / GW It is less than the first threshold. The release condition (3) is a condition for determining the separation distance RK between the two clusters 224. When the two clusters 224 are grouped, (CWa + CWb) / GW of the release condition (1) can be rephrased as (GW−RK) / GW. Therefore, the release condition (1) and the release condition (3) Can be regarded as substantially equal conditions.

  Therefore, when determining the group 226 in which the two clusters 224 are grouped, the group cancellation unit 176 may determine whether or not the cancellation condition (1) and the cancellation condition (2) are satisfied.

  In this case, the first threshold value may be determined based on the relative distance z of the plurality of clusters 224 in addition to the group width GW. More specifically, when the cluster 224 is generated from one solid object, the number of blocks constituting the cluster 224 increases when the relative distance z is small, and decreases when the relative distance z is large. Here, if the coefficient multiplied by the group width GW is a fixed value, the group 226 may be canceled even if the group 226 is generated from one solid object when the relative distance z is large. .

  A case will be described where the relative distance z from the own vehicle 1 to the falling object is equal, the relative distance z from the own vehicle 1 to the preceding vehicle is different, and the horizontal width of the falling object is equal. In this case, if the preceding vehicle is divided (divided) by the fallen object and is reflected in the luminance image 126, the cluster 224 based on the preceding vehicle having the large relative distance z is compared with the cluster 224 of the preceding vehicle having the small relative distance z. Thus, the cluster width CW is small and the separation distance RK is large. Here, when the coefficient multiplied by the group width GW is a fixed value, when the relative distance z is large, even when the group 226 is generated from one solid object, the cluster width CW is small and less than the first threshold value. Or the separation distance RK is large and exceeds the third threshold, and the group 226 may be released.

  Therefore, for the first threshold, a coefficient to be multiplied by the group width GW is changed according to the relative distance z. For example, when the relative distance z is large, a coefficient smaller than that when the relative distance z is small is multiplied. This makes it possible to avoid a situation where the group 226 is released due to the relative distance z.

  Returning to FIG. 3, the specific object specifying unit 178 sets the group 226 and the ungrouped cluster 224 as one solid object, and specifies which specific object the solid object corresponds to. In the present embodiment, the specific object specifying unit 178 determines the three-dimensional object as the specific object “vehicle” particularly when the three-dimensional object satisfies a predetermined condition corresponding to a predetermined vehicle (S328). . For example, when the three-dimensional object is located on the road, the specific object specifying unit 178 determines whether or not the size of the three-dimensional object as a whole corresponds to the size of the specific object “vehicle”. If it is determined that it corresponds to the size of “,” the three-dimensional object is identified as the specific object “vehicle”.

  Thus, the vehicle exterior environment recognition device 120 can extract a three-dimensional object from the distance image 128 as a specific object, for example, a vehicle (preceding vehicle), and use the information for various controls. For example, when an arbitrary three-dimensional object in the detection area 124 is identified as a vehicle, the identified vehicle (preceding vehicle) is tracked to detect a relative distance and a relative acceleration to avoid a collision with the preceding vehicle. Or the distance between the preceding vehicle and the preceding vehicle can be controlled to be kept at a safe distance.

  As described above, according to the outside environment recognition device 120 according to the present embodiment, it is possible to avoid a situation in which a fallen object such as raindrops or snowdrops is erroneously determined as a specific object such as a vehicle or a pedestrian. The specific accuracy can be improved.

(2nd Embodiment: Outside vehicle environment recognition apparatus 420)
In the first embodiment, a configuration has been described in which the cluster 224 is grouped into the group 226 and then the group 226 is released and returned to the cluster 224 when all the release conditions (1) to (3) are satisfied. . However, when the clusters 224 are grouped, the release condition may be used as a join condition to determine whether or not the join condition is satisfied. In the second embodiment, a configuration for determining a join condition when grouping the clusters 224 will be described.

  FIG. 10 is a functional block diagram showing a schematic function of the external environment recognition device 420 according to the second embodiment. FIG. 11 is a flowchart showing an external environment recognition process according to the second embodiment.

  As shown in FIG. 10, the vehicle exterior environment recognition device 420 includes an I / F unit 150, a data holding unit 152, and a central control unit 454.

  The central control unit 454 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM in which programs are stored, a RAM as a work area, and the like. 152 and the like are controlled. In the present embodiment, the central control unit 454 includes the image acquisition unit 160, the image processing unit 162, the position information deriving unit 164, the cluster generation unit 166, the cluster position deriving unit 168, the cluster width deriving unit 172, and the group generating unit 470. Also, it functions as the specific object specifying unit 178.

  Note that components having substantially the same functions as the components in the first embodiment are denoted by the same reference numerals and description thereof is omitted, and here, the group generation unit 470 having different functions will be mainly described. Also, processes that are substantially the same as the processes in the first embodiment are denoted by the same reference numerals and description thereof is omitted. Here, the join condition determination process S510 and the grouping process S512 will be described.

  First, the group generation unit 470 derives a group width GW when a plurality of clusters 224 whose cluster three-dimensional positions are within a second range that is larger than the first range are grouped. Then, the group generation unit 470 applies the release conditions (1) to (3) to the combination conditions (1) to (3) for the plurality of clusters 224 whose cluster three-dimensional positions are in the second range larger than the first range. It is used as 3) to determine whether any one of the coupling conditions (1) to (3) is satisfied (S510).

  If any one of the above join conditions (1) to (3) is satisfied (YES in S510), the group generation unit 470 groups a plurality of clusters 224 to generate a group 226 (S512).

  Further, when grouping the two clusters 224, the group generation unit 470 sets the combination condition (1) and the combination condition (3) in the same manner as the relationship between the release condition (1) and the release condition (2). Can be regarded as substantially equal conditions. For this reason, when the two clusters 224 are grouped, the group generation unit 470 may determine whether or not the above combination condition (1) or combination condition (2) is satisfied.

  As described above, the outside environment recognition apparatus 420 according to the present embodiment also avoids a situation in which a fallen object such as raindrops or snowdrops is erroneously determined as a specific object such as a vehicle or a pedestrian, and specifies a specific object. The accuracy can be improved.

  Also provided are a program that causes the computer to function as the vehicle environment recognition devices 120 and 420, and a computer-readable storage medium such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD that records the program. . Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the first embodiment described above, when the group cancellation unit 176 determines a group 226 composed of three or more clusters 224, it is determined whether or not all the cancellation conditions (1) to (3) are satisfied. The configuration for determining has been described. However, it is only necessary to determine whether or not at least the release condition (1) and the release condition (2) are satisfied.

  In the second embodiment described above, when the group generation unit 470 groups three or more clusters 224, the combination conditions (1) to (3) (release conditions (1) to (3)) are satisfied. The configuration for determining whether or not any of the above is satisfied has been described. However, it may be determined whether or not a plurality of coupling conditions are satisfied among the coupling conditions (1) to (3).

  In the above-described embodiment, when the group generation units 170 and 470 group the two clusters 224 to generate one group 226, the first threshold value is set to the plurality of clusters 224 in addition to the group width GW. The configuration determined based on the relative distance z has been described as an example. However, the first threshold value may be determined based only on the group width GW.

  It should be noted that each step of the vehicle environment recognition processing in the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an external environment recognition device that specifies which specific object corresponds to a three-dimensional object existing in a detection region.

120, 420 Outside environment recognition device 160 Image acquisition unit 164 Position information deriving unit 166 Cluster generating unit 168 Cluster position deriving unit 170, 470 Group generating unit 172 Cluster width deriving unit 174 Group width deriving unit 176 Group releasing unit 178 Specific object identifying unit

Claims (5)

An image acquisition unit for acquiring images;
A position information deriving unit that divides the acquired image into a plurality of blocks composed of one or a plurality of pixels and derives a three-dimensional position for each block;
A cluster generation unit that generates a cluster by clustering a plurality of blocks in which the three-dimensional position is within a predetermined first range;
Based on the three-dimensional position of the block derived by the position information deriving unit, a cluster position deriving unit that derives, for each cluster, a cluster three-dimensional position that is a three-dimensional position of the cluster;
A group generation unit that generates a group by grouping a plurality of clusters in which the cluster three-dimensional position is in a second range that is larger than the first range;
A cluster width deriving unit for deriving a horizontal width of the cluster for each cluster based on an occupation area of the cluster in the image;
A group width deriving unit for deriving a horizontal width of the group based on a horizontal width of the cluster;
The total value of the horizontal widths of the plurality of clusters constituting the group is less than a first threshold set based on the horizontal width of the group, and the plurality of clusters constituting the group When all the horizontal widths are less than a second threshold set based on the horizontal width to be specified as at least a part of the predetermined specific object, the group is ungrouped and the plural Ungrouping unit to return to the cluster of
A specific object specifying unit for specifying the three-dimensional object as the predetermined specific object when the group and the cluster that is not grouped are set as one three-dimensional object and the three-dimensional object satisfies a predetermined condition;
An outside environment recognition device comprising: When the group generation unit groups three or more clusters to generate one group,
The group cancellation unit is configured such that a total value of horizontal widths of the plurality of clusters constituting the group is less than the first threshold value, and a horizontal width of all of the plurality of clusters constituting the group is In the case of being less than the second threshold value, any one of the horizontal separation distances between two adjacent clusters in the group is set based on the separation distance to be the first three-dimensional object. The outside environment recognition device according to claim 1, wherein when it is equal to or greater than a third threshold value, the grouping is canceled and returned to the plurality of clusters. An image acquisition unit for acquiring images;
A position information deriving unit that divides the acquired image into a plurality of blocks composed of one or a plurality of pixels and derives a three-dimensional position for each block;
A cluster generation unit that generates a cluster by clustering a plurality of blocks in which the three-dimensional position is within a predetermined first range;
Based on the three-dimensional position of the block derived by the position information deriving unit, a cluster position deriving unit that derives, for each cluster, a cluster three-dimensional position that is a three-dimensional position of the cluster;
A cluster width deriving unit for deriving a horizontal width of the cluster for each cluster based on an occupation area of the cluster in the image;
The horizontal direction in which the total value of the horizontal widths of the plurality of clusters is greater than or equal to a first threshold value, or the horizontal widths of all of the plurality of clusters are to be specified as at least part of a predetermined specific object A group generation unit that generates a group by grouping the plurality of clusters when the second threshold is greater than or equal to a second threshold set based on the width of
A specific object specifying unit for specifying the three-dimensional object as the predetermined specific object when the group and the cluster that is not grouped are set as one three-dimensional object and the three-dimensional object satisfies a predetermined condition;
With
The vehicle exterior environment recognition apparatus, wherein the first threshold is set based on a horizontal width of the group derived according to a horizontal width of the cluster. The group generation unit
If there are more than two clusters in the second range,
The total value of the horizontal widths of the plurality of clusters is not equal to or greater than the first threshold value, and the horizontal widths of all the plurality of clusters are not equal to or greater than the second threshold value. If any one of the horizontal widths between two adjacent clusters is less than a third threshold set based on the separation distance to be the first three-dimensional object, the plurality of clusters are grouped. The outside environment recognition device according to claim 3 characterized by things. When the group generation unit generates two groups by grouping two clusters,
The external environment recognition device according to claim 1 or 3, wherein the first threshold value is set based on a relative distance between the plurality of clusters in addition to a horizontal width of the group.

Images