JP6014867B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that calculates a distance to a corresponding point by a corresponding point search method based on a stereo image, and detects an object present in the image based on distance image data obtained based on the result. .

Japanese Patent No. 4714104

  For example, driving assistance technologies for assisting the driver, such as collision avoidance such as detecting the approach of a preceding vehicle and other obstacles to brake the vehicle, and auto-cruising, are known. In such driving support technology, the distance to an object existing in front of the vehicle is detected, and vehicle control such as vehicle braking and throttle control is performed based on the result.

As a method for obtaining information on the distance to the imaged object, for example, a method using a corresponding point search method as described in Patent Document 1 is known.
Conventionally, distance information for each corresponding point calculated by the corresponding point search method is stored as a distance image in association with the coordinates of each corresponding point (coordinates in a two-dimensional space as an image plane). Is used to detect an object present in the image.

The outline of the conventional object detection process will be described with reference to FIG.
First, in the conventional object detection process, the distance image obtained as described above is expressed as a plurality of regions (hereinafter referred to as “vertical region VR”) by dividing lines along the vertical direction of the image as shown in FIG. 10A. ). Then, for each vertical region VR, a distance histogram representing the distance distribution in the depth direction is created from the distance data existing in the vertical region VR, and the distance of the position (corresponding point) at which the frequency is the maximum is determined as the vertical region VR. The representative distance of the object existing inside. FIG. 10B schematically represents the representative distance obtained by such processing for each vertical region VR by a black circle.

  Next, for each corresponding point with the maximum frequency for which the representative distance is obtained, pixel ranges that are considered to be the same object are grouped from the relationship such as the distance to each corresponding point and the direction. FIG. 10C schematically shows that the three preceding vehicles existing in the image shown in FIG. 10A are grouped as groups G1, G2, and G3, respectively.

In this way, each object existing in the captured image is detected including information on the distance to the object.
The object detection process as described above is performed on an object that exists above the road surface, but when a slope exists in front of the vehicle, the slope is also detected as an object that exists on the road surface. There is.
Considering this point, in the conventional method described in Patent Document 1, for each object detected by the object detection process, it is determined whether or not the object is a slope.

However, according to the above-described conventional method, in object detection processing, grouping is performed based on the relationship between distances and directions between corresponding points where the frequency of the distance histogram obtained for each vertical region VR is maximized. If these objects are close to each other, they may be grouped together.
Then, regarding the group in which hills and objects other than hills are collected in this way, when it is determined whether or not it is a hill, the density of distance data is relatively sparse in the hill part, while other than the hill Since the distance data of the object is dense, there is a possibility that it cannot be correctly determined as a slope.
In particular, when the slope is steep, the distance distribution mode in the depth direction is approximated by the slope and an object close to the slope, so that there is a high possibility that they will be grouped together in the object detection process. This increases the possibility of misidentification of slopes.

  As described above, if an erroneous determination is performed and an object that should originally be determined to be a slope cannot be correctly determined, the detected object is recognized as an obstacle and is braked, resulting in an erroneous vehicle control based on image recognition. Operation will occur.

  Accordingly, an object of the present invention is to improve the accuracy of determining whether or not the vehicle is a hill and prevent malfunction of vehicle control.

The image processing apparatus of the present invention detects a corresponding point between a pair of image data captured in stereo by pattern matching, and based on a result of calculating a coordinate shift between the detected corresponding points as a parallax, A distance image generation unit that generates distance image data representing the distance to the corresponding point on the image, and individual regions obtained by dividing the distance image data into strips by dividing lines along the vertical direction of the image For each vertical region, a slope index value representing the slope of the subject is calculated based on the distance distribution, and the subject is considered to be other than a slope based on the slope index value. A region selection unit that selects a vertical region, and an object detection processing unit that detects an object present in the image based on the distance image data, with the vertical region selected by the region selection unit as a detection target region. Than is.
According to the present invention, a slope is not discriminated for a slope and objects other than the slope that are grouped together.

  According to the present invention, it is possible to improve slope discrimination accuracy and prevent vehicle control malfunction.

It is the figure which showed typically the system configuration for performing vehicle control. It is the figure which extracted and showed the structure of the part which concerns on the object detection in the system configuration | structure shown in FIG. It is the figure which showed an example of the distance image. It is explanatory drawing about the distance distribution in the image vertical direction of a vertical area | region. It is explanatory drawing about the discrimination method of a slope. It is explanatory drawing of the hill discrimination | determination as re-discrimination. It is the flowchart which showed the procedure of the specific process which should be performed in order to implement | achieve area | region selection processing. It is explanatory drawing about the modification of the slope discrimination | determination based on a 1st slope index value. It is explanatory drawing about the modification of the slope discrimination | determination based on a 2nd slope index value. It is a figure for demonstrating the outline | summary of the conventional object detection process.

<1. Overall system configuration>
FIG. 1 schematically shows a system configuration that includes an image processing apparatus according to an embodiment of the present invention and performs vehicle control of a vehicle 1. This system includes an imaging device 2, an outside environment recognition device 3, a vehicle control device 4, a steering angle sensor 5, a vehicle speed sensor 6, an actuator 7, and a display 8 provided for the vehicle 1.
In the present system, the vehicle environment recognition device 3 recognizes an object existing in front of the vehicle based on the captured image obtained by the imaging device 2. In this system, the vehicle control device 4 avoids a collision with an object recognized by the vehicle exterior environment recognition device 3, and keeps a separation distance (inter-vehicle distance) from the object as the preceding vehicle at a safe distance. Execute control etc.

Specifically, the vehicle control device 4 includes information on the current traveling state of the vehicle 1 acquired through the steering angle sensor 5 that detects the steering angle and the vehicle speed sensor 6 that detects the traveling speed of the vehicle 1, and the outside environment recognition device. 3, the vehicle 1 is maintained at a safe distance from the preceding vehicle based on the information on the object existing in front of the vehicle 1 detected in 3, and when a collision with the object is assumed, the vehicle 1 is Control for automatically braking the actuator 7 is executed. Here, the actuator 7 comprehensively represents an actuator for controlling a brake, a throttle valve, a steering angle, and the like for braking the vehicle 1.
Further, when a collision with an object is assumed, the vehicle control device 4 performs control so that a warning display to that effect is displayed on the display 8 disposed in front of the driver.
The vehicle control device 4 can also be formed integrally with the outside environment recognition device 3.

  The image processing apparatus according to the present embodiment executes detection processing on an object existing in the image (present in front of the vehicle 1) based on the captured image obtained by the imaging apparatus 2, and in the above-described system configuration Is inherent in the outside environment recognition device 3.

<2. Configuration related to object detection>
FIG. 2 shows the internal configuration of the imaging device 2 shown in FIG. 1 and the configuration of the main part related to object detection in the internal configuration of the outside-vehicle environment recognition device 3. In the figure, the vehicle control device 4 shown in FIG. 1 is also shown.
The imaging apparatus 2 includes a first camera unit 20-1, a second camera unit 20-2, an A / D converter 21-1, an A / D converter 21-2, an image correction unit 22, and an interface unit (I / F section) 23 is provided.
Each of the first camera unit 20-1 and the second camera unit 20-2 includes a camera optical system and an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). A subject image is formed on the imaging surface of the imaging device by the optical system, and an electrical signal corresponding to the amount of received light is obtained in pixel units by the imaging device.

The first camera unit 20-1 and the second camera unit 20-2 are installed so as to enable distance measurement by a so-called stereo method. The first camera unit 20-1 and the second camera unit 20-2 in this example are arranged at a predetermined interval in the vehicle width direction in the vicinity of the upper part of the windshield of the vehicle 1. The optical axes of the first camera unit 20-1 and the second camera unit 20-2 are parallel.
In addition, it is assumed that the focal lengths of the first camera unit 20-1 and the second camera unit 20-2 are the same value, and the frame periods are synchronized. The frame rate is 60 fps, for example.

  The electrical signal obtained by the image sensor of the first camera unit 20-1 is sent to the A / D converter 21-1, and the electrical signal obtained by the image sensor of the second camera unit 20-2 is sent to the A / D converter 21. -2 and A / D conversion is performed respectively. As a result, a digital image signal (image data) representing a luminance value with a predetermined gradation in pixel units is obtained.

The image correction unit 22 includes image data based on an image captured by the first camera unit 20-1 obtained through the A / D converter 21-1 (hereinafter referred to as “first captured image data”), A Image data (hereinafter referred to as “second captured image data”) based on an image captured by the second camera unit 20-2 obtained via the / D converter 21-2 is input.
For example, the image correction unit 22 corrects a deviation caused by an error in the attachment positions of the first camera unit 20-1 and the second camera unit 20-2 for each of the first captured image data and the second captured image data. Use affine transformation or the like. The image correction unit 22 also corrects the luminance value including noise removal for each of the first captured image data and the second captured image data.

  The interface unit 23 is provided for exchanging various data with an external device. The first captured image data and the second captured image data corrected by the image correction unit 22 are transferred to the vehicle environment recognition apparatus 3 via the interface unit 23.

The outside environment recognition device 3 includes an interface unit 31, a bus 32, an image processing unit 33, and a memory unit 34. The interface unit 31, the image processing unit 33, and the memory unit 34 are respectively connected via the bus 32, and mutual data communication via the bus 32 is enabled.
The interface unit 31 is provided for exchanging various data with an external device. The first captured image data and the second captured image data transferred from the imaging device 2 side by the previous interface unit 23 are received by the interface unit 31. The first captured image data and the second captured image data received by the interface unit 31 in this way are held in the memory unit 34 via the bus 32 and are used for processing of the image processing unit 33.
In addition, the result of an object detection process (described later) performed by the image processing unit 33 is transferred to the vehicle control device 4 via the interface unit 31. As can be understood from the description of FIG. 1, the vehicle control device 4 executes the above-described vehicle control based on the object detection processing result thus transferred.

The image processing unit 33 is configured by a semiconductor integrated circuit including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) as a work area, and follows a program stored in the ROM. Perform various processes.
Particularly in the case of the present embodiment, the image processing unit 33 performs the following processing based on the first captured image data and the second captured image data held in the memory unit 34 via the interface unit 31.
First, a distance image generation process for generating a distance image based on the first captured image data and the second captured image data held in the memory unit 34 is executed. Next, an area selection process based on the distance image generated by the distance image generation process is executed, and then an object detection process for detecting an object existing in the image with respect to the distance image is executed using the result of the area selection process. .
In the case of this example, the distance image generation process, the area selection process, and the object detection process are realized by software processing by the CPU according to a program stored in the ROM. In FIG. 2, for the sake of convenience, it is assumed that there is hardware for executing these processes, which are respectively represented as a distance image generation processing unit 33A, a region selection processing unit 33B, and an object detection processing unit 33C.

<3. Distance image generation processing>
First, the distance image generation processing executed by the distance image generation processing unit 33A will be described. The distance image generation process detects corresponding points between a pair of image data captured in stereo by pattern matching, and based on the result of calculating the deviation of coordinates between the detected corresponding points as parallax, corresponding points in real space Is a process of generating distance image data in which the distance up to is represented on the image.

Below, the outline | summary is demonstrated about the method for calculating | requiring a parallax by the corresponding point search method.
In the following description, the case where the first captured image data is the reference image T and the second captured image data is the comparison image C is exemplified.
In the following description, the position of each pixel in the reference image T is represented by coordinates (i, j). The coordinates (i, j) represent the i and j coordinates of the pixel when the lower left corner of the reference image T is the origin, the horizontal direction (lateral direction) is the i coordinate axis, and the vertical direction (vertical direction) is the j coordinate axis. . For the comparison image C, the i-coordinate and the j-coordinate are similarly taken with the pixel previously associated with the origin of the reference image T as the origin.
In the following description, the luminance value for each pixel is represented by the luminance value p, the luminance value p of the pixel specified by the coordinate (i, j) in the reference image T is the luminance value p1ij, and the coordinate (i in the comparative image C is i , J) is expressed as a luminance value p2ij.

  First, in the distance image generation process, the reference image T is divided in units of pixel blocks composed of a predetermined plurality of pixels, for example, 4/4 pixels, and pattern matching processing with the comparison image C is performed in units of pixel blocks. A point corresponding to the reference image T and the comparison image C (a point where the same subject is considered to be captured) is detected as the corresponding point.

Specifically, the comparative image C is divided into horizontal lines of 4 pixels width extending in the horizontal direction, one pixel block of the reference image T is taken out, and the corresponding horizontal line of the comparative image C is taken as one pixel unit. The luminance values p1ij of the 16 pixels in the pixel block of the reference image T and the luminance values p2ij of the 16 pixels in the corresponding comparison image C are acquired while sequentially shifting in the horizontal direction (i direction). Then, the pixel blocks on the horizontal line that minimize the city block distance CB obtained by the following [Equation 1] in which the absolute values of the differences between the luminance value p1ij of the reference image T and the luminance value p2ij of the comparative image C are summed. Is specified as a pixel block (corresponding point) on the comparison image C having the luminance value characteristic closest to the pixel block of the reference image T.
CB = Σ | p1ij−p2ij | [Formula 1]

  The coordinate shift amount between the pixel block on the comparison image C specified in this way and the pixel block on the original reference image T is calculated, and the shift amount is obtained as the parallax dp. The value of the parallax dp is obtained for each detected corresponding point. In the distance image generation process, the parallax dp value obtained in this way is used as the position information of the corresponding point where the parallax dp is detected. Hold in association.

  The parallax dp determined as described above is the horizontal direction related to the mapping position of the same object in the reference image T and the comparison image C derived from the separation distance between the first camera unit 20-1 and the second camera unit 20-2. Represents the relative shift amount. Using such a value of the parallax dp, the distance from the center position of the first camera unit 20-1 and the second camera unit 20-2 to the subject as a corresponding point (distance in real space) is triangulated. Calculate based on the principle.

The calculation of the distance in the real space to the subject as the corresponding point can be performed as follows, for example.
First, as a premise, a point on the road surface located directly below the center position of the first camera unit 20-1 and the second camera unit 20-2 is used as the origin, and the vehicle width direction (horizontal direction) of the vehicle 1 is the X axis. A three-dimensional space is defined with the vehicle height direction as the Y axis and the vehicle length direction (distance direction) as the Z axis.
In the three-dimensional space, the coordinates (X, Y, Z) of the position of the subject as the corresponding point for which the parallax dp has been obtained are the coordinates on the image of the corresponding point for which the parallax dp has been obtained (ic , Jc), it is obtained by the following [Expression 2] to [Expression 4] using the calculated value of the parallax dp.
X = CD / 2 + Z · PW · (ic−IV) [Formula 2]
Y = CH + Z · PW · (jc−JV) (Formula 3)
Z = CD / (PW · (dp−DP)) (Formula 4)
However, CD is the interval between the first camera unit 20-1 and the second camera unit 20-2, PW is the viewing angle per pixel, and CH is the first camera unit 20-1 and the second camera unit 20-2. The attachment height, IV, and JV represent i-coordinate and j-coordinate on the image of the infinity point in the front of the vehicle 1, and DP represents the vanishing point parallax.
The distance to the subject as the corresponding point for which the parallax dp has been calculated (hereinafter referred to as “distance L”) is calculated using the values of the parallax dp and the values of CD, PW, and DP calculated by [Equation 4]. Can be calculated.

At this time, in the distance image generation process, a filtering process for the obtained parallax dp is performed for the purpose of improving the reliability of the parallax dp. For example, when a 4/4 pixel pixel block having only a feature image of a roadway is scanned on a horizontal line having a width of 4 pixels in the comparison image C, all the portions of the comparison image C where the roadway is imaged are correlated. Even if the corresponding pixel block is specified and the parallax dp is calculated, the reliability of the parallax dp is low. For this reason, such a parallax dp is invalidated in the filtering process, and 0 (invalid value) is assigned as the value of the parallax dp of the pixel block.
As a result of performing such filtering processing, the parallax dp is normally data having an effective value only for a so-called edge portion where the difference in luminance value p1ij is large between pixels adjacent in the horizontal direction of the reference image T. Along with this, the distance image obtained from the calculation result of the parallax dp is also data having an effective value only for the edge portion.

In the distance image generation process, for the target pixel block, the distance L is calculated from the calculated parallax dp value according to the previous [Expression 4], and the calculated distance L value is used as information on the position of the pixel block (coordinate ( i, j)) and hold it.
Information that associates the value of the distance L with the position information on the image in this way is expanded on a two-dimensional plane by the i-axis and the j-axis in the real space up to each corresponding point (each subject). The distance L is represented on the image. In this sense, the information in which the value of the distance L is associated with the position information on the image as described above is hereinafter referred to as “distance image”.

<4. Area selection processing>
[4-1. First discrimination process]
Next, the area selection process executed by the area selection processing unit 33B will be described.
FIG. 3A schematically shows a distance image when a slope exists in front of the vehicle 1 and an object other than a slope as the road sign I exists in the vicinity of the slope as an example of the acquired distance image. Represents.
In the region selection process, first, the distance image is divided into a plurality of regions (hereinafter referred to as “vertical region VR”) by dividing lines along the vertical direction of the image.
FIG. 3B shows an example in which the distance image shown in FIG. 3A is divided into a plurality of vertical regions VR. As can be seen from FIG. 3B, by dividing the distance image into a plurality of vertical regions VR, the hill and objects other than the hill can be included in different vertical regions VR. In the example of FIG. 3B, when the n-th vertical region VR from the left end of the screen is the vertical region VR_n, the road sign I is included in the vertical region VR_n and the vertical region VR_n + 1, and the slope is included in the vertical regions VR_n + 2 to VR_n + 8. An example is shown.

  In the area selection process, after the distance image is divided into a plurality of vertical areas VR as described above, a slope index value representing the slope of the subject is calculated for each of the vertical areas VR based on the distance distribution. Then, based on the slope index value, a vertical region VR in which the subject is regarded as other than a slope is selected.

FIG. 4 is an explanatory diagram of the distance distribution in the vertical direction of the image of the vertical region VR. FIG. 4A is a vertical region VR including the road sign I as the vertical region VR_n and the vertical region VR_n + 1 shown in FIG. 3B. 4B illustrates the distance distribution of the vertical region VR including the slope as the vertical regions VR_n + 2 to VR_n + 8 illustrated in FIG. 3B.
Comparing FIG. 4A and FIG. 4B, it can be confirmed that the manner of the distance distribution is completely different between the case where the subject is a slope and the case where the subject is an object other than a slope. Therefore, it can be seen that for each vertical region VR, it is possible to determine whether or not the subject is a slope based on the distance distribution.

Specifically, when the subject is other than a slope, a distance distribution densely arranged at a certain distance as shown in FIG. 4A is shown, and when the subject is a slope, the distance is higher toward the upper side of the image as shown in FIG. 4B. A distance distribution in which the value of L gradually increases is shown. It is possible to determine whether the subject is a hill or an object other than a hill based on the difference in the manner of distance distribution.
More specifically, if the same distance is dense, it can be determined that the subject is an object other than a hill, and if the same distance is less dense, it can be determined that the subject is not an object other than a hill. In view of this point, in the present embodiment, an index value representing the degree of denseness of the same distance is calculated as a slope index value (first slope index value), and based on the magnitude relationship between the slope index value and a predetermined threshold value. Then, it is determined whether or not the subject is an object other than a hill.

  Various specific methods for calculating the first slope index value representing the degree of denseness of the same distance as described above can be considered, but an example thereof will be described below.

FIG. 5A is a diagram for explaining a specific procedure of the slope discrimination method.
First, in the slope discrimination method in this case, the total number of data of the distance L included in the vertical region VR is J, the total number of sections when the total number of data J is evenly divided is K, and the total number of data for each section is J. When / K, the number of distances L is counted in order from the lower side of the image for each vertical region VR, and the representative distance value is obtained for each counted J / K distances L.
In FIG. 5A, the total number J of the distance L included in the target vertical region VR is 40, and each division when the total number J of data is equally divided every 12.5% is illustrated (FIG. 5A). Middle right). In this case, J / K = 5. That is, the boundary value of each section is a multiple of 5 (5-35).

In this method, as described above, the number of data of the distance L is counted in order from the lower side of the image, and until the count value reaches the boundary value of the section (that is, until it reaches a multiple of 5). ), The representative distance value for each category is obtained sequentially. The representative distance may be an average value or a peak value of the distance L of the section. Or you may use the value of the distance L acquired (counted) initially about the division.
Here, the average value of the distance L of the section is acquired as the representative distance value.
In this way, the number of data of the distance L is counted in order from the lower side of the image, and the representative distance for each section is sequentially acquired, and the number of the sections with the representative distance overlapping is determined as the slope index value (first slope index). Value). Then, it is determined whether or not the subject is other than a hill by determining whether or not the hill index value is equal to or greater than a predetermined threshold (“threshold th1”: first threshold). In this example, “3” is set as an example of the threshold th1.

FIG. 5B illustrates the value of the representative distance for each category when the subject is an object other than a slope, and FIG. 5C illustrates the value of the representative distance for each category when the subject is a slope. In these figures, it is assumed that the total number J = 40 and J / K = 5.
When the subject is an object other than a slope, as illustrated in FIG. 4A, the distance distribution is such that the distance is concentrated at a certain distance, and therefore, the representative distances overlap in a plurality of sections. FIG. 5B shows an example in which the representative distance values overlap in the 10-15 section, the 15-20 section, and the 20-25 section, and the representative distance values differ in the other sections. ing.
On the other hand, when the subject is a slope, as illustrated in FIG. 4B, the distance distribution is such that the value of the distance L gradually increases toward the upper side of the image. Therefore, the representative distances overlap in a plurality of sections. The possibility is very low, and the representative distances do not overlap in the three sections as in the case of an object other than the slope of FIG. 5B. In FIG. 5C, in the slope portion (here, the portion from the 5-10 section to the 25-30 section), the value of the representative distance of each section is about 0.2 m to 0.3 m. An example is shown.

In this example, as described above, the number of sections where the representative distances overlap is calculated as a slope index value, and it is determined whether or not the slope index value is equal to or greater than a threshold th1 (in this case, “3” in this case). To do. As a result, when the slope index value is equal to or greater than the threshold th1, a determination result that the subject is an object other than a slope is obtained.
Accordingly, when the subject is an object other than a hill as shown in FIG. 5B, it can be properly determined that the object is an object other than a hill. Further, when the subject is a slope as shown in FIG. 5C, it can be determined that the subject is not an object other than a slope.

[4-2. Second discrimination process]
Next, the second determination process will be described with reference to FIG.
Depending on the first discrimination processing as described above, an object whose distance value is entirely concentrated in a certain range, such as road sign I, can be reliably identified as an object other than a hill as an object other than a hill. However, as an object other than a hill, there is an object similar to a hill as in the carrier car shown in FIG. 6A, for example, so the following second determination process for determining an object similar to a hill is performed. The second determination process is a re-determination process that is performed by further obtaining a second slope index value for the vertical region VR that is determined to be other than a slope (is a slope) in the first determination process.

FIG. 6B shows a slope distance distribution (plotted by ◆) and a carrier car distance distribution (plotted by □) in comparison.
As shown in FIG. 6B, the carrier car distance distribution has a relatively large peak, whereas the slope distance distribution does not have such a large peak. This is because the slope is inclined almost uniformly within a certain range in the image, whereas the carrier car has a part of the inclined portion within a certain range of the screen. to cause. Further, the slope is continuously inclined with respect to the road surface, whereas the carrier car is caused by the rear end portion not being continuous with the road surface when traveling as a preceding vehicle.
Based on such a difference in distance distribution, it is possible to distinguish between a slope and a carrier car (an object other than a slope). Specifically, a value representing the smoothness of the distance distribution is calculated as the second slope index value for the vertical region VR that is regarded as not a slope (is a slope) in the previous first determination process. Based on the second slope index value, it is determined again whether or not the subject in the vertical region VR is other than a slope.

  Various specific methods for calculating the second slope index value are conceivable. In this example, the frequency of the peak of the distance distribution of the target vertical region VR is used as the second slope index value. That is, for the target vertical region VR, the distance L with the highest frequency is detected, and the detected peak frequency is set as the second slope index value.

  In this re-discrimination, it is determined whether or not the second slope index value as the frequency of the acquired distance distribution peak is greater than or equal to a predetermined threshold (“threshold th2”: second threshold). If it is determined that the second slope index value is equal to or greater than the threshold th2, the subject of the vertical region VR is assumed to be an object other than a slope, and the vertical region VR is set as a target region for object detection processing. select. On the other hand, if it is determined that the second slope index value is not equal to or greater than the threshold th2, the subject of the vertical area VR is not an object other than a slope (that is, a slope), and the distance data of the vertical area VR is deleted. By doing so, it is excluded from the target area of the object detection process.

  In this way, a slope and an object similar to a slope such as a carrier car can be separated from a slope and an object other than the slope by using the characteristic that the slope has a higher degree of smoothness of the distance distribution. .

<5. Object detection processing>
Next, the object detection process executed by the object detection processing unit 33C will be described.
In the object detection process, the vertical area VR in which the distance data is deleted because it is determined as a slope by the second determination process is excluded from the processing target.
Since the specific processing contents of the object detection processing are the same as those conventionally performed, only the main points will be described below.
First, in the object detection process, for example, a distance histogram representing a distance distribution in the depth direction is created from distance data existing in the vertical region VR, and the distance L of the position (corresponding point) at which the frequency is maximum is set in the vertical region VR. The representative distance of an object existing in Then, for each corresponding point with the maximum frequency for which the representative distance is obtained, pixel ranges that are regarded as the same object are grouped from the relationship such as the distance and direction to each corresponding point adjacent to each other, and each existing in the image Specify the range of the object. Thereby, an object existing in the image is detected including information on the distance L to the object.

<6. Region selection process>
Next, the procedure of the area selection process described above will be described with reference to the flowchart of FIG.
In FIG. 7, the CPU (the CPU of the image processing unit 33, the same below) divides the distance image into strips of N (N is a natural number of 2 or more) vertical regions VR by dividing lines along the image vertical direction. (Step S101).
Then, the value of the vertical area identification number n is set to “1” (step S102). The value of the vertical area identification number n is a value managed by the CPU in order to identify the vertical area VR to be processed among the N vertical areas VR obtained by shortening the strip in step S101.

Next, for the nth vertical region VR, the total number J of data of the distance L is equally divided every X% (step S103). Here, let K be the number of sections obtained by the equal section processing.
Further, the CPU sets a threshold thc = J / K (step S104). The threshold value thc is a threshold value for detecting a break between sections.

  Next, the category identification number k = 0 is set (step S105). The value of the category identification number k is a value managed by the CPU to identify the category to be processed among the K categories obtained in the process of step S103.

Subsequently, counting of the number of data of the distance L is started in order from the lower side of the image (step S106), and the process waits until the count value becomes equal to or greater than the threshold thc (step S107).
If the count value is equal to or greater than the threshold thc, the representative distance of the kth section is acquired (step S108). In the case of this example, the average value of the J / K distances L counted as the kth section is calculated and acquired as the representative distance value of the kth section.
Further, the value of the classification identification number k is incremented (k ← k + 1) (step S109), and J / K is added to the value of the threshold thc (thc ← thc + J / K) (step S110). Thereby, preparation for acquisition of the representative distance of the next section is made.

  Next, it is determined whether or not the number of sections where the representative distances overlap is greater than or equal to the threshold th1 (S111). In the case of this example, the number of sections with the same representative distance value overlapped is obtained as the first slope index value, and the first slope index value is equal to or greater than the threshold th1 (“3” in the above example). It is determined whether or not.

If it is determined in step S111 that the number of sections with overlapping representative distances is greater than or equal to the threshold th1, the process proceeds to step S112, and the nth vertical region VR is selected as a target region for object detection. Subsequently, in step S113, it is determined whether or not the value of the vertical area identification number n is equal to or greater than the total number N of vertical areas VR.
When it is determined in step S113 that the value of the vertical region identification number n is N or more, the processing (slope determination) for all the vertical regions VR in the target frame image (distance image) is completed. The area selection process ends.

  On the other hand, if it is determined in step S113 that the value of the processing target vertical area identification number n is not equal to or greater than N, the process proceeds to step S114, and the value of the vertical area identification number n is incremented (n ← n + 1). Returning to step S103, the sorting process for the nth vertical region VR is executed. Thereby, the slope discrimination process for the next vertical region VR is started.

  On the other hand, if it is determined in step S111 described above that the number of sections with overlapping representative distances is not equal to or greater than the threshold th1, the process proceeds to step S115 to determine whether the count value has reached the total number of data J. . That is, it is determined whether or not the data of all the distances L in the target n-th vertical region VR has been counted, and the determination processing of step S111 based on the acquisition of the representative distances of all the sections and the representative distances is completed. It is equivalent to doing.

If it is determined in step S115 that the count value has not reached the total number of data J, the process returns to step S107 and waits until the count value reaches the threshold value thc.
On the other hand, if it is determined in step S115 that the count value has reached the total number J of data, that is, if it is determined that the number of sections with overlapping representative distances is not equal to or greater than the threshold th1, the nth vertical region VR Both the case where the subject is a slope and the case where the subject is an object other than a slope similar to a slope like a carrier car can be considered. For this reason, the re-discrimination process based on the second slope index value described above is executed in the following steps S116 to S118.

First, in step S116, the peak of the distance distribution of the nth vertical region VR is detected, and in the subsequent step S117, it is determined whether or not the peak frequency (second slope index value) is equal to or greater than a threshold th2.
If it is determined in step S117 that the peak frequency is greater than or equal to the threshold th2, the process proceeds to step S118, and after selecting the nth vertical region VR as the object detection target range, the process proceeds to the previous step S113. . That is, when the frequency of the peak is equal to or greater than the threshold th2, the subject in the nth vertical region VR can be regarded as an object other than a hill such as a carrier car, and thus the process proceeds to step S113, and n It is determined whether or not there is a remaining vertical region VR based on ≧ N. If it exists, the processing for the next vertical region VR is started (after S114), and if it does not exist, the region selection processing is ended.

On the other hand, if it is determined in step S117 that the frequency of the peak is not equal to or greater than the threshold th2, the subject in the nth vertical region VR can be regarded as a slope, so the distance for the nth vertical region VR is By deleting the data, the object detection target area is not selected, and the process proceeds to step S113 without proceeding to step S118.
As described above, if it is determined in step S113 that n ≧ N is not satisfied, processing for the next vertical region VR is started (after S114), and if it is determined that n ≧ N, region selection processing is performed. finish.

<7. Summary>
As described above, in the present embodiment, the distance image is divided into a plurality of vertical regions VR in a strip shape, and whether the subject is an object other than a hill based on the slope index value for each of the divided vertical regions VR. The object detection process is executed using the vertical region VR determined to be an object other than a slope by the determination as a detection target region.
Therefore, it is possible to more accurately determine whether the object is a hill or an object other than a hill without the determination of a hill for a group in which hills and objects other than the hill are grouped together as in the past. This improves the accuracy of slope discrimination. As a result, malfunction of vehicle control can be prevented.

Further, for each vertical region VR, an index value indicating the degree to which the same distance is dense in the image vertical direction is calculated as the first slope index value, and based on the magnitude relationship between the first slope index value and the predetermined threshold th1. Whether or not the subject is an object other than a hill is determined.
Therefore, it is possible to appropriately separate cases where the subject can be reliably regarded as an object other than a slope.

Further, for each vertical region VR, the number of data of the distance L is counted in order from the lower side of the image, the representative distance value is obtained for each of the counted J / K distances L, and the representative distance values overlap. The number is calculated as the first slope index value, and it is determined whether or not the first slope index value is equal to or greater than the first threshold th1.
According to this, it is possible to complete the determination as to whether or not the subject is an object other than a hill before referring to all the data in the target vertical region VR, and accordingly, the processing burden related to the determination is increased. Can be reduced and the processing time can be shortened.

Furthermore, a second slope index value based on the distance distribution is calculated for the vertical region VR determined that the subject is not an object other than a slope, and the subject in the vertical region VR is calculated based on the second slope index value. It is redetermined whether it is other than a slope.
In this way, the vertical region VR that can be reliably regarded as an object other than a hill by the first determination process is divided, and then whether or not the subject is a hill in the vertical region VR that is left as the second determination process. By re-determination, it is possible to reduce the processing load related to the discrimination.
That is, when it is determined whether the subject is a hill or an object other than a hill by a single determination process (including separation from a carrier car or the like), a process to be executed for each vertical region VR in the image However, if the discrimination is performed in two stages as described above, the processing burden for the first stage discrimination can be reduced, and the overall processing burden can be reduced. In addition, the processing time can be shortened.

In addition, a value representing the degree of smoothness of the distance distribution is calculated as the second slope index value, and it is determined again whether or not the subject is an object other than a slope.
According to this, it is possible to determine whether the subject is a slope or an object other than a slope similar to a slope like a carrier car.

Further, the frequency of the peak of the distance distribution is calculated as the second slope index value, and it is determined again whether or not the subject is other than the slope based on the comparison result between the second slope index value and the predetermined threshold th2. It is carried out.
According to this, it is possible to reliably determine whether the subject is a slope or an object other than a slope similar to a slope like a carrier car.

<8. Modification>
Below, the modification of the technique of hill discrimination is demonstrated.
For example, for slope determination based on the first slope index value, a frequency distribution table (may be a distance histogram) is created for the distance L existing in the target vertical region VR, and the first slope is obtained from the result. It is also possible to obtain an index value and determine whether or not the subject is an object other than a slope.

Alternatively, as a hill discrimination method based on the first hill index value, the following method based on the data density of the distance L can be adopted.
Specifically, first, on the condition that the absolute value of the difference between the maximum value (L_max) and the minimum value (L_min) of the distance L existing in the target vertical region VR is larger than the threshold value thl. Impose (| L_max−L_min |> thl). The vertical region VR that does not satisfy this condition is determined not to be selected as a target region for object detection processing because it is determined that the subject is an object other than a slope.
On the other hand, if the above condition is satisfied, the number of data of the distance L is counted for each predetermined distance section, for example, the distance section divided by a distance of 0.5 m, and the number of data is set to a predetermined threshold (“threshold thd”). (Notation) The above-described distance section is detected as a distance section in which data of the distance L exists.
Then, the ratio (%) of the number of detected distance sections to the total number of distance sections is obtained as a first slope index value, and the first slope index value is a predetermined threshold (“threshold th1 ′”). If the result is positive, the subject is not an object other than a hill. If the result is negative, the subject is an object other than a hill. And For example, it is assumed that the threshold th1 ′ is set to 75%.

FIG. 8A schematically shows an example of the distribution of data (distance L) for each distance section when the subject is a slope, and FIG. 8B shows the data for each distance section when the subject is an object other than a slope. A distribution example is schematically shown.
According to the above method, in the case of FIG. 8A, the first slope index value is calculated as 5/5 and 100%, and the threshold th1 ′ = 75% or more. Can be obtained.
On the other hand, in the case of FIG. 8B, the first slope index value is calculated as 2/5 and 40%, and is less than the threshold th1 ′, so that it is possible to appropriately obtain the discrimination result that the object is other than the slope. it can.

Further, as a method of re-discrimination based on the second slope index value, for example, a method as shown in FIG. 9 can be adopted in addition to the method using the frequency of the distance distribution peak as the index value as described above. .
9, FIG. 9A shows an example of a distance distribution for a vertical region VR whose subject is a slope, and FIG. 9B shows an example of a distance distribution for a vertical region VR whose subject is an object other than a slope similar to a slope. ing.
First, based on the data of the distance L of the target vertical region VR, a representative distance value (for example, an average value) for each predetermined distance section such as 0.5 m unit is obtained. The black circles shown in FIGS. 9A and 9B represent the representative distance for each distance section.
Next, after calculating the approximate straight line ap for the representative distance for each distance section by, for example, the least square method, the total value of the divergence amount of the representative distance for each distance section with respect to the approximate straight line ap is obtained as the second slope index value. Asking. It is determined whether or not the second slope index value is equal to or greater than a predetermined threshold (denoted as “threshold th2 ′”), and if the determination gives a positive result, the subject is an object other than a slope, If a negative result is obtained, it is assumed that the subject is not an object other than a slope.

As described with reference to FIG. 6B, when the subject is an object other than a slope similar to a slope, a relatively large peak occurs in the distance distribution. Therefore, the total value of the divergence amounts of the representative distances from the approximate straight line ap is the subject. Compared to the case where is a slope, it becomes large. In FIG. 9B, the double arrow represents the representative distance where the deviation from the approximate straight line ap is large.
Therefore, even by the above method, it is possible to appropriately separate a slope and an object other than a slope similar to the slope.

  DESCRIPTION OF SYMBOLS 1 * Vehicle, 2 * Imaging device, 3 * Outside environment recognition apparatus, 31 * Interface (I / F) part, 32 * Bus, 33 * Image processing part, 33A * Distance image generation processing part, 33B * Area selection processing part 33C · Object detection processing unit

Claims (5)

A corresponding point between a pair of image data captured in stereo is detected by pattern matching, and a distance to the corresponding point in real space is calculated based on a result of calculating a deviation of coordinates between the detected corresponding points as a parallax. A distance image generator for generating distance image data represented on the image;
When each area obtained by dividing the distance image data into strips by dividing lines along the image vertical direction is defined as a vertical area, the slope of the subject is determined for each vertical area based on the distance distribution. A region selection unit that calculates a slope index value that represents, and based on the slope index value, selects the vertical region in which the subject is considered to be other than a slope;
An image processing apparatus comprising: an object detection processing unit configured to detect an object existing in an image based on the distance image data using the vertical region selected by the region selection unit as a detection target region. The region selection unit
For each of the vertical regions, an index value representing the degree to which the same distance is dense in the image vertical direction is calculated as a first slope index value,
2. For each of the vertical regions, it is determined whether or not the subject is a vertical region that is considered to be other than a hill based on a magnitude relationship between the first slope index value and a predetermined first threshold value. An image processing apparatus according to 1. The region selection unit
When the total number of distance data included in the vertical region is J, the total number of sections when the total number J is equally divided is K, and the number of data for each section is J / K,
For each of the vertical regions, distance data is counted in order from the lower side of the image, and a representative distance value is obtained for each of the counted J / K distance data, and the number of sections in which the representative distance values are overlapped is determined. The image processing apparatus according to claim 2, wherein the image processing apparatus calculates a first slope index value and determines whether the first slope index value is equal to or greater than the first threshold value. The region selection unit
As a result of the determination based on the first slope index value and the first threshold value, a second slope index value based on the distance distribution is further calculated for the vertical region regarded as not being a slope, and the second slope index value is calculated. The image processing apparatus according to claim 2, wherein redetermining whether the subject in the vertical region is other than a slope is performed based on the slope index value of 2. The region selection unit
The image processing apparatus according to claim 4, wherein a value representing a smoothness degree of the distance distribution of the vertical region that is regarded as other than the slope is calculated as the second slope index value.

Images