JP2001116513A - Distance image calculating device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate the distribution of the distance upto a body more closely. SOLUTION: This is a correlative operation unit which uses stereographic matching. Images of a body are picked up by a couple of image pickup devices 10 and 11. A measurement point is set in one piece of image data and correlative operation by a city block distance is carried out with the other image in an (x) direction to obtain a correlation curve. A parallax candidate setting part 16 sets a parallax candidate to minimum values of the correlation curve. A matching degree evaluation part 17 decides the reliability of the parallax candidate from the minimum values and the vertical angles of minimum parts. The candidate is regarded as a high-reliability measurement point when the reliability is larger than a specific level or as a low-reliability measurement point when smaller. A parallax correction part 18 selects the parallax which is closest to parallax candidates of the low-reliability measurement point out of those candidates when the low-reliability measurement succeeds to the high- reliability measurement point in two directions. Consequently, the number of high-reliability measurement points constituting the body increases and the distribution of the distance from the body is closely found from the parallax.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an arithmetic unit for photographing an object with a camera and calculating a distance distribution to the object based on the image data. For example, a correlation operation device that calculates a distance distribution to an object by stereo matching of a pair of cameras. Considering the continuity of parallax at the measurement point,
To calculate the disparity of a measurement point with low reliability, based on the disparity of surrounding measurement points with high reliability, by selecting an appropriate disparity from a large number of disparity candidates at that measurement point,
The present invention relates to a calculation device that obtains a distance distribution to an object. INDUSTRIAL APPLICABILITY The present invention can be applied to a range image calculation device mounted on a vehicle and for obtaining a distance distribution to surrounding objects such as the front and rear of the vehicle at a high density.

[0002]

2. Description of the Related Art Conventionally, there is an object recognition apparatus using image processing. For example, a vehicle exterior monitoring device disclosed in Japanese Patent Application Laid-Open No. 5-265747 is one of them. that is,
This is a system in which two cameras are mounted on a vehicle, the object is imaged in real time, and the two screen images are subjected to stereo matching processing to calculate and recognize the position, size, and distance of the object. If two cameras arranged at a predetermined distance capture an image of an object in front, the object is shifted in the left and right images in the X direction. This is because parallax occurs between the two cameras. If the object is near the camera, the parallax increases, and if the object is far away, the parallax decreases. The stereo matching process is a method of detecting a distance from an object using this principle.

FIG. 12 shows a flowchart of a conventional distance distribution measurement. In step 101, an image of an object is input from each of the two cameras, and in step 102, it is converted into image data digitized by A / D conversion. The digitized image data is, for example, an image that has been grayed in 0 to 255 steps. Next, in step 103, for example, 4 × 4 small blocks are cut out from the left and right images, and a correlation operation (matching processing) is performed to obtain parallax between the small blocks. The parallax is a shift amount between small blocks.
The distance from the camera to the small block, which is a small area constituting the object, is obtained from the shift amount. If the separation distance of the camera is B, the focal length is f, and the shift amount is ΔX, the distance z to the object is expressed by the following equation.

[Equation 1] z = f · B / △ X (1)

However, when a correlation curve based on the city block distance is actually searched, a plurality of minimum values, that is, a plurality of shift amounts ΔX are generated. In other words, there are a plurality of correspondence candidates. In order to examine this, the process proceeds to step 104 to calculate the maximum value and the minimum value of the minimum value, and it is determined in step S105 whether the minimum value and the maximum value satisfy predetermined conditions. If so, the amount of deviation up to the minimum value has been determined as the final amount of deviation ΔX.
By repeating this, the distance distribution of the entire object has been obtained.

[0005]

However, in the above method, when determining the minimum minimum value, one threshold is set,
Only the minimum value is determined based on the magnitude relationship with the threshold value.
In this determination method, a plurality of matching points exist when the predetermined threshold is high, and conversely, no matching points exist when the predetermined threshold is low. In other words, there is a disadvantage that if the threshold value is high, erroneous correspondence occurs, and an accurate distance distribution cannot be obtained. Conversely, when the threshold value is low, the number of corresponding points decreases, and the distance becomes accurate, but there is a point where the distance cannot be obtained, and there is a disadvantage that the object surface shape cannot be recognized.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and has as its object to correct a parallax obtained based on the continuity of the parallax with respect to an object and to provide a plurality of distances to an imaged object. That is, the distance distribution to the object is accurately calculated.

[0007]

According to a first aspect of the present invention, there is provided a distance image calculating apparatus for picking up an object using a plurality of image pickup devices and calculating a distance to the object from a plurality of obtained image data. And a stereo matching unit that matches one image data with the other image data to obtain a plurality of parallax candidates for each measurement point on the image, and calculates a matching degree of the plurality of parallax candidates, A collation degree evaluation means for giving an evaluation to the candidate, and when the collation degree is higher than a predetermined value, a parallax candidate having the highest collation degree is selected as the parallax of the measurement point, and when the collation degree is lower than the predetermined value, the measurement point is selected. And a parallax correcting unit that corrects the parallax of the measurement point based on the matching degree of the surrounding measurement point and the determined parallax of the surrounding measurement point.

Further, in the distance image calculating apparatus according to the present invention, the stereo matching means sets a small area image including the measurement point on the image data, and moves on the small area image and the other image data. The collation is performed by a correlation operation with the small area image. That is, it is characterized in that a region-based correlation operation is performed.

Further, in the distance image calculating device according to the third aspect, the matching degree is a value corresponding to the city block distance obtained by the stereo matching means. Further, in the distance image calculating apparatus according to the fourth aspect, the matching degree is constituted by the minimum value related value at the minimum point of the city block distance obtained by the stereo matching means, the minimum point, and points on both sides thereof. It is a value obtained from the apex angle related value.

Further, in the distance image calculating apparatus according to the present invention, the correction is performed by selecting a parallax candidate having a value closest to the parallax determined at the surrounding measurement point from the parallax candidates at the measurement point. . In the distance image calculating apparatus according to the sixth aspect, the measurement point is one point on the edge.

[0011]

According to the distance image calculation device of the present invention, a plurality of imaging devices image an object to obtain a plurality of image data. Then, one image data is collated with the other image data, and a parallax candidate is obtained for each measurement point on the image. For example, 1 including a plurality of pseudo correlations by a correlation operation
The above minimum value is obtained. The amount of displacement giving the minimum value is the parallax, and the amount of displacement giving the plurality of minimum values is a parallax candidate. The plurality of parallax candidates are set as measurement points.

Next, the collation evaluation unit evaluates each parallax candidate. For example, a value obtained by multiplying the city block distance of the correlation curve obtained by the stereo matching by −1 is set as the matching degree, the magnitude of the matching degree is calculated, and the ranking is set to them. That is, the collation degree evaluation means uses the 1 set at the measurement point.
One collation degree, rank, and the like are given to one parallax candidate. Note that the city block distance is defined as a value based on the sum of absolute differences, and the matching degree is defined such that the smaller the city block distance, the greater the matching degree. As long as the degree of collation represents the magnitude of the correlation, an arbitrary definition such as a cross-correlation function other than the city block distance can be used.
Further, the matching degree may be defined in consideration of other factors in addition to the city block distance, the cross-correlation function, and the like.

The parallax correcting means selects the parallax candidate having the highest matching degree as the parallax when the matching degree by the matching degree evaluating means is higher than a predetermined value. On the other hand, when the matching degree is lower than the predetermined value, the parallax is corrected and determined from the information of the surrounding measurement points of the measurement point. This is because the constituent points of the object are continuous and it is estimated that the parallax has continuity. If the degree of collation of the surrounding measurement point is high, the parallax at the surrounding measurement point is determined to have high reliability, and correction is performed based on the parallax determined for the surrounding measurement point. For example, from among a large number of parallax candidates at the measurement point, the parallax closest to the first-order parallax determined at the surrounding measurement points is selected. Alternatively, the disparity closest to the average value of the first-order disparities determined at the surrounding measurement points is selected. Or, if there is no parallax close to a predetermined value or less in the first-order parallax or the average parallax of the surrounding measurement points, a representative parallax of the surrounding measurement points is used, or the first-ranking parallax of the surrounding measurement points is used. The average of the parallaxes or a corrected parallax by a function using the parallax as a parameter may be obtained.

As described above, when the matching degree of the parallax candidate at the measurement point is low, the correction is performed based on the parallax of the surrounding measurement points based on the continuity of the parallax. In other words, the correction increases the reliability of the parallax of the measurement point. Therefore, if the parallax candidate is converted into a distance to a distance object according to a predetermined conversion formula,
A highly reliable distance distribution is required at high density. Note that an area to be compared in the two images is arbitrary. Further, the measurement point means one or more pixels on the image data.

Further, according to the distance image calculating apparatus of the present invention, the stereo matching means sets a small area image including the measurement point on the image data, and sets the small area image and the other image data on the small area image. The collation is performed by a correlation operation with the moving small area image. That is, the matching area is limited. Since the area-based correlation calculation is performed, the calculation time can be reduced.

According to a third aspect of the present invention, the matching degree is a value corresponding to the city block distance obtained by the stereo matching means. This defines that the matching degree is determined according to the city block distance. Assuming that the city block distance is a value obtained by the correlation operation, the smaller the value is, the more similar the areas are, and thus, it can be said that the reliability of the obtained parallax is high. As a result, it is possible to obtain a matching degree that provides a reference that can be adopted as parallax.

Further, according to the distance image calculating apparatus of the present invention, the matching degree is determined by the minimum value related value of the minimum point of the city block distance obtained by the stereo matching means and the minimum point and the points on both sides. This is a value obtained from the constituted vertex angle related value. Here, the minimum value-related value is, for example, the minimum value itself, the difference between the minimum value and a predetermined threshold value, or the difference between the minimum minimum value and the second minimum value. The vertex angle-related value is the vertex angle, the inclination of a side forming the vertex angle, or the area of a polygon formed including the vertex angle, and represents the steepness of the minimum point. In claim 4, the matching degree is defined by a function including at least one of the above parameters and the like.

For example, the sum of the difference between a predetermined threshold value and the minimum value and the reciprocal of the apex angle is used as one factor for determining the matching degree.
If the minimum value is significantly lower than a predetermined threshold value, a value proportional to the minimum value is added to the collation degree. If the apex angle is more acute, a larger numerical value is added. This is because when the two image data match, the minimum value becomes the minimum value, and the valley shape including the minimum point becomes steep. If the collation degree, which is the total numerical value, is greater than a predetermined value, the measurement point is determined to be, for example, a high reliability measurement point, and conversely, if it is smaller, the determination point is a low reliability measurement point. In this way, a matching degree that gives a degree of matching between two image data can be obtained. In this way, since not only the minimum value of the city block distance but also other factors are taken into account, a more accurate distance distribution is required to improve the reliability of determining the parallax.

According to the distance image calculating apparatus of the present invention, the correction is performed by selecting a parallax candidate having a value closest to the parallax determined at the surrounding measurement point from the parallax candidates at the measurement point. Since the selection is a parallax candidate, the correction processing can be performed more quickly. In addition, the parallax candidate indicates a more accurate distance because it is an actual measurement. Therefore, the distance image calculation method is to calculate the distance distribution to the object faster and more accurately.

According to the distance image calculating apparatus of the present invention, the image data is an edge image, and is a measurement point and one point on the edge. The edge image is obtained by, for example, Sobel operation processing, and the contours, edges, and boundaries between the regions forming the object are emphasized. The present invention is to collate the edge image. When performing a region-based correlation operation, a small region image is set on an edge image. Since the correlation calculation is performed on the edge image, the calculation amount of the stereo matching is significantly reduced. Thereby, the distance distribution to the contour of the object can be obtained at higher speed.

[0021]

(Embodiment) FIG. 1 shows an embodiment to which a range image calculating apparatus according to the present invention is applied. The figure is a system configuration diagram. This system includes cameras 10 and 11 arranged at a predetermined distance, and frame memories 12 and 13 for high-speed A / D conversion of video signals transmitted from the cameras 10 and 11 and storing the signals. Frame memory 12,
A small area image setting unit 15 is provided for setting a small area image including a measurement point in the image data of the object captured in 13. A parallax candidate for correlating with the other image data with respect to the small area image, obtaining a minimum point from a correlation curve obtained by a correlation operation, and setting a parallax candidate obtained from a displacement amount of the small area image giving the minimum point A setting unit 16 is provided. A collation evaluation unit 17 that is a collation evaluation unit that calculates and evaluates the collation of the parallax candidate, and a parallax correction unit that corrects the parallax of the measurement point based on the result of the collation evaluation unit 17. A correction unit 18 is provided.
The small area image setting unit 15, the parallax candidate setting unit 16,
The collation degree evaluation unit 17 and the parallax correction unit 18 include a C (not shown)
Not shown ROM with PU and arithmetic processing program
And a RAM (not shown) which is a work area memory for executing the program. The above elements are connected by a system bus 19 composed of various signal lines,
Is transmitted and received by various programs written in the ROM.

Next, the operation of the distance image calculation device of the present embodiment will be described with reference to FIGS. By this operation, one or two or more parallax candidates having a distance (parallax) to each point of the object at each point are calculated. From the parallax candidates, an appropriate candidate is selected based on the continuity of the parallax, or is corrected and obtained. The target object is not particularly limited,
In this embodiment, it is an obstacle or a preceding vehicle. The operation is started by a switch (not shown), and is executed in step S10 of FIG.
It is executed from 0.

In step S100, object images from cameras 10 and 11, which are imaging devices, are stored in frame memories 12 and 1.
3 The frame memories 12 and 13 are composed of a plurality of RAMs, and one unit (1
Is a RAM having a total number of 512 × 512, for example, corresponding to each pixel of the CCD image sensor. An arbitrary pixel is represented by coordinates (x, y) and intensity I. The intensity I is, for example, 0.
Up to 255 levels are obtained. Hereinafter, the gradation image is referred to as image data. At this time, the upper left corner of the screen is x
The origin of the y coordinate system, the x axis in the horizontal direction and y in the vertical direction
The axis is set.

Next, the process proceeds to step S110. Step S110 is extraction of an edge image. Step S12
As preprocessing for determining a small area image of 0, an edge image is extracted from one of the image data. More specifically, it can be obtained by performing a Sobel operation or the like, which is a matrix weighting operation, on an image taken into any of the frame memories 12 and 13. The obtained images are differential images each representing a luminance gradient. Since the differential image emphasizes the change point of the intensity, the outline of the preceding vehicle, the boundary of the component, and the like are thinned as edges.

FIG. 3 shows the thinned edge image of the preceding vehicle.
Shown in The edge images of the preceding vehicle are classified into vertical edges, horizontal edges, and diagonal edges. For example, vertical edges V _s and V _{e having} edge constituent points in the y direction and horizontal edges H ₁ having edge constituent points in the x direction are obtained. The portions surrounded by broken lines are vertical edges V _s , V
_e is determined, and its extension is determined to be a curve or diagonal line. The pixels forming the vertical edges V _s and V _e are continuous in the vertical direction, and the pixels forming the horizontal edge H ₁ are continuous in the horizontal direction. In addition, the vertical edges V _s and V _e and the horizontal edge H
₁ is continuous at the intersection P _s and P _e. Due to the continuity, parallax correction described later is performed. Next, step S
The process moves to 120 (FIG. 2).

In step S120, a small area image used for calculation is set to minimize the amount of calculation for correlation calculation. For this, first, a plurality of measurement points P _k to be measured on the edge of the edge image and a plurality of small areas A _k composed of N pixels vertically and M pixels horizontally centered on the measurement points P _k are set (FIG. 3). The measurement point P _k, 'a small-area image A _k' small region image A measuring points corresponding to _k P _k a, step S
It is set on a plurality of image data (shaded images) obtained in step 100 (FIG. 4A). Next, the process proceeds to step S130.

In step S130, step S120
Using the small area image A _k ′ obtained in
Stereo matching is performed with the other image data obtained in 00. Stereo matching is a correlation operation between two pieces of image data. The correlation calculation will be described with reference to FIG. FIG. 4 (a)
4B is a right image by the frame memory 12, and FIG.
Is a left image by the frame memory 13. First, FIG.
The measurement point P obtained in step S120 is displayed on the right image of (a).
_k ′ and a small area image A _k ′ are set. Reference point coordinates (x,
By setting y) in the upper left corner of the matrix, the luminance I _r of any point (m, n) in the small area image A _k 'in the, I
_r (x + m, y + n). However, n and m are 0
<N ≦ N, 0 <m ≦ M, where N and M are, for example, 4
It is.

The small image shown in the left image of FIG.
A_k’Small area image B _kSet. this
When the reference point coordinates are (x + disp, y),
Area image B_kAt any point (m, n) at_lIs I
_l(X + disp + m, y + n). here,
disp is a displacement amount for searching for a correlation. And
By changing the displacement disp from 0 to, for example, 511 pixels,
According to the following equation (2), the city block distance S_xySeeking
Confuse.

[0029]

[Number 2] _{S xy = Σ m Σ n |} I r (x + m, y + n) -I l (x + disp + m, y + n) | ··· (2) Measurement points by the calculation P Get the correlation of _k '. Then, the displacement amount disp that minimizes Expression (2) is set as the parallax at the measurement point P _k ′. Strictly speaking, the displacement disp is a unit of distance, but for convenience, the unit is a pixel as the parallax x. In the case of conversion into a unit of distance, it is sufficient to multiply the size of one pixel in the x direction (several μm). In this case, the correlation operation is performed only when the two image data are shifted left and right by the arrangement of the cameras 10 and 11, so that the y coordinate is fixed and x
Perform only in the direction.

However, in reality, the cameras 10 and 11
Because there is an individual difference such as a sensitivity difference, image data having the same brightness is not obtained. Therefore, in practice, the average value of the intensities of the small area image A _k ′ and the small area image B _k is subtracted, and the city block distance R _xy is obtained by the equation (3). If there is no sensitivity difference between the two cameras, equation (2)
May be.

R _xy = [Σ _m Σ _n | (I _r (x + m, y + n) −ave _r ) − (I _l (x + disp + m, y + n) −ave _l ) |] / MN ··· (3) However, ave _r = _{[Σ m Σ n I r (x} + m, y + n) ] / MN ave _l = _{[Σ m Σ n I l (x} + disp + m, y + n)] / MN. FIG. 5 shows the result of performing stereo matching by the above equation (3) along the x direction. The figure shows a correlation curve, in which the horizontal axis represents the displacement x and the vertical axis represents the city block distance _Rxy . Here, the displacement amount disp is replaced with a displacement amount x for simplicity. As shown in the figure, a plurality of minimum points U _i and U _j generally occur in the correlation curve. In this state, the process proceeds to step S140.

In step S140, the minimum points U _i , U
_j , and its x-coordinates x _i , x _j are measured at the measurement point P _k ′.
Remember against That is, these are the parallax candidates for the measurement point P _k ′. Next, the process proceeds to step S150 (FIG. 2).

In step S150, the measurement point P _k '
Is calculated. The matching degree may be represented by a value obtained by multiplying the city block distance by -1 or a reciprocal of a deviation of the city block distance from a certain value. In short, the collation degree may be set so as to increase as the city block distance decreases. In the present embodiment, the degree of collation is determined by more factors. As shown in FIG. 6, if close to 2 images match point in the correlation calculation, city block distance is smaller than the predetermined threshold V _th, the distance L (x _j) is increased with the predetermined threshold value V _th . Then, the greater the distance L (x _j ), the higher the coincidence. This is called a high degree of collation or correlation. Furthermore, it can be said that the smaller the vertex angle θ _j formed by the minimum point U _j and the points U _{j + 1} and U _j-1 on both sides thereof, the greater the degree of coincidence. This is because when the two image data match, the minimum value becomes the minimum value, and the valley shape including the minimum point becomes steep. That is, the vertex angle θ _j of the minimum point is inversely proportional to the degree of coincidence. Therefore, the matching degree is defined by the sum of the two, and is used to determine the reliability of the parallax candidate. The collation degree is calculated by the following equation (4).

[0033]

V (x _j ) = aL (x _j ) + b / (θ (x _j ) + c) L (x _j ) = R _th −R (x _j ) (4) where j: Exponents indicating minimum points a, b, c: constants R _th : threshold value R (x _j ): city block distance according to equation (3) θ (x _j ): apex angle of minimum value.

As a result, the collation degree V (x _j ) for each minimum point U _j , that is, the parallax candidate x _j is obtained, arranged according to the magnitude of the collation degree, and arranged and stored. At this time, if the collation degree V (x _j ) of the first rank is larger than a predetermined threshold value V _th , it is determined to be high reliability, and high reliability information, for example, 1 is given to the measurement point P _k ′. Conversely, if the collation degree of the first rank is smaller than the predetermined threshold value _Vth , it is determined to be low reliability, and low reliability information, for example, 0 is given. Therefore, at this stage, the measurement point P
_The disparity information and reliability information of _k ′ are represented in FIG. Next, the process proceeds to step S160 (FIG. 2).

In step S160, it is determined whether or not the above routine has been completed for all measurement points. If not, the process returns to step S120 to repeat the above-described routine, and the calculation of the parallax candidates and the evaluation of the matching degree are performed for all the measurement points P _k ′. If completed, the process moves to step S170.

In step S170, the continuity of the parallax with respect to the measurement points obtained by the above routine is examined and corrected. FIG. 8 shows the details of step S170. First, in step S171, the measurement point P _k ′ is designated, and the flow shifts to step S172.

In step S172, the measurement point P _k '
Is highly reliable. If the reliability is high, the process proceeds to step S173, and the first place of the parallax candidates in FIG. 7 is selected as the true parallax of the measurement point. If the reliability is low, the process moves to step S174.

In step S174, the measurement point P_k’
Is a highly reliable measurement point P_{k + 1}’, P _k-1’Etc. with brightness
It is determined whether the connection is continued. If not,
The process moves to step S176. If they are continuous,
The process proceeds to step S175, and the parallax is corrected.

In step S175, the measurement point P_kTo
Continuous high reliability measurement points P_{k + 1}’, P_k-1Etc.
Measure the parallax having the value closest to the top-ranked parallax candidate
Point P _kAre selected from the parallax candidates included in. This correction result
Is shown in FIG. FIG. 9A shows the state before the correction, and FIG.
(B) is after the correction. Unreliable measurement point P before correction_k’
Is a highly reliable measurement point P_k-1’, P_{k + 1}’
Therefore, the continuity of parallax is regarded as high reliability. So
Then, the parallax 10 closest to the parallax of the first rank on both sides is
Selected and corrected to a reliable measurement point. Next,
The process proceeds to step S176 (FIG. 8).

In step S176, it is determined whether the correction routine has been completed for all the measurement points P _k ′. If not, the process returns to step S171 to set a new measurement point P _k ′, and the above routine is repeated. That is, all the measurement points P _k ′ are examined. If completed, the process returns from step S176.

As described above, among the images captured by the cameras 10 and 11, if the measurement point determined to be low in reliability due to the influence of noise or the like is continuous with the high reliability measurement point, the parallax correction is performed. Gives a true parallax. This means that the low-reliability measurement point is regarded as a high-reliability measurement point at the same time. Therefore, the imaged object is composed of a number of highly reliable measurement points by correction. As a result, a highly reliable distance distribution is obtained at a high density. Therefore, according to this method, the distance to the object and the three-dimensional shape of the surface of the object can be grasped more accurately.

(Modifications) Although one embodiment of the present invention has been described, various other modifications are conceivable. For example, in the above embodiment, when the measurement points have low reliability, the correction is performed using the parallax of the high reliability measurement points connected on both sides, that is, on the same straight line. However, in some cases, the measurement point P _k ′ may form a corner as shown in FIG. That is, it is connected to the high-reliability measurement point, but is not interposed between them. In such a case, even if the average of the top-ranked parallaxes of the plurality of connected high-reliability measurement points is averaged and the closest parallax is selected from the plurality of parallax candidates of the measurement point P _k ′, Good. That is, the flowchart shown in FIG. 11 may be used instead of the flowchart shown in FIG.

The flowchart of FIG. 11 will be described briefly. In FIG. 11, step S175a and step S175
Steps S171 to S176 except for 175b
Is equivalent to the above embodiment. Therefore, step S174
The following will be described.

In step S174, it is determined whether or not the current measurement point P _k ′ is connected to a highly reliable measurement point, as in the above embodiment. Here, concatenation includes continuation. If not, the process moves to step S176. If connected, the process proceeds to step S175a.

In step S175a, the average value of the first-ranked parallax candidates having the high-reliability measurement points P _{k + 1} and P _k−1 connected to the measurement point P _k ′ is _obtained . Then, the process shifts to step S175b to select the parallax closest to the average value from the parallax information (FIG. 7) of the measurement point P _k ′. Then, the parallax is regarded as the true parallax of the measurement point P _k ′. That is, if the measurement point P _k ′ is connected to the high reliability measurement point in any direction, the parallax closest to the parallax average of the first rank is selected. In FIG. 10, parallax =
8 is adopted. Then, high reliability information is given. With such correction, the object is represented by more reliable measurement points as in the above embodiment. Therefore, the distance distribution to the object is obtained with high precision and high density.

In the above embodiment, the measurement points P _k and P _k ′ are one pixel constituting an image, but may be matrix-connected small area images (4 pixels and 9 pixels).
Thereby, the processing speed can be improved.

In addition, for example, in the calculation of the matching degree of parallax candidates in the above embodiment, the vertex angle θ _j of the minimum point is used as the evaluation variable, but the differential coefficient of a point adjacent to the minimum point may be used as the evaluation variable.
Alternatively, the area on the local minimum side surrounded by the predetermined threshold value _Rth and its correlation curve may be used.

In the above-described embodiment, the stereo matching is performed on the gradation-converted image data. However, the stereo matching may be performed between edge images that have been edged by a Sobel operation or the like.
That is, the processing may be performed between edge images as shown in FIG.
By performing the correlation operation between the edge images, the vicinity of the minimum point becomes steeper, and the minimum point, that is, the parallax can be calculated more easily.

[Brief description of the drawings]

FIG. 1 is a system block diagram to which a distance image calculation device according to an embodiment of the present invention is applied.

FIG. 2 is a flowchart showing a processing procedure of the distance image calculation device according to the embodiment of the present invention.

FIG. 3 is an edge image of an object according to the embodiment of the present invention.

FIG. 4 is an explanatory diagram of a correlation calculation by stereo matching according to the embodiment of the present invention.

FIG. 5 is an explanatory diagram of a correlation curve and parallax candidates according to the embodiment of the present invention.

FIG. 6 is an explanatory diagram of evaluation of matching degree by a correlation curve according to the embodiment of the present invention.

FIG. 7 is a data structure diagram of measurement points by the evaluation of the degree of collation according to the embodiment of the present invention.

FIG. 8 is a flowchart illustrating an operation of a parallax correction unit according to the embodiment of the present invention.

FIG. 9 is an explanatory diagram for correcting a low reliability measurement point according to the embodiment of the present invention.

FIG. 10 is an explanatory diagram for correcting a low-reliability measurement point according to a modification of the present invention.

FIG. 11 is a flowchart of a parallax correction unit according to a modification of the present invention.

FIG. 12 is a flowchart for obtaining a conventional distance distribution.

[Explanation of symbols]

 10, 11 camera 12, 13 frame memory 15 small area image setting unit 16 parallax candidate setting unit 17 collation degree evaluation unit 18 parallax correction unit 19 system bus

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Mitsuhiko Ota 41-41, Chuchu-yokomichi, Nagakute-cho, Aichi-gun, Aichi F-term in Toyota Central Research Laboratory Co., Ltd. (reference) 2F065 AA06 BB05 CC11 CC16 DD04 FF01 FF05 FF09 KK02 QQ03 QQ04 QQ13 QQ32.

Claims (6)

[Claims] 1. A distance image calculating device that images an object by a plurality of image pickup devices and calculates a distance to the object from the obtained plurality of image data, wherein one of the image data and the other image data is A stereo matching unit that collates and obtains one or more parallax candidates for each measurement point on an image; a collation degree evaluating unit that calculates a collation degree of the parallax candidate and evaluates the parallax candidate; If it is higher than a predetermined value, the parallax candidate having the highest matching degree is selected as the parallax of the measurement point, and if the matching degree is lower than the predetermined value, the matching degree of the surrounding measurement points related to the measurement point and the surrounding measurement A distance image calculation device, comprising: a parallax correction unit that corrects parallax of a measurement point based on a parallax in which a point is determined. 2. The stereo matching means sets a small area image including a measurement point on image data, and performs matching by a correlation operation between the small area image and a small area image moving on the other image data. The distance image calculation device according to claim 1, wherein: 3. The distance image calculation apparatus according to claim 1, wherein the degree of matching is a value corresponding to a city block distance obtained by the stereo matching means. 4. The collation degree is obtained from a minimum value related value at a minimum point of the city block distance obtained by the stereo matching means and a vertex angle related value composed of the minimum point and points on both sides thereof. The distance image calculation device according to any one of claims 1 to 3, wherein the distance image calculation device is a value. 5. The parallax candidate having a value closest to the parallax determined at the surrounding measurement point is selected from the parallax candidates at the measurement point. 2. The distance image calculation device according to claim 1. 6. The distance image calculation apparatus according to claim 1, wherein the image data is an edge image, and the measurement point is one point on the edge. .