JP2000035329A - Three dimensional image processing method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate distance with high reliability by reducing the error of parallax to some degree even in the case of causing the error by image noise. SOLUTION: This three dimensional image processing device is provided with a corresponding point detector 8 detecting each pixel of one image corresponding to each pixel of the other image, an absolute value calculator 12 separating the image into pixel blocks of m×m pixels and obtaining the absolute value of parallax vector for each pixel block, an absolute value distribution observer 14 observing the frequency distribution following the absolute value of the obtained parallax vector, a correction quantizer 16 eliminating the absolute value of the parallax vector with frequency lower than a given value, extracting maximum value of the absolute value of the parallax vector over or equal to the given value and determining it to be the representative value and quantizing each parallax vector in the pixel blocks making the reference value be a reference, and a distance calculator 10 calculating the distance to the object, based on the quantized parallax vector.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image processing method and apparatus for calculating a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector. Things.

[0002]

2. Description of the Related Art When a stereo image consisting of a pair of left and right images relating to the same target object is obtained by a stereo camera, and a point on the target object is projected on one screen,
If the projection destination on the other screen is known, the distance to a point on the target object can be calculated based on the principle of triangulation. This is disclosed, for example, in the document "Graphics and Vision" (published by Ohmsha, pages 28 to 31). The present invention relates to a three-dimensional image processing method and apparatus to which this search for a projection destination, that is, a corresponding point search technique for a stereo image is applied.

As shown in FIG. 7, in the above document, a point Q (x, y, z) on a target object 2 is arranged at a fixed distance (= base line length) B between the optical axes Ol and Or. Coordinate Q on the XY coordinate plane of the corresponding pixel in the pair of left and right stereo images captured by the pair of left and right imaging cameras 4 and 6
1 (Xl, Yl) and Qr (Xr, Yr) are obtained, and using the parallax Ql−Qr = (Xl−Xr, Yl−Yr),
Distance Z to measurement point on target object 2 corresponding to corresponding pixel
Ask for. Here, in a three-dimensional image input device for processing a stereo image, two cameras 4 and 6 usually have optical axes Ol and
Since Or is arranged so as to be included on the same XZ plane, Yl = Yr, and the corresponding point search is performed on the epipolar line Ep.
It suffices to perform the scanning on the same scanning line of l, Epr. That is, assuming that the distance between the optical axes Ol and Or of the imaging cameras 4 and 6 is B and the focal length is f, the point Q on the target object 2 is
(X, y, z) is a point Ql (Xl, Xl,
Yl) and the point Qr (Xr, Yr) when projected onto the point Q (x, y, z) of the target object 2, the distance Z is as follows: Z = B * f / (X1-Xr) (1) ). Here, Xl-Xr represents parallax, and when the left image is used as the reference image, the corresponding point Qr (Xr, Y
r) is generally Vp (X1-Xr,
Yl-Yr).

FIG. 8 is a block diagram showing a conventional distance detecting device. In FIG. 8, regarding the target object 2, C
A stereo image is obtained by the imaging camera 4 for the left image and the imaging camera 6 for the right image, such as a CD, and the corresponding point detector 8 performs a corresponding point search based on the stereo image. Here, the two cameras 4 and 6 have the same optical axis Ol and Or X-Z.
It is assumed that they are arranged in a relation of Yl = Yr so as to be included on the coordinate plane. A disparity vector for each pixel is calculated using the corresponding points searched by the corresponding point detector 8. Distance calculation is performed by the distance calculator 10 using the obtained disparity vector.

Here, as described above, by arranging the two imaging cameras 4 and 6 so that their optical axes Ol and Or are included on the same XZ coordinate plane, a search for a corresponding point is performed. Need only be performed on the scanning lines that are the epipolar lines Epl and Epr. For example, the point Ql (X
When (l, Yl) and the point Qr (Xr, Yr) on the right image correspond, the disparity vector at the point Ql (Xl, Yl) is Vp (Xl-Xr, Yl-Yr). Here, since the two points Ql and Qr are on the same scanning line (epipolar line), Yl = Yr, and the disparity vector is Vp (Xl
−Xr, 0). Such a disparity vector Vp is obtained for all pixel points on the image, and a disparity vector group is created.

[0006]

In the corresponding point search according to the conventional method, the noise of the image is directly related to the incorrect correspondence search,
Erroneous correspondence occurs due to image noise, and as a result, the detection accuracy of the disparity vector is reduced, and it has been difficult to obtain highly reliable distance data.

The present invention has been made in consideration of the above problem. Even if an error occurs in a disparity vector due to image noise, the error can be reduced to a certain extent, and highly reliable distance calculation can be performed. It is an object of the present invention to provide a three-dimensional image processing method and apparatus.

[0008]

The invention according to claim 1 is
In a three-dimensional image processing method for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the disparity vector, each image is m pixels × n pixels (m, n Is an integer of 2 or more), and a frequency distribution according to the magnitude of the absolute value of each disparity vector is observed for each pixel block, and the absolute value of the disparity vector whose frequency is less than a predetermined value is observed. The maximum value of the disparity vector absolute value is extracted from the distribution of the absolute value of the disparity vector equal to or larger than a predetermined value, and the maximum value is determined as a representative value. And calculating a distance from the stereo image to the target object based on the quantized disparity vector.

According to a second aspect of the present invention, there is provided a three-dimensional image processing method for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector. The image is represented by m pixels × n pixels (m,
n is an integer of 2 or more).
For each pixel block, the variance of the current disparity vector with respect to the average value of the current disparity vector and the surrounding disparity vectors is obtained for each disparity vector, the frequency distribution related to the obtained variance is observed, and a standard obtained in advance is calculated. The dispersion degree region having a predetermined value or more greatly deviated from the distribution characteristic is corrected according to the standard distribution characteristic, the corrected disparity vector is quantized, and the distance from the stereo image to the target object is calculated based on the quantized disparity vector. It is characterized by calculating.

According to a third aspect of the present invention, there is provided a three-dimensional image processing apparatus for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector. Corresponding point detection means for detecting each pixel of the other image corresponding to each pixel of the image of the above, and dividing the image into a pixel block of size m pixels × n pixels (m, n is an integer of 2 or more); For each pixel block, a disparity vector for each pixel, a disparity vector absolute value calculating unit for obtaining an absolute value of each disparity vector, and a disparity vector for observing a frequency distribution according to the magnitude of the obtained disparity vector absolute value Absolute value distribution observation means, exclude the absolute value of the disparity vector whose frequency is less than the predetermined value, extract the maximum value of the disparity vector absolute value from the distribution of the disparity vector absolute value of the predetermined value or more, and determine it as the representative value A disparity vector correction quantization unit that quantizes each disparity vector in a pixel block using the representative value as a maximum reference, and a distance calculation unit that calculates a distance from a stereo image to a target object based on the quantized disparity vector. It is characterized by having.

According to a fourth aspect of the present invention, there is provided a three-dimensional image processing apparatus for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector. Corresponding point detection means for detecting each pixel of the other image corresponding to each pixel of the image of the above, and dividing the image into a pixel block of size m pixels × n pixels (m, n is an integer of 2 or more); For each pixel block, disparity vector dispersity calculating means for calculating the disparity of the current disparity vector with respect to the average value of the current disparity vector and surrounding disparity vectors for each disparity vector, and a frequency distribution for the calculated dispersity. A disparity vector variance distribution observing means for observing, and a variance degree equal to or more than a predetermined value in which the observed disparity vector variance greatly deviates from a standard distribution characteristic obtained in advance. Is corrected according to the standard distribution characteristics, a disparity vector correction quantization unit that quantizes the corrected disparity vector, and a distance calculation unit that calculates the distance from the stereo image to the target object based on the quantized disparity vector. It is characterized by having.

[0012] According to the three-dimensional image processing method and apparatus according to the present invention, there is a certain amount of noise in the input image. Even if the size has changed,
It is represented by the absolute value distribution of the disparity vector and the quantized representative value determined by the variance of the current disparity vector and the surrounding disparity vector. be able to.

[0013]

(Embodiment 1) FIG. 1 shows an embodiment of the present invention. In FIG. 1, a left image Pl and a right image Pr of the target object 2 are obtained by a pair of left and right imaging cameras 4 and 6, and image signals are sent to corresponding point detectors 8. Here, JP-A-9-33
In accordance with an evaluation function such as a correlation function disclosed in Japanese Patent Publication No. 249, the association of each pixel is performed. By arranging the two imaging cameras 4 and 6 so that the optical axes Ol and Or are included on the same XZ coordinate plane, the search for the corresponding point may be performed only on the scanning line that is an epipolar line. Will be. For example, assuming that a point Ql (Xl, Yl) on the left image corresponds to a point Qr (Xr, Yr) on the right image, the disparity vector at the point Ql (Xl, Yl) is represented by Vp (Xl
-Xr, Yl-Yr). Here, two points Ql, Qr
Exist on the same scanning line, so that Yl = Yr, and the disparity vector is Vp (X1−Xr, 0). Such a disparity vector Vp is obtained for all pixel points on the image.

The disparity vector Vp output from the corresponding point detector 8 is sent to a disparity vector absolute value calculator 12. Here, the absolute value of each disparity vector Vp is calculated and output. The obtained disparity vector absolute value is sent to the disparity vector absolute value distribution observer 14. Here, the relationship between the absolute value of the disparity vector and its appearance frequency is represented as a distribution curve. FIG. 2 shows an example of this distribution curve.

Since the calculated disparity vector absolute value includes noise, the obtained disparity vectors are not always valid data. If there is noise in a flat portion where there is no change in image data, even a small amount of noise may significantly disturb the parallax vector. Therefore, in the present invention, the disparity vector absolute value distribution observer 14 converts each image into a pixel block having a size of m pixels × n pixels (m, n is an integer of 2 or more), for example, a size of 8 × 8 pixels. Then, a vector absolute value distribution map is created according to the magnitude of the absolute value of the disparity vector for each pixel block, and is sent to the disparity vector correction quantizer 16.

The disparity vector correction quantizer 16 sets a threshold value Th1 at a predetermined frequency on the obtained distribution map, excludes disparity vector data whose frequency is less than the threshold value Th1, and sets a threshold value Th1. Of the magnitudes of the absolute values of the disparity vectors appearing at the above frequencies, the maximum value DV is used as a representative value, and each disparity vector in the pixel block is quantized as shown in FIG. In this quantization, as illustrated in FIG. 3, the maximum value DV is set to 100% and quantized in units of eight equal parts. In this case, the absolute value of the disparity vector is the maximum value D
Data exceeding V is subjected to data rounding processing assuming that it has the maximum value DV. The data distribution quantized in this way is shown in FIG. 3 together with the original data. By performing such processing, it is possible to relatively suppress minute variation data of the disparity vector and obtain more reliable distance data.

The disparity vector data thus quantized is sent to the distance calculator 10. Distance calculator 1
0 calculates the distance Z for each pixel by using the above-described equation (1), and outputs the calculation result.

(Embodiment 2) FIG. 4 shows another embodiment which suppresses the influence of a minute variation due to noise or the like of a disparity vector. Here, paying attention to the degree of dispersion of the data, the degree of dispersion of the disparity vector is calculated by the disparity vector dispersion degree calculator 22 shown in FIG. Here, the degree of dispersion refers to, for example, as shown in FIG. 9, three pixels in the horizontal direction (X direction) and three pixels in the vertical direction (Y direction) around the current pixel (pixel of interest), for a total of 3 × 3. = Average value of parallax vectors Vi, j (0 ≦ i ≦ 2, 0 ≦ j ≦ 2) of 9 pixels <Vp
> By Equation 2 described below, the absolute value of the difference between the disparity vectors Vi, j of the nine pixels with respect to the average value <Vp>, and summing those absolute values. Is a value represented by Expression 3. This value is used as an index indicating the degree of correlation of the disparity vector between the current pixel and surrounding pixels.
The smaller the value is, the larger the correlation is (the difference between the disparity vector of the surrounding pixels is small), and the larger the value is, the smaller the correlation is (the difference between the disparity vector of the surrounding pixels is large). In terms of statistics, there are concepts such as “variance” and “standard deviation”.
This is somewhat troublesome because it involves multiplication and square root calculations, and the numerical values to be handled are large, and the present invention does not require much rigor. Use Of course, variance and standard deviation can be used instead of the variance of the present invention. Such a degree of dispersion is obtained for all disparity vectors.

[0019]

(Equation 1) The thus obtained disparity vector dispersion degree β is sent to the disparity vector dispersion degree distribution observer 24. Here, for example, the distribution curve β of each disparity vector and the distribution curve of its appearance frequency are observed for each pixel block having a size of 8 × 8 pixels.
FIG. 5 is a distribution curve 50 showing one example. It is known that this kind of distribution curve basically shows a characteristic similar to the Laplace distribution unless there are many small errors due to noise or the like. Therefore, as shown in FIG. 6, a reference distribution curve 52 statistically obtained in advance from various images without being affected by noise or the like.
Be prepared. FIG. 6 shows an example of the case where the distribution curve 50 has a relatively large deviation from the reference distribution curve 52 as a result of observation by the disparity vector dispersion degree distribution observer 24. In this case, it can be recognized that both characteristic curves are relatively largely shifted in a region where the degree of dispersion is large. Therefore, the threshold value Th2 is set to be slightly lower than the dispersion level region where the large shift amount occurs.
Is set, and the disparity vector indicating the dispersion degree level equal to or greater than the threshold Th2 is forcibly corrected by the disparity vector correction quantizer 26 to the disparity vector value of the pixel whose adjacent pixel is smaller than the threshold Th2. do. After this correction processing, quantization is performed according to the above-described embodiment, and the quantized disparity vector data is sent to the distance calculator 10. The distance calculator 10 calculates the distance Z for each pixel using Expression (1) as in the case of FIG.

[0020]

According to the present invention, even if a certain amount of noise exists in the input image, and as a result, the direction and magnitude of the parallax vector change more than the original parallax in the same target object, Since the disparity vector is represented by an absolute value distribution or a quantized representative value determined by a shift amount between a current disparity vector and a plurality of disparity vectors around the current disparity vector, it is possible to reduce a change in fine disparity. And highly reliable distance data can be obtained.

Therefore, even if a certain amount of noise is present in the input image, erroneous correspondence can be reduced. As a result, the accuracy of detecting a disparity vector is improved, and highly reliable distance data can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a three-dimensional image processing apparatus including a corresponding point search device according to the present invention.

FIG. 2 is a diagram showing an example of a distribution of a disparity vector absolute value and a frequency in the apparatus of FIG. 1;

FIG. 3 is an explanatory diagram for explaining a state in which the absolute value of the disparity vector in FIG. 2 is corrected and quantized.

FIG. 4 is a block diagram showing a configuration example of a three-dimensional image processing device provided with another corresponding point search device according to the present invention.

FIG. 5 is a distribution curve diagram showing an example of a distribution of a disparity vector dispersion degree level and a frequency in the apparatus of FIG. 4;

FIG. 6 is an explanatory diagram illustrating a correction area of the distribution curve in FIG. 5;

FIG. 7 is a view for explaining the principle of distance calculation using a stereo image.

FIG. 8 is a block diagram of a known corresponding point searching device based on the principle of FIG. 7;

FIG. 9 is an explanatory diagram for explaining the degree of dispersion according to the present invention.

[Explanation of symbols]

2 target object 4 left imaging camera 6 right imaging camera 8 corresponding point detector (corresponding point detecting means) 10 distance calculator (distance calculating means) 12 disparity vector absolute value calculator (disparity vector absolute value calculating means) 14 disparity vector absolute Value distribution observer (disparity vector absolute value distribution observation means) 16 Disparity vector correction quantizer (disparity vector correction quantization means) 22 Disparity vector dispersion degree calculator (disparity vector dispersion degree calculation means) 24 Disparity vector dispersion degree distribution observation (Disparity vector dispersion degree distribution observation means) 26 Disparity vector correction quantizer (disparity vector correction quantization means)

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA04 AA06 DD03 DD04 FF01 FF05 FF09 JJ03 JJ05 JJ26 QQ03 QQ28 QQ31 QQ38 QQ41 2F112 AC06 BA06 BA07 CA08 CA12 DA28 FA01 FA03 FA07 FA35 FA38 FA41 FA50 5B057 DC19 DB03 DC03

Claims (4)

[Claims] 1. A three-dimensional image processing method for calculating a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the disparity vector.
Each image is divided into pixel blocks of size m pixels × n pixels (m and n are integers of 2 or more), and the frequency distribution according to the magnitude of the absolute value of each disparity vector is observed for each pixel block. Then, the absolute value of the disparity vector whose frequency is less than the predetermined value is excluded, and the maximum value of the disparity vector absolute value is extracted from the distribution of the disparity vector absolute value of the predetermined value or more, and it is determined as a representative value.
A three-dimensional image processing method comprising: quantizing each disparity vector in a pixel block using the representative value as a maximum reference; and calculating a distance from the stereo image to a target object based on the quantized disparity vector. . 2. A three-dimensional image processing method for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector.
Each image is divided into pixel blocks having a size of m pixels × n pixels (m and n are integers equal to or greater than 2), and for each pixel block, a current disparity with respect to an average value of a current disparity vector and surrounding disparity vectors. Obtain the degree of dispersion of the vector for each disparity vector, observe the frequency distribution related to the obtained degree of dispersion, and correct the degree of dispersion of a predetermined value or more greatly deviated from the previously obtained standard distribution characteristic according to the standard distribution characteristic, A three-dimensional image processing method, comprising: quantizing a corrected disparity vector; and calculating a distance from the stereo image to a target object based on the quantized disparity vector. 3. A three-dimensional image processing apparatus for obtaining a disparity vector between corresponding pixel points of a stereo image of a target object and calculating a distance to the target object based on the vector.
Corresponding point detecting means for detecting each pixel of the other image corresponding to each pixel of the one image;
n is an integer of 2 or more).
For each pixel block, a disparity vector for each pixel, a disparity vector absolute value calculating unit for obtaining an absolute value of each disparity vector, and a disparity vector for observing a frequency distribution according to the magnitude of the obtained disparity vector absolute value Absolute value distribution observing means, excluding the absolute value of the disparity vector whose frequency is less than the predetermined value, extracting the maximum value of the disparity vector absolute value from the distribution of the disparity vector absolute value of the predetermined value or more, and determining it as the representative value. ,
A disparity vector correction quantization unit that quantizes each disparity vector in the pixel block with the representative value as a maximum reference, and a distance calculation unit that calculates a distance from the stereo image to a target object based on the quantized disparity vector. And a three-dimensional image processing apparatus. 4. A three-dimensional image processing apparatus for calculating a disparity vector between corresponding pixel points of a stereo image related to a target object and calculating a distance to the target object based on the vector.
Corresponding point detecting means for detecting each pixel of the other image corresponding to each pixel of the one image;
n is an integer of 2 or more).
For each pixel block, disparity vector dispersity calculating means for calculating the disparity of the current disparity vector with respect to the average value of the current disparity vector and surrounding disparity vectors for each disparity vector, and a frequency distribution for the calculated dispersity. A disparity vector dispersion degree distribution observing means to be observed, and a disparity degree area having a predetermined value or more in which the observed disparity vector dispersion degree is significantly deviated from a predetermined standard distribution characteristic is corrected according to the standard distribution characteristic, and the corrected disparity is corrected. Three-dimensional image processing comprising: disparity vector correction quantization means for quantizing a vector; and distance calculation means for calculating a distance from the stereo image to a target object based on the quantized disparity vector. apparatus.