JPS6280768A - Stereoscopic image processing system - Google Patents

Abstract

PURPOSE:To obtain highly accurate distance information or height information by estimating and correcting geometrical distortion on an image which may be generated due to the difference of a visual point on the basis of the relation of stereoscopic corresponding points found on an image with low resolution due to gradual thinning. CONSTITUTION:A pair of stereoscopic image data 1 is thinned to 1/2 the quantity of data by an image blurring thinning processor 2. Since simple image thinning does not satisfy a sampling theorem, blurring processing is adopted. The image with 1/2 size obtained by the processor 2 is stored in an image file 3 together with the original image. The image with 1/2 size stored in the image file 3 is read out again and furthermore blurred and thinned by the processor 2 and the blurred and thinned image is stored in the file 3 as an image with 1/4 size of the original image. After repeating image blurring thinning plural times, reduced images having plural levels are stored in the file 3. Thus, the specially rough stereoscopic corresponding relation is expressed as stereoscopic corresponding relation on the image with the current resolution.

Description

Detailed Description of the Invention [Field of Application of the Invention] The present invention relates to a stereo image processing method, and in particular, by correcting geometric distortions on image pieces due to differences in viewpoints,
This invention relates to a stereo image processing method suitable for obtaining highly accurate distance information or altitude information.

[Background of the invention]

Conventionally, stereo image processing methods using digital image processing systems have been described in Photogramstric Engineering, Mort Sensing, Vol. 44, No. 2, pp. 1499-1512 (1978).
eeringandRemote Sensing,
Vol, 44. Na2.1499-1512 (19
78) Panton (D, J, Panton
“A Flexible Approach to Digital Stereoplotting”
Digital Stereo Mapping”).

Stereo image processing systems, particularly systems aimed at processing aerial photographs, have been using image segment matching techniques to detect corresponding points because the target image data has a rich texture. The image piece matching method is a method of finding a point corresponding to a certain point by moving an image piece containing that point on another image and searching for the point with the highest similarity. As the degree of similarity between image pieces, (1) a correlation coefficient, (2) a total sum of absolute differences, etc. are usually used.

In stereo image processing, it is important to remove relative geometric distortion between corresponding image pieces due to differences in viewpoints in order to detect corresponding points with high accuracy. Removal of geometric distortion of an image is an interpolation calculation for each pixel and point based on mapping from the coordinate system of the removed image to the coordinate system of the original image. By the way, the above-mentioned geometric mapping, that is, the relative geometric distortion between image pieces, can only be determined after accurate elevation information is obtained. Therefore, conventional methods have been discussed in the above-mentioned literature, in which the geometrical distortion is changed by trial and error and the best geometrical distortion and corresponding points are simultaneously determined by an iterative hill-climbing method of the similarity of image pieces. A method has been used to extrapolate and predict the amount of distortion based on the search results for corresponding points up to adjacent image points, assuming that the change in elevation, that is, the undulation of the terrain is gradual.

However, with the method of changing the distortion of image pieces by trial and error,
When the degree of freedom of the mapping representing distortion is large, the corresponding point search process becomes enormous. Note that the degree of freedom of a mapping is the number of coefficients when the mapping is expressed by a polynomial, the number of piecewise end points when it is expressed as a piecewise linear equation, and the like.

In addition, the method assumes that changes in elevation, that is, the topography is gentle, and extrapolates and predicts the geometric distortion of the imaged soil based on the search results for corresponding points up to adjacent image points. If a mistake is made in the point search, the error will be propagated to the search for corresponding points in the image portion after that point, resulting in the disadvantage that elevation information cannot be obtained uniformly and accurately on one image.

[Purpose of the invention]

The present invention has been made to eliminate the above-mentioned drawbacks, and its purpose is to prevent errors from propagating on images in corresponding point search processing that includes prediction and correction of geometric distortion between corresponding image pieces. Without a word.

An object of the present invention is to provide a stereo image processing method that can uniformly obtain accurate altitude information on an image.

[Summary of the invention]

The present invention corrects and removes the relative geometric distortion of the ground surface due to the difference in viewpoint and the unevenness of the ground surface when determining the correspondence between each point in two stereo images based on the similarity of the image pieces.
In stereo image processing systems that require highly accurate elevation information, multiple levels of low-resolution images are processed using image processing that blurs and thins data in advance.
By using the spatially m stereo correspondence obtained using a lower resolution image as the stereo correspondence on the current resolution image and interpolating it, it is possible to reduce the distortion of the image soil due to the difference in viewpoint on the current image. This method predicts the relative distortion, and sequentially calculates the stereo correspondence from the low-resolution image to the original image.

[Embodiments of the invention]

An embodiment of the present invention will be described below with reference to FIGS. 1 to 5. Figure 2 shows how stereo images are captured by a satellite or an aircraft. Figure 3 (
It is schematically shown in a) and (b). One set of stereo image data1. For example, image data consisting of 4096 pixels x 4096 lines is first thinned out to 1/2 the amount of data by the image blur thinning processing device 2. In this case, simply thinning the image does not satisfy the sampling theorem, so the blurring process shown in the equation below is performed.

Here, I''(x'rj') is the blurred thinned image, and the load ω(p+q) is the si
This is a real space realization of a high frequency cut filter with a cross-sectional shape of the nC function.

According to equation (1), the 1/2 size image obtained by the image blurring and thinning processing device 2 is stored in the image file 3 together with the original image. The 1/2 size image of the image file 3 is read out again and sent to the image blurring and thinning processing device 2.
As a result, the image is further blurred and thinned out, and is stored in the image file 3 as an image of 1/4 the size of the original image. After such image blurring and thinning processing is repeated multiple times, multiple levels of reduced images are stored in one image file 3. In FIG. 1, the lowest resolution image data in the schematic image file 3 of reduced images at multiple levels is, for example, 256
First, this low-resolution stereo image, which has only a small amount of data such as 256 pixels compared to the original image data, is read from the image file 3, and the initial stereo correspondence detection means 4 converts it into a schematic image as shown in FIG. 3(a). Find the corresponding points of the grid points shown in . Point a on the left eye image indicating point a on the ground surface
When searching for stereo corresponding points of This is done by finding a point. The image similarity is determined by the following correlation coefficient, etc., assuming that the left eye and right eye images are Im (x+ J) Ir (k, Q), respectively. here,
1+J+ and +Q are image coordinate systems, r(m, n)=Σ Im(i, j) Ir(i+m, j+
n)(3)1+jθΩC The region Ω represents an image piece of the left eye image. As schematically shown in Figure 3, the image pattern of the image soil is.

The two images are different due to the difference in viewpoint and the unevenness of the ground surface. Then, one image fragment (before determining the degree of development using the former, geometric transformation image processing is applied to the image pattern of the right eye image, and the similarity with the geometric transformation that gives the highest similarity is calculated.) , is the degree of similarity in that point.As the geometric transformation, a piecewise linear model, a low-order polynomial model, etc. are used, and in either case, the transformation equation is expressed by a number of parameters on the order of 10. , the maximum similarity calculation is a hill-climbing method 41 calculation in the parameter space.The initial values of the parameters for the image piece correspondence calculation using such a hill-climbing method are calculated from the geometric transformation formula of the image piece with respect to the adjacent grid points. The geometric transformation of the overlapping parts of the pieces is the same, and the geometric transformation of the image soil is selected so that it is gentle.

Furthermore, linear interpolation processing of pixel data values is used for image processing of geometric transformation.

It was found by the initial stereo corresponding point detection means 4.

The corresponding point position coordinates (kp, Qp) (p=1+..., number of grid points) on the right eye image of the grid point on the left eye image in FIG. 3 are stored in the corresponding point file 5. The corresponding point position coordinates are
The geometric transformation calculating means 6 first doubles the coordinates and transforms them into coordinates on the image with high Lebel resolution, and then calculates the coefficients of the following dark linear mapping for each block surrounded by grid points on the main image as shown in FIG. ao',...

Converted to asq.

k' =, BO'i'j'+az'i'+az''
j' + aa' (4) i', j', and ni' indicate the relative coordinates within each block q6 Note that since the stereo image 1 has been corrected in advance and the corresponding points are on the same line, n
'= j l. The coefficients can be easily calculated by solving four simultaneous equations at each grid point.

According to the geometric transformation coefficient of each block determined by the geometric transformation calculation means 6, the geometric correction device 7 performs geometric correction on the right eye image with high Lebel resolution.

In the geometric correction, equation (4) is calculated for each pixel position on the corrected image to determine the coordinates on the right eye image, and the pixel data read from the image file 3 is interpolated. The geometrically corrected image is written to the image file 8. In the geometrically corrected image, as shown in FIG. 5(c), it was determined on a Lebel low resolution image. Corresponding points in the right eye binocular grid are on the same grid. The image piece matching device 9 reads the left eye image having the same resolution as the image file 8 from the image file 3, and detects corresponding points on grid points twice as fine as before. Corresponding point detection is an image piece matching method,
This is performed by searching for the maximum value of the correlation coefficient in equation (3). Figure 5 (
The X marks in c) are corresponding point detection positions on the geometrically corrected image, and it can be seen that correspondence, that is, elevation information is obtained on finer grid points. The size of the image piece for searching for corresponding points is 1/2 that of the one on the ground, allowing for more accurate corresponding point detection.As shown in Figure 1, more accurate corresponding points can be found even at old grid positions Since the detected corresponding point coordinates are on the right eye image that has undergone geometric transformation, the coordinate inverse transformation means 10 calculates the coordinates for each block, which are calculated by the geometric transformation calculation means 6. Dark linear coefficient aoq,...

As'', the coordinates are converted into coordinates on the original right eye image by calculation using equation (4) and stored in the corresponding point file 5.

FIG. 5(d) shows the positions of the converted corresponding points on the original right eye image using grid points.

The stereo correspondence search process performed by the designated resolution image stereo correspondence search means 11 in FIG. This is repeated until the correspondence search is completed.

The coordinates of the corresponding points on the original 1x image of the corresponding point file 5 are converted into elevation information by triangulation calculation by the elevation information calculation means 13, and output as elevation data 14 on the mesh. In triangulation calculation, as shown in Figure 2, the line-of-sight vector of each of the two corresponding points is determined from the position coordinates on the image,
Imaging position WQ.

This is to find the intersection point or the closest point of both line-of-sight vectors from r.

〔Effect of the invention〕

According to the present invention, it is possible to estimate and correct the geometric distortion of the image medium due to the difference in viewpoint based on the stereo correspondence point relationship obtained on the blurred and thinned out lower resolution image. Since corresponding points can be detected efficiently and accurately, a stepwise stereo image processing method from low resolution to high resolution can be realized, and highly accurate distance information or altitude information can be obtained.

[Brief explanation of drawings]

FIG. 1 is an overall configuration diagram of the stereo image processing method according to the present invention, FIG. 2 is a diagram showing how stereo aerial photographs or satellite images are captured, FIG. 3 is an explanatory diagram of the obtained stereo images and their corresponding points, and FIG. Figure 4 shows an example of the load relationship used in a blurred thinned image, and Figure 5 shows the detection of corresponding points on grid points and image distortion due to parallax based on the detected corresponding point coordinates on a low-resolution image. FIG. 2 is an explanatory diagram regarding correction of Figure 1 Chaz [! 1 $3 Hard No. 40 No. 51 ¥I]

Claims (1)

[Claims] means for determining the correspondence between points in two stereo images based on the similarity of the image pieces; image processing means for correcting in advance relative geometric distortions on the image pieces due to differences in viewpoint; In a stereo image processing system that obtains distance information or elevation information to each point from the correspondence relationship, the stereo image is blurred and thinned out to obtain a low-resolution image with a small amount of data in advance to obtain multiple levels of low-resolution images. , the spatially coarse stereo correspondence obtained using the lower resolution image is used as the stereo correspondence on the current resolution image, and by interpolation, the difference in viewpoint on the current resolution image is determined. A stereo image processing method characterized by predicting relative distortion on an image piece and sequentially calculating stereo correspondence from a low resolution image to a high resolution image.