JP2001148020A - Method and device for judging reliability in corresponding point search - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily detect a part where the reliability in corresponding point search is degraded due to an occlusion, low contrast, etc. SOLUTION: In this method of judging the reliability of a corresponding point in corresponding point search by a gradient method, an error value is calculated by substituting an obtained correspondence vector about each point in a window including a corresponding points under consideration into a gradient method expression Ex.u+Ey.v+Et=0, the error value id divided by the magnitude of luminance gradient, and an obtained value is evaluated to judge the reliability.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a corresponding point search for finding two corresponding points by a gradient method for two images having similar contents, such as two images of a target object photographed in stereo. And a device and a device for determining reliability.

[0002]

2. Description of the Related Art Conventionally, a plurality of images of an object have been obtained using a plurality of cameras, and a three-dimensional image or a three-dimensional image has been obtained from these images by using a calibration technique, a corresponding technique, and a reconstruction technique. Stereo image input system (image input device) for obtaining panoramic images, etc.
It has been known.

[0003] As a corresponding technique used for this, a gradient method (optical flow) for comparing the luminance gradient between images has been conventionally known. In the gradient method, a window is set for each image, and a plurality of luminance gradients are obtained within the window. That is, the value of the luminance gradient is obtained for each pixel using the luminance of the corresponding pixel between the windows and the peripheral pixels.

In the gradient method, the following equation is established between the luminance gradient and the corresponding vector. dE / dt = Ex · u + Ey · v + Et = 0 (1) where Ex · u = ∂E / ∂x, Ey · v = ∂E / ∂y,
Et = ∂E / ∂t, u = ∂x / ∂t, v = ∂y / ∂t E represents luminance, x and y represent image coordinates, t represents a plane image, and u and v represent corresponding vectors, respectively.

[0005] Theoretically, if the luminance gradient at two points is determined, the corresponding vectors u and v are determined. However, it may not be obtained due to the influence of image noise, quantization error, shading, and the like.

Therefore, assuming that the corresponding vector is constant within the window of n × n pixels, n × n simultaneous linear equations are set up and solved using the least squares method, and a set of corresponding vectors u, Find v.

As a pre-process for performing the above calculations, a high-pass filter is applied to the image. As a result, the shading and the luminance difference between the images are eliminated, and an image having only the frequencies necessary for the calculation is created.

[0008]

The size of the corresponding vector that can be detected by the above calculation is up to about three pixels. Therefore, for example, when the same point of the target object is shifted by several pixels or more between images due to a large parallax, an accurate corresponding point cannot be obtained.

Therefore, when the corresponding points are farther apart,
As a method for accurately searching for the corresponding point, use of a multi-resolution image has been proposed. By creating a multi-resolution image and propagating the corresponding vector calculated on the low-resolution image to the calculation on the high-resolution image, it is possible to correspond to points separated by several pixels or more.

However, occlusion may occur due to a difference in luminance or density between the object and the background, and in that case, there is a problem that data in that portion is largely lost. In addition, if the corresponding vector calculated in the low-resolution image is incorrect, the corresponding vector is propagated to the subsequent calculation in the high-resolution image, so that all of the corresponding parts are erroneous.

Therefore, if the reliability of those parts can be evaluated, it is possible to take measures such as interpolating a low reliability area with data of a high reliability area.

[0012] In order to improve the reliability of the corresponding point search, it has been proposed to perform a corresponding point search by exchanging a reference image for two images and use the degree of coincidence as an index of reliability. Kaihei 6-215111). However, this method has a problem in that it takes a long time to perform the corresponding point search twice since it needs to be performed.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has as its object to easily detect a portion where reliability in matching point search is reduced due to occlusion or low contrast.

[0014]

According to a first aspect of the present invention, there is provided a method for determining the reliability of a corresponding point in a corresponding point search by a gradient method, wherein each point in a window including a corresponding point of interest is determined. Is substituted into the gradient method equation Ex.u + Ey.v + Et = 0 to calculate the error value, and evaluates the error value to determine reliability.

According to a second aspect of the present invention, in calculating the error value, the value obtained by the gradient method is divided by the magnitude of the luminance gradient. In the method according to the third aspect of the present invention, when calculating the error value, a weight is provided for each point in the window, and the value obtained by the gradient method is multiplied by each weight.

According to a fourth aspect of the present invention, there is provided an apparatus for determining the reliability of a corresponding point in a corresponding point search by a gradient method, wherein means for obtaining a corresponding vector for each point in a window including a corresponding corresponding point of interest. And means for substituting the obtained correspondence vector into the equation of the gradient method, Ex · u + Ey · v + Et = 0, to calculate an error value thereof, and means for evaluating the error value to determine reliability. Have.

[0017]

FIG. 1 is a block diagram showing the configuration of a three-dimensional input system 1 using a reliability determination method according to the present invention.

Referring to FIG. 1, a three-dimensional input system 1 includes:
CPU 10, camera 11a, camera 11b, display device 12, keyboard 13, storage device 14, memory 1
5, each interface section 21 and the like. The camera 11a and the camera 11b may be described as a camera A or a camera B, respectively.

Each of the cameras A and B has a lens system and an image sensor. Although color photography is possible, monochrome photography may also be used. Using these two cameras A and B,
Two two-dimensional images FA having parallax about the object,
FB is input and stored in the memory 15.

The storage device 14 stores a program for generating three-dimensional data from the two-dimensional images FA and FB stored in the memory 15, a program for performing various processes necessary for the three-dimensional data, and other programs. Have been.

The CPU 10 executes a program stored in the storage device 14 to capture images by the cameras A and B, take in two-dimensional images FA and FB, search for corresponding points, and determine reliability.

For example, for two images FA and FB,
Corresponding points are obtained by the gradient method, the reliability of the obtained corresponding points is determined, and corresponding points with low reliability are corrected using values of corresponding points in the vicinity.

That is, the high reliability points are used as they are,
For low reliability points, recalculation is performed using the calculation results around the points. As a method of recalculation, for example, interpolation is performed at a highly reliable point. As an interpolation method, an appropriate filter is applied to the surrounding pixels. For example, a weighted median filter or Gaussian filter is applied. For low reliability points, the surrounding corresponding vectors are used. In that case, a method using the average value of the surrounding corresponding vectors, a method using the most reliable corresponding vector as it is, or the like is applied.

Then, based on the determined corresponding points, reconstruction of three-dimensional data is performed. The generated three-dimensional data is displayed on the display device 12 and stored in the storage device 14.

Next, the corresponding point search by the gradient method in the three-dimensional input system 1 will be described. Figure 2 shows the object Q
It is a figure showing a situation at the time of imaging.

As shown in FIG. 2, an object (target person) Q
Is photographed by two cameras A and B, and two two-dimensional images F
A and FB are obtained. The two cameras A and B are calibrated in advance, and camera parameters such as a projection matrix and lens aberration are known in advance. By determining the correspondence between points on the two images FA and FB, three-dimensional data can be reconstructed using camera parameters. Note that the calibration can be performed based on actual data. However, in that case, the accuracy decreases.

FIG. 3 is a diagram showing an example of two two-dimensional images FA and FB. One of the two-dimensional images FB is used as a reference image,
The other two-dimensional image FA is used as a reference image.

The images FA and FB include images QA and QB of the object Q, respectively. Using the gradient method, a corresponding point on the images QA and QB of the object Q is searched. The purpose of the corresponding point search is to find, in the reference image, points corresponding to all points (pixels) on the image QB of the object Q in the reference image.

When two images FA and FB are superimposed, a vector connecting the corresponding points is called a corresponding vector. The correspondence vector has a point on the reference image as a base point and a point on the reference image as an end point.

Windows WA and WB are set in the images FA and FB, respectively. The window WB is a reference window set in the reference image FB, and the window WA is a reference window set in the reference image FA. In FIG. 3, these windows WA and WB are:
The object Q is located at a position including the eyes.

A method for obtaining a corresponding vector from the windows WA and WB by using the gradient method will be described below.
In the following description, the reference image and the reference image may be replaced with the example described above.

FIG. 4 is an enlarged view showing windows WA and WB, FIG. 5 is a view showing a pixel of interest GA and its surrounding pixels,
FIG. 6 is a diagram for explaining the relationship between the luminance gradient Ex and the luminance difference Et.

As shown in FIG. 4, the windows WA, WB
Is composed of 9 × 9 pixels. But this is 3
× 3 pixels, 5 × 5 pixels, 7 × 7 pixels, or another pixel configuration may be used.

In the windows WA and WB, attention is paid to attention pixels GA and GB located at the same coordinates. Referring to FIG. 5, a luminance gradient E is set for pixels of interest GA and GB.
x, Ey, and the luminance difference Et are defined as follows.

Ex = [([GAc] − [GAb]) + ([GBc] − [GBb])] / 2 (2) Ey = [([GAd] − [GAa]) + ([GBd] − [GBa])] / 2 (3) Et = [GB] − [GA] (4) where [] indicates the value of each pixel.

The following equation (1) holds as described in the section of the prior art between these equations. Ex · u + Ey · v + Et = 0 (1) where u and v are the corresponding vectors in the x and y directions, respectively.

In order for the above equation (1) to hold, the pixels of interest GA and GB need to correspond to substantially the same location on the object Q. Here, the above equation (1) will be described with reference to FIG. In the windows WA and WB, it is assumed that the corresponding vectors u and v for all the pixels are the same. In addition, each window WA, W
In B, it is assumed that the luminance gradients Ex and Ey are constant.

FIG. 6 shows the windows WA and WB shown in FIGS. 4 and 5 in a one-dimensional manner only in the horizontal direction (x direction). The vertical direction in FIG. 6 indicates the magnitude of the luminance.

[0039] In FIG. 6, the pixel of interest GA, x-coordinate of the GB is x _1, a vector toward the target pixel GB from the pixel of interest GA is luminance difference Et. The difference in luminance when the pixel is separated by one pixel, for example, the difference in luminance between the pixel GA and the pixel GAc is the luminance gradient Ex. That is, the luminance gradient Ex is a gradient of the straight lines LA and LB indicating the luminance of the images FA and FB. Assuming that the brightness changes of the images FA and FB are constant and equal, the two straight lines LA and LB are parallel.

Then, for the object Q, the same part
Because the same brightness becomes the same, it corresponds to the attention pixel GA, for example.
Is the point of interest pixel GA on the straight line LA
CS extending in the x direction (lateral direction) and intersecting with the straight line LB
It is. X coordinate of point CS _TwoThen the corresponding vector
u is u = x_Two-X₁ Becomes The size of the corresponding vector u (x_Two-X₁Absolute
Value) is typically one pixel to several pixels. Correspondence vector
For u, the following equation (5) holds.

Ex · u + Et = 0 (5) This equation (5) holds as many times as the number of pixels in the x direction in the window WA. When the equation (5) is expanded two-dimensionally, the above equation (1) is obtained.

Therefore, equation (1) for u and v is
The same holds for the number of pixels in the windows WA and WB. For example, a window W of 9 × 9 pixels as in the example shown in FIG.
When A and WB are used, 81 equations (1) for u and v are generated. Therefore, 81 simultaneous equations are solved using the least squares method, and corresponding vectors u and v are obtained.

However, when the corresponding vector is obtained by the above-described method, the amount of displacement of the image within 2 to 3 pixels can be measured at most. That is, for example, when the same point is shifted by several pixels or more due to a large parallax, an accurate corresponding point cannot be obtained. This is a restriction from preconditions in the gradient method, and two images FA and FB must be similar in order to obtain a corresponding point.

Therefore, if the corresponding points are farther apart,
A multi-resolution image is used as a method for accurately searching for the corresponding point. The method using the multi-resolution image itself is conventionally known as described above.

FIG. 7 is a diagram for explaining a multi-resolution image.
It is. As shown in FIG. 7, the multi-resolution image FM
Image F and (1/2) ^Two, (1/4)^Two, (1
/ 8)^TwoResolution obtained by compressing each, such as
Multiple types of images F1, F2, F3 with low degree (resolution)
Become. Lower resolution images are better than higher resolution images.
The size becomes smaller. Place the higher resolution image below
When images with lower resolutions are placed on top of each other,
The shape becomes That is, the multi-resolution image FM is
Form a mid.

According to the multi-resolution image FM, when the resolution is high, that is, when the size is large, the two images FA and FB
Even two points that are far apart are close when the resolution is low, that is, when the size is small. Therefore, the range in which the corresponding point can be accurately obtained by the gradient method is expanded.

When such a multi-resolution image is used, first, a corresponding vector is obtained from an image having the lowest resolution. Then, using the obtained corresponding vector as an initial value, a corresponding vector is obtained for an image having the next lowest resolution. By repeating this process, even if the same point on the target object Q is largely displaced between the two images FA and FB, a corresponding vector can be finally obtained with a high resolution.

In the present embodiment, the reliability is evaluated for each resolution (each layer) of the multi-resolution image, and low reliability data is corrected for each resolution. In this way, if there is an error in the corresponding point determined at one resolution, it is corrected in that resolution, so that the corresponding point error is not propagated to the corresponding point search at the next resolution. As a result, the accuracy of the corresponding point search can be improved, and more accurate three-dimensional data or a panoramic image can be obtained.

In the case where the evaluation and the correction of the reliability are not performed for each resolution, when an erroneous correspondence is obtained in an image of a lower hierarchy, the correction of the correspondence is performed in a subsequent higher hierarchy. Since this is impossible, the accuracy of the corresponding point search is greatly reduced.

Next, a method of determining reliability will be described. In the present embodiment, in order to determine reliability, the obtained corresponding vectors u and v are substituted into the expression (1) of each point, and the error of the expression is obtained. Then, the sum of these errors is used as a measure of reliability. If the assumption that the corresponding vectors are the same is not established due to a step in the image or the like, the error value increases.

FIGS. 8 and 9 are diagrams for explaining a method of determining reliability by the method of the present embodiment. FIG. 8 shows a case where the object Q1 is parallel to the arrangement of the cameras A and B, and FIG. 9 shows a case where the object Q1 is inclined with respect to the arrangement of the cameras A and B.

Although the present technique is valid even when the optical axes are not parallel as shown in FIG. 9, the case where the optical axes are parallel as shown in FIG. 8 will be described first to simplify the problem.
In FIG. 8, an object Q1 is photographed by two cameras A and B whose optical axes are parallel. The imaging plane SA of each camera A, B,
SB is perpendicular to the optical axis.

The focal lengths of the lenses of the cameras A and B are f, the base line length is H, and the image pickup planes SA,
It is assumed that x coordinates on the SB are x ₁ and x ₂ , respectively, and a distance between the object Q1 and the imaging planes SA and SB is Z. The coordinates are set such that the center of the optical axis of each of the cameras A and B is the origin and the right direction is positive. Assuming that x ₁ and x ₂ point to the same point on the object Q1, the following relationship is established.

[0054] If _{x 2 = (fH / Z)} + x 1 ...... (6) Further, the surface and the imaging surface SA of the object Q1, and the SB in parallel, the difference between x ₂ and x ₁ c, c = X ₂ -x ₁ (7) is constant. That is, in this case, the correspondence vectors u and v are equal to each other for all points on the surface of the target object Q1.

Next, a case where the surface of the object Q1 is inclined as shown in FIG. 9 will be described. In FIG. 9, the broken line indicates the case where the surfaces of the object Q1 are parallel, and the solid line indicates the case where the surface of the object Q1 is parallel.
Are tilted, respectively.

As can be understood from the figure, (x ₂ −x ₁ ) corresponding to the corresponding vector indicates that the object Q1 is the camera A, B
The greater the distance from, that is, the greater the distance, the larger. Considering the window, the object Q1 included in the window
The greater the inclination of the surface, the greater the variation of the corresponding vector.

From this, the same can be said for the error when the corresponding vectors u and v obtained from the simultaneous equations are substituted into the equations obtained from the respective pixels in the window.

That is, the sum of the error values of the equations obtained from the pixels in the window is obtained, and this is used as a value for evaluating the inclination of the object Q1. Assuming that this evaluation function is F1, the evaluation function F1 is expressed by the following equation (8).

[0059]

(Equation 1)

Here, N is a value for indicating all points (pixels) in the window. For example, for a window of 5 × 5 pixels, N = 25. It should be noted that the expression in Σ of the above expression (8) is the absolute value of the left side of the above expression (1) divided by the magnitude of the luminance gradient. Since the expression (1) is an expression in which the weights of the luminance gradients are added to the corresponding vectors u and v, the variation of the corresponding vectors u and v is not seen. By dividing by the magnitude of the brightness gradient,
It is changed to substantially correspond to the variation.

That is, if the luminance on the left side of the equation (1) remains, that is, if the luminance is high, the error value increases. Conversely, when the image is dark, the error value becomes small. Then, if a simple comparison is made irrespective of the brightness, even if there is some error in the case of darkness, the error value by calculation is small, so that the reliability may be high. Thus, the influence of brightness is eliminated by dividing the denominator by the magnitude of the luminance gradient.

Further, by taking the absolute value, the magnitude, not the direction of the variation, is evaluated. Even when the data to be compared has no continuity due to steps or occlusion, the value of the evaluation function F1 is large.
Therefore, the evaluation function F1 indicates the reliability of the corresponding vectors u and v, and the smaller the value, the higher the reliability.

By the way, in obtaining the corresponding vectors u and v, what is obtained by using the window is the corresponding vector u and v of the pixel at the center of the window. Therefore, it does not make much sense to evaluate the error of the equations of the pixels around the window. Therefore, a weight Wi is added to each pixel in the window as in the following equation (9).

[0064]

(Equation 2)

FIGS. 10A and 10B are diagrams showing examples of the weight table WT. FIGS. 10A and 10B show a case where the size of the window is 5 × 5 pixels. In the weight table WT shown in FIG. 10A, the weight Wi of each pixel can be set, for example, as in the example of the weight table WT1 shown in FIG. 10B.

For example, if the size of the window is 5 × 5 pixels, 25 equations can be obtained. The corresponding vectors u and v obtained by these equations are evaluated using the above equation (9). At this time, if a weight is set so as to have a Gaussian distribution centering on the center weight w13 of the window as in a weight table WT1 shown in FIG. be able to.

The value obtained by the evaluation function F1 of the above equation (9) is compared with an appropriate threshold value Th. If the value of the evaluation function F1 is smaller than the threshold value Th, that is, if F1 <Th, the corresponding vectors u and v have high reliability.

As in the above equations (8) and (9), the luminance obtained from the equation of the gradient method is used as the denominator, and the result is divided by the magnitude of the luminance gradient to eliminate the influence of the luminance gradient. And reliability can be accurately determined. Therefore, it is possible to easily detect a portion where the reliability in the corresponding point search is reduced due to occlusion or low contrast.

Further, as shown in the above equation (9), by adding the weight Wi to each pixel in the window, the error value of the pixel to be obtained can be mainly evaluated. Further, such determination of reliability is performed to determine a corresponding vector at the time of obtaining a correspondence between images. Therefore, when reliability is low, a correct corresponding vector is corrected before the corresponding vector is determined. , Or other suitable processing.

Next, the processing operation of the three-dimensional input system 1 will be described with reference to a flowchart. FIG. 11 is a flowchart showing a processing operation when three-dimensional data is reconstructed using a multi-resolution image.

In FIG. 11, two images FA and FB are input (# 21). A multi-resolution image (image pyramid) is generated (# 22). The image at the lowest resolution level is selected as the image to be processed first (# 2).
3).

The corresponding vector is calculated using the gradient method (# 24). It is determined whether corresponding vectors have been calculated for all points on the reference image (# 25). Normally, the pixel is calculated one pixel at a time from the upper left pixel to the right, starting from the upper right pixel, moving to the next lower line when reaching the right end, and ending when reaching the lower right pixel.

Then, it is determined whether the corresponding vector for the bottom of the image pyramid, that is, the hierarchy having the highest resolution has been calculated (# 26), and the above processing is repeated until the result is YES. When the corresponding points are determined, the three-dimensional data is reconstructed based on the data calibrated in advance and the result of calculating the corresponding vector (# 2).
7).

FIG. 12 is a flowchart showing the processing for calculating the corresponding vector. In FIG. 12, first, a reference window is created from a reference pixel (pixel of interest) and its surrounding pixels (# 41). Using the reference window and the corresponding vector propagated from the upper layer, that is, the lower resolution layer, a reference window of the relevant layer is created (# 42).
In the first hierarchy, an initial value is set. For example, “0” is set as an initial value.

The luminance gradients Ex and Ey and the luminance difference Et are calculated based on the reference window and the reference window (# 43). For all points in the window, Ex,
It is determined whether Ey and Et have been calculated, and the process is repeated until the calculation is completed (# 44).

Ex, E obtained from each point in the window
The above equation (1) is created from y and Et, and the corresponding equations u and v are obtained by solving the simultaneous equations using the least squares method (# 45).

Then, an error value is obtained by using the above equation (9) (# 46), and reliability is determined by comparing the error value with a threshold value (# 47). If the reliability is low, for example, the method of calculating the corresponding vector is changed. The determination of the reliability of the corresponding vector is performed for each resolution layer.

As an example of changing the calculation method, a method of changing the window size can be considered. For example, if a 5 × 5 pixel window is used, the window is changed to, for example, a 7 × 7 pixel window. By increasing the size of the window, the effect of steps between images becomes relatively small. In addition, increasing the size of the window increases the number of equations, thereby reducing the effect of noise. For example, when the contrast of the surface of the object Q is low, the denominator of the above equations (8) and (9) becomes small, so that the value of the evaluation function F1 becomes large. Then, for example, there is a possibility that the influence of the CCD noise becomes large and a correct value cannot be calculated. Increasing the size of the window reduces the effects of noise and allows for the calculation of correct values.

When the reliability is low, there is a method in which the values of the corresponding vectors u and v are used as they are from the upper layer. In the embodiment described above, 3
The configuration, circuit, processing contents, processing order, processing timing, and the like of the dimension input system 1 can be appropriately changed in accordance with the gist of the present invention.

[0080]

According to the present invention, it is possible to easily detect a portion where the reliability in the corresponding point search is reduced due to occlusion or low contrast.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a three-dimensional input system using a reliability determination method according to the present invention.

FIG. 2 is a diagram illustrating a situation when an object is photographed.

FIG. 3 is a diagram illustrating an example of two two-dimensional images.

FIG. 4 is an enlarged view showing a window.

FIG. 5 is a diagram showing a target pixel and its peripheral pixels.

FIG. 6 is a diagram for explaining a relationship between a luminance gradient Ex and a luminance difference Et.

FIG. 7 is a diagram for explaining a multi-resolution image.

FIG. 8 is a diagram for explaining a reliability determination method according to the method of the present embodiment.

FIG. 9 is a diagram for explaining a reliability determination method according to the technique of the present embodiment.

FIG. 10 is a diagram illustrating an example of a weight table.

FIG. 11 is a flowchart illustrating a processing operation when reconstructing three-dimensional data.

FIG. 12 is a flowchart showing a corresponding vector calculation process.

[Explanation of symbols]

Reference Signs List 1 3D input system (reliability determination device) 10 CPU (means for finding corresponding vector, means for calculating error value, means for determining reliability)

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Takayuki Hamaguchi 2-3-13 Azuchicho, Chuo-ku, Osaka-shi, Osaka Inside Osaka International Building Minolta Co., Ltd. (72) Inventor Yuichi Kawakami Azuchi, Chuo-ku, Osaka-shi, Osaka 2-13-3, Osaka International Building Minolta Co., Ltd. F-term (reference) 5B057 BA02 CA08 CA13 CA16 DA07 DC22 DC34 5L096 AA06 BA08 CA05 EA39 GA02 HA01

Claims (4)

[Claims] 1. A method for determining the reliability of a corresponding point in a corresponding point search by a gradient method, wherein a corresponding vector obtained for each point in a window including a corresponding point of interest is represented by an equation of a gradient method, Ex U + Ey.v + Et = 0, and calculating an error value thereof; evaluating the error value to determine reliability; and a method of determining reliability in a corresponding point search. 2. The method according to claim 1, wherein, in calculating the error value, a value obtained by an equation of a gradient method is divided by a magnitude of a luminance gradient. 3. The method according to claim 1, wherein a weight is provided for each point in the window in calculating the error value, and a value obtained by an equation of a gradient method is multiplied by each weight. Method for determining reliability in point search. 4. An apparatus for determining the reliability of a corresponding point in a corresponding point search by a gradient method, comprising: means for obtaining a corresponding vector for each point in a window including a corresponding point of interest; , A method of substituting Ex.u + Ey.v + Et = 0 to calculate an error value thereof, and a means of evaluating the error value to determine reliability. Device for determining the reliability in searching for corresponding points.