JP2008090608A - Image processing apparatus, method for controlling image processing apparatus, and control program of image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus smoothly computing the correspondence between a standard image and a reference image. <P>SOLUTION: The apparatus inputs a standard image and a reference image, which are taken by two cameras provided on the right and left. The pixels of the two images are associated with each other, and after association data are calculated, the reliabilities of the association data are determined (S106). For the data with the lower reliability, interpolation data is calculated from association data of peripheral pixels (S107), and the interpolation data is evaluated (S108). Based on the evaluation result, parameters or initial values are changed to recalculate the association (S109). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing device, a control method for the image processing device, and a control program for the image processing device, and in particular, an image processing device capable of calculating a correspondence relationship between a reference image and a reference image, a control method for the image processing device, And an image processing apparatus control program.

  Each pixel of two images captured and acquired from different viewpoints by a stereo camera is associated with each other, and the three-dimensional position of each point on the image based on the principle of triangulation based on the obtained correspondence There is known a method for calculating. In recent years, attention has been focused on corresponding point search by a phase-only correlation (POC) method that enables robust and highly accurate association. In this method, the amplitude component of the image is removed and the correlation calculation is performed using only the phase component of the image, so that the method is less susceptible to luminance fluctuations and noise. Although this method can search for corresponding points with high accuracy, it is difficult to search for corresponding points with high reliability at all points, and there are points with low reliability at a certain rate. It is necessary to take some measures against this point.

  FIG. 18 is a diagram schematically showing a method of calculating a three-dimensional position from the corresponding points between images based on the principle of triangulation.

  When the difference between the reference camera and the reference camera corresponding point position is Δd, the distance D to the object is calculated by the following equation.

D = fB / Δd
Here, the base line length of the camera is B, and the focal length of the camera lens is f.

At this time, the three-dimensional position (X, Y, Z) of the object is calculated by the following equation.
X = xD / f, Y = yD / f, Z = D
Here, x and y are the coordinates of the pixel of interest on the reference image.

  The following Patent Document 1 discloses a method of correcting with the peripheral parallax when the reliability of the obtained parallax (displacement between pixels) is low.

Non-Patent Document 1 below describes that when the reliability is low, interpolation is performed with peripheral parallax, and POC is recalculated using the interpolated data as an initial value.
Japanese Patent Laid-Open No. 10-289316 “A High-Accuracy Passive 3D Measurement System Using Phase-Based Image Matching”, IEICE (The Institute of Electronics, Information and Communication Engineers) TRANS. FUNDAMENTALS, VOL.E89-A, NO.3 MARCH 2006

  However, in the technique of Patent Document 1, since correction from peripheral parallax is performed, there is a problem in that it cannot be determined whether the data is correct and accuracy cannot be guaranteed.

  In addition, the technique of Non-Patent Document 1 has a case where the reliability is low even when the parallax obtained first is correct. In such a case, recalculation with the same parameters is performed on the interpolation data. However, there is a problem that the reliability is lowered again. Further, since the peripheral parallax used when generating the interpolation data is used regardless of the reliability, there is a problem that the quality of the interpolation data cannot be guaranteed.

  The present invention has been made to solve such a problem, and is an image processing device capable of satisfactorily calculating a correspondence relationship between a standard image and a reference image, a control method for the image processing device, and image processing. The object is to provide a control program for the apparatus.

  In order to achieve the above object, according to one aspect of the present invention, an image processing apparatus associates corresponding points between a base image and a reference image, and corresponding point searching means for calculating corresponding data and corresponding point searching means include: A reliability determination means for determining the reliability of the calculated corresponding data, and when the reliability determined by the reliability determination means is lower than a predetermined threshold, based on the corresponding data around the corresponding point, Interpolation data generation means for generating interpolation data for the corresponding points, interpolation data evaluation means for evaluating the generated interpolation data, and when recalculating the corresponding data of the corresponding points, the evaluation result of the interpolation data evaluation means And recalculating means for changing the parameters used in the corresponding point searching means.

  Preferably, the interpolation data evaluation unit performs the evaluation based on a difference between the interpolation data for the corresponding point and the corresponding data.

  Preferably, when the evaluation by the interpolation data evaluation unit is larger than a predetermined threshold value, the recalculation unit sets the generated interpolation data as an initial value and uses the same parameter as the parameter used by the corresponding point search unit. Recalculate the corresponding data of the corresponding points.

  Preferably, when the evaluation by the interpolation data evaluation unit is smaller than a predetermined threshold value, the recalculation unit uses the generated interpolation data or the corresponding data for the corresponding point as an initial value in the corresponding point search unit. Recalculate the corresponding data of the corresponding point with a parameter different from the parameter.

  Preferably, the parameter includes a parameter relating to a window size used in the corresponding point search means.

  Preferably, the parameter includes a parameter related to a cut frequency of the low pass filter used in the corresponding point search means.

  Preferably, the interpolation data generation means includes correspondence data evaluation means for evaluating the correspondence data around the corresponding point, and when the evaluation is high by the correspondence data evaluation means, based on the correspondence data around the corresponding point, Interpolation data for the corresponding point is generated.

  Preferably, the interpolation data evaluation means does not perform evaluation when no interpolation data is generated.

  Preferably, when the interpolation data for the corresponding point is not created, the image processing apparatus changes the parameter used by the corresponding point search unit and recalculates the corresponding data of the corresponding point.

  Preferably, the corresponding data evaluation unit evaluates a plurality of corresponding data around the corresponding point, and the interpolation data generating unit determines the reliability of the plurality of corresponding data around the corresponding point that is equal to or higher than a predetermined threshold value. When the number of pieces of data having the number is larger than a predetermined number, interpolation data is generated with a high evaluation.

  Preferably, the interpolation data generating means performs interpolation using only data having a reliability level equal to or higher than a predetermined threshold value.

  Preferably, the interpolation data generating means generates the interpolation data using the average, median, or other statistics.

Preferably, the interpolation data generation means generates the interpolation data in consideration of the data distribution.
Preferably, the corresponding point search means associates corresponding points between the standard image and the reference image using the phase only correlation method.

  According to another aspect of the present invention, an image processing apparatus control method associates corresponding points between a base image and a reference image, calculates corresponding data, and corresponding points calculated in a corresponding point searching step. A reliability determination step for determining the reliability of the data, and when the reliability determined in the reliability determination step is lower than a predetermined threshold, based on the corresponding data around the corresponding point, for the corresponding point An interpolation data generation step for generating interpolation data, an interpolation data evaluation step for evaluating the generated interpolation data, and when recalculating the corresponding data of the corresponding points, based on the evaluation result of the interpolation data evaluation step, And a recalculation step for changing parameters used in the corresponding point search step.

  According to still another aspect of the present invention, the control program of the image processing apparatus associates corresponding points between the base image and the reference image, and calculates the corresponding data by calculating a corresponding point and a corresponding point searching step. A reliability determination step for determining the reliability of the corresponding data, and when the reliability determined in the reliability determination step is lower than a predetermined threshold, the corresponding point based on the corresponding data around the corresponding point An interpolation data generation step for generating interpolation data for, an interpolation data evaluation step for evaluating the generated interpolation data, and when recalculating the corresponding data of the corresponding points, based on the evaluation result of the interpolation data evaluation step The computer executes a recalculation step for changing the parameters used in the corresponding point search step.

  According to these inventions, it is possible to provide an image processing device, a control method for the image processing device, and a control program for the image processing device that can favorably calculate the correspondence relationship between the standard image and the reference image.

FIG. 1 is a diagram showing a configuration of a three-dimensional measuring apparatus according to one embodiment of the present invention.
Referring to the figure, the three-dimensional measuring apparatus is composed of two cameras 100R and 100L arranged side by side, an arithmetic processing device 200 connected to the cameras, and a display device 300.

  The cameras 100R and 100L capture a two-dimensional image of the subject O (here, a human face is illustrated). The camera 100R and the camera 100L are set apart from each other by a predetermined distance. Thereby, a stereo image can be obtained.

  The arithmetic processing device 200 calculates the three-dimensional shape of the subject from images taken by two cameras. The display device 300 displays the measured three-dimensional shape of the subject. In addition, you may make it memorize | store the obtained three-dimensional shape data (data which consist of X, Y, Z coordinate etc.) in a storage medium, without displaying.

FIG. 2 is a block diagram illustrating a hardware configuration of the arithmetic processing device 200 of FIG.
Referring to the figure, an arithmetic processing unit 200 includes a CPU 601 for controlling the entire apparatus, a LAN (local area network) card 607 (or modem card) for connecting to a network or communicating with the outside, a keyboard, An input device 609 configured with a mouse or the like, a flexible disk drive 611, a CD-ROM drive 613, a hard disk drive 615, a RAM 617, and a ROM 619 are provided.

The arithmetic processing apparatus 200 inputs a camera image from the video input unit 651.
The arithmetic processing device 200 is connected to a display device (display) 300.

  Data such as programs and images recorded on the flexible disk F can be read by the flexible disk drive 611, and data such as programs and images recorded on the CD-ROM 613a can be read by the CD-ROM drive 613. Is possible.

  Note that the arithmetic processing unit 200 may be provided with a DVD drive, a memory card reader, or the like for inputting images such as stereo photographs. A hard disk drive is preferably used for image storage.

  In addition, a video signal output terminal to a television may be provided for displaying an image such as a photograph.

FIG. 3 is a block diagram illustrating a functional configuration of the arithmetic processing device 200.
The arithmetic processing device 200 includes a corresponding point search unit 301 and a three-dimensional position calculation unit 302, and inputs a reference image I1 that is an image captured by the camera 100L and a reference image I2 that is an image captured by the camera 100R. .

  The corresponding point search unit 301 searches for a pixel on the reference image I2 corresponding to each pixel of the standard image I1.

FIG. 4 is a diagram for explaining scanning of a point of interest.
By setting a target point P on the reference image I1 and finding a position corresponding to the target point P on the reference image I2, a positional shift between the two images at the target point is detected.

FIG. 5 is a diagram illustrating window setting processing for a reference image.
Referring to the figure, a window W1 having a predetermined size centered on the point of interest P in the reference image I1 is set.

FIG. 6 is a diagram illustrating window setting processing for a reference image.
The window W2 (size and shape is the same as the window W1) is scanned on the reference image I2, and the pattern closest to the image in the window W1 on the standard image I1 is set as a corresponding portion on the reference image. The proximity determination can be performed by correlation calculation between windows, SAD (Sum of Absolute Differences) calculation, or the like. Thereby, a point corresponding to the attention point P on the reference image I1 can be set.

  In FIG. 6, the window is also moved in the vertical direction, but if two cameras are installed horizontally, the image shift between the base image and the reference image occurs only in the horizontal direction. The window may be moved only in the horizontal direction.

  Furthermore, the reference image window may be checked for correspondence with the reference image by moving it over a wide range, but the “initial value” of the search is determined in advance, and the reference image window is only within a predetermined range from the initial value. By moving, the correspondence between the standard image and the reference image may be checked. That is, a correspondence relationship between the base image and the reference image is obtained using an image with a low resolution, and an initial value for an image with a high resolution is set using the result.

FIG. 7 is a diagram illustrating the association between the base pixel and the reference pixel.
One point in the standard image shown on the left is called a standard pixel, and a pixel of the reference image corresponding to the standard pixel (a pixel that is photographing the same subject) is called a reference pixel as shown on the right in the figure. . A vector directed from the position of the reference pixel to the position of the reference pixel is referred to as a “parallax vector”.

  Assuming that the coordinates of the reference pixel are (bx, by) and the coordinates of the reference pixel are (rx, ry), the data (corresponding data) representing the correspondence between them is the disparity vector (vx, vy) = (rx, ry). ) − (Bx, by) and hold as (bx, by, vx, by).

  FIG. 8 is a diagram for explaining a phase only correlation method as another method executed by the corresponding point search unit 301.

  The corresponding point search unit 301 sets the position of a window for scanning the reference image I1, and performs a two-dimensional discrete Fourier transform (2D DFT) on a portion indicated by the window, and a reference image I2. The position of the window for scanning is set, and the Fourier transform unit 203 that performs two-dimensional discrete Fourier transform (2D DFT) on the portion indicated by the window, and normalization for the two Fourier transformed images Two normalization units 205 and 207 to perform, a synthesis unit 209 that synthesizes the normalized data, and an inverse Fourier transform unit 211 that performs an inverse Fourier transform on the synthesized data. By the inverse Fourier transform, a POC value indicating a deviation between the two images is obtained. Based on the position shift, the three-dimensional coordinates of the surface of the subject O are obtained.

  The phase-only correlation (POC) function is a function having a steep correlation peak in the coordinates of the movement amount between images (between the reference window and the reference window), and is said to have high robustness and high movement amount estimation accuracy in image matching. There is a feature. Therefore, the amount of positional deviation of the image is calculated by estimating the peak of POC. Since POC is obtained discretely, high resolution corresponding point coordinates can be obtained by estimating the peak position with subpixels. When performing interpolation, fitting a function such as a parabola is conceivable.

The size of the window is N ₁ × N ₂ pixels, and the images in the standard image and reference image windows are f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ).

The images after the DFT processing are F (k ₁ , k ₂ ) and G (k ₁ , k ₂ ), and the normalized images are F ′ (k ₁ , k ₂ ) and G ′ (k ₁ , k _2). ). When the synthesized data is R (k ₁ , k ₂ ) and the image after inverse Fourier transform is r (k ₁ , k ₂ ), the processing executed by the configuration of FIG. 8 is expressed as follows. .

Note that the cut frequency of the low-pass filter may be applied to the synthesized data obtained by the synthesis unit 209 before the inverse Fourier transform is executed by the inverse Fourier transform unit 211. An example of the low-pass cut filter is as follows, and U ₁ and U ₂ are parameters.

  The three-dimensional position calculation unit 302 in FIG. 3 calculates the three-dimensional position of each point on the image based on the principle of triangulation from the correspondence relationship between the reference image I1 and the reference image I2 obtained by the corresponding point search unit 301. calculate.

9 and 10 are diagrams illustrating processing executed by the corresponding point search unit 301. FIG.
Of these, FIG. 9 shows the process of creating the correspondence data of the base pixel and the reference pixel by scanning the attention point, and FIG. 10 evaluates the created correspondence data, and if necessary, creates interpolation data and regenerates it. The process which performs calculation is shown.

  In step S101 of FIG. 9, a stereo image composed of a base image and a reference image is input. In step S102, it is determined whether or not there is a pixel to be scanned for the point of interest in the reference image. If YES, the corresponding point is searched for the reference pixel and the reference pixel at the target point by the above-described processing in step S103, and corresponding data is generated. As described above, the correspondence data is data representing association between pixels of the base image and the reference image. The reliability of each corresponding data is recorded together with the corresponding data.

  The “reliability” of the corresponding data indicates the peak height of the POC value. If the peak is high, the reliability is high.

  In step S104, the point of interest which is the position to be scanned is moved, and the process returns to step S102.

Thereby, it is possible to obtain a corresponding data group based on the scan of the point of interest.
In FIG. 9, the entire image may be processed, only a preset area in the image may be processed, or the entire image may be processed at a certain sampling interval. You may change the point processed according to a situation.

  In FIG. 10, a rescan is performed on each corresponding point scanned in FIG. In step S105, it is determined whether or not there remains a scan target. If YES, the process proceeds to step S106, and if NO, the process ends.

  In step S106, the reliability of the corresponding data of the corresponding points to be scanned is determined. When the reliability is high (when it is equal to or greater than a predetermined threshold), the process proceeds to step S111, and when the reliability is low, the process proceeds to step S107.

  In step S107, interpolation data for the corresponding point is generated by performing interpolation using the corresponding data for the pixels around the corresponding point. In step S108, the generated interpolation data is evaluated.

In step S109, the corresponding data is recalculated at the corresponding points.
In step S110, the scan position is updated, and the process returns to step S105.

  If it is determined in step S106 that the reliability is high, the data output in step S103 is determined as the final output for the corresponding data of the corresponding point (S111), and the process proceeds to step S110.

11 to 13 are diagrams illustrating specific examples of the interpolation data creation process.
FIG. 11 shows a situation in which the reliability of correspondence data (indicated by a black circle) of pixels around the correspondence data (indicated by a dotted circle) having a low reliability is high. In the figure, the horizontal axis indicates the pixel position, and the vertical axis indicates the value of the corresponding data (that is, the parallax that is the difference between the reference pixel and the reference pixel).

  Interpolated data (indicated by hatching circles) is created by interpolation from corresponding data of surrounding pixels (step S107 in FIG. 10). At this time, the interpolation data is evaluated (step S108 in FIG. 10).

  In the evaluation, the absolute value of the difference between the value indicated by the interpolation data and the corresponding data with low reliability is used. In FIG. 11, since the difference is quite large, it is meaningful to create interpolation data, and there is a high possibility that the interpolation data is close to the correct data (the corresponding data with low original reliability may be noise or the like). Is high).

  Accordingly, in this case, in step S109 in FIG. 10, the POC is recalculated using the same parameters as those in the search performed in step S103 in FIG. Further, since interpolation data is likely to be close to correct data, recalculation is performed using the interpolation data as an initial value.

  FIG. 12 shows a situation in which the corresponding data (indicated by black circles) of the surrounding pixels of the corresponding data (indicated by dotted circles) having a low reliability is high in reliability.

  Interpolated data (indicated by hatching circles) is created by interpolation from corresponding data of surrounding pixels (step S107 in FIG. 10). At this time, the interpolation data is evaluated (step S108 in FIG. 10).

  In the evaluation, the absolute value of the difference between the value indicated by the interpolation data and the corresponding data with low reliability is used. In FIG. 12, since the difference is small, it is less important to create interpolation data.

  Therefore, in this case, in step S109 in FIG. 10, the parameters in the search performed in step S103 in FIG. 9 are changed, and the POC is recalculated. At this time, recalculation is performed using the interpolation data (or the original corresponding data) as an initial value.

  If the interpolated data is close to the corresponding data before interpolation, even if recalculation is performed using the close interpolated data, it is highly likely that corresponding data with low reliability will be output again. It is to calculate. There is a possibility that a good peak of POC can be obtained by changing parameters and performing recalculation.

  FIG. 13 shows a situation where the reliability of the corresponding data of the pixels around the corresponding data with low reliability is low.

  In this case, interpolation data cannot be generated because there is no highly reliable data in the vicinity. In other words, the parameter can only be changed and recalculated. Further, since the initial value cannot be set, recalculation is performed from the beginning without using the initial value.

FIG. 14 is a flowchart showing the contents of the interpolation processing (S107) of FIG.
Referring to the figure, in step S201, the corresponding data (peripheral data) of pixels around the corresponding data with low reliability is scanned to determine whether there is any unprocessed peripheral data. If there is peripheral data, the reliability is evaluated in step S202. If the reliability is equal to or greater than the predetermined threshold, the number of reliable peripheral data is incremented by 1 in step S203.

  After the process in step S203, or when the reliability is low in step S202, the process returns to step S201.

  Note that the scan in step S201 targets a 3 × 3 pixel region, a 5 × 5 pixel region, or the like with the corresponding data as the center of gravity.

  If there is no peripheral data to be scanned in step S201, it is determined in step S204 whether the number counted in step S203 is larger than a predetermined threshold value. If NO, since there is little data necessary for interpolation, the reliability of the interpolation data cannot be secured, and the process ends without generating interpolation data in step S205.

  On the other hand, if “YES” in the step S204, the data necessary for the interpolation is sufficiently prepared, so that the interpolation data is generated in the step S206.

  In interpolation, statistics such as the average and median (median) of each data are used, or interpolation is performed in consideration of the positional relationship (distribution status) of each data with high reliability (internal interpolation, etc.). Insertion, extrapolation, etc.).

Alternatively, interpolation may be performed using only data having a reliability higher than a predetermined threshold value.
Step S206 in FIG. 14 corresponds to the situation in FIGS. 11 and 12, and step S205 corresponds to the situation in FIG.

  FIG. 15 is a flowchart showing the contents of the interpolation data evaluation process (S108) of FIG.

  In step S301, it is checked whether or not the generated interpolation data exists. When there is no interpolation data (in the case of FIG. 13), the process proceeds to step S304, and when there is interpolation data (in the case of FIGS. 11 and 12), the process proceeds to step S302.

  In step S302, the difference (or the absolute value thereof) between the interpolation data and the corresponding data is calculated.

  In step S303, when the calculated difference is large (when it is equal to or larger than the predetermined threshold), the process proceeds to step S305, and when small, the process proceeds to step S306.

  In step S305, recalculation is performed using the interpolation data as an initial value (S109) without changing the corresponding point search parameters (state in FIG. 11).

  In step S306, the corresponding point search parameters are changed, and recalculation is performed using the interpolation data as an initial value (S109) (state in FIG. 12).

  In step S304, the corresponding point search parameters are changed and recalculation is performed (S109) (state in FIG. 13).

16 and 17 are diagrams for explaining a method of generating interpolation data.
As shown in FIG. 16, in order to perform interpolation of the pixel c indicated by hatching, the pixel data of a, b, d, and e can be referred to. In this case, only highly reliable data among the pixel data of a, b, d, and e may be used.

  Also, as shown in the graph of FIG. 17, when the distribution of highly reliable data (data of pixels a and b) is biased, the average of pixels a and b is not used as interpolation data for pixel c. The extrapolated data may be used as the interpolation data.

  In the present embodiment, examples of the parameters include the window size and the cut frequency parameter of the low-pass filter, but are not limited thereto, and include parameters related to corresponding point search accuracy.

[Effects of the embodiment]
According to the above embodiment, in the corresponding point search method between two images, it is possible to realize high accuracy of processing for a point with low reliability. More specifically, peripheral data of corresponding point data with low reliability is evaluated to determine whether or not to generate interpolation data. As a result, high-quality interpolation data can be generated.

  Furthermore, if there is only peripheral data with low reliability, the corresponding points are searched by changing the parameters without generating interpolation data. Thereby, highly accurate association is possible.

  In addition, the interpolation data generated from the surrounding corresponding data is evaluated. Based on the evaluation result, the recalculation is performed by changing the parameter of the corresponding point search, so that the association can be performed with high accuracy.

[Others]
The processing in the above-described embodiment may be performed by software or by using a hardware circuit.

  In addition, a program for executing the processing in the above-described embodiment can be provided, and the program is recorded on a recording medium such as a CD-ROM, a flexible disk, a hard disk, a ROM, a RAM, and a memory card and provided to the user. You may decide to do it. The program may be downloaded to the apparatus via a communication line such as the Internet.

  In addition, it should be thought that the said embodiment is an illustration and restrictive at no points. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure which shows the structure of the three-dimensional measuring apparatus in one of the embodiment of this invention. It is a block diagram which shows the hardware constitutions of the arithmetic processing unit 200 of FIG. 3 is a block diagram illustrating a functional configuration of an arithmetic processing device 200. FIG. It is a figure for demonstrating the scan of an attention point. It is a figure which shows the window setting process of a reference | standard image. It is a figure which shows the window setting process of a reference image. It is a figure which shows matching with a reference | standard pixel and a reference pixel. It is a figure explaining the phase only correlation method as another method which the corresponding point search part 301 performs. It is a figure which shows the process which the arithmetic processing unit 200 performs. It is a flowchart following FIG. It is a figure which shows the specific example of the creation process of interpolation data. It is a figure which shows the specific example of the creation process of interpolation data. It is a figure which shows the specific example of the creation process of interpolation data. It is a flowchart which shows the content of the interpolation process (S107) of FIG. It is a flowchart which shows the content of the evaluation process (S108) of the interpolation data of FIG. It is a figure for demonstrating the production | generation method of interpolation data. It is a figure for demonstrating the production | generation method of interpolation data. It is the figure which showed typically the method of calculating a three-dimensional position from the corresponding point between images based on the principle of triangulation.

Explanation of symbols

  100R, 100L camera, 200 arithmetic processing unit, 201, 203 Fourier transform unit, 205, 207 normalization unit, 209 synthesis unit, 211 inverse Fourier transform unit, 300 display device.

Claims (16)

A corresponding point search means for associating corresponding points between the reference image and the reference image and calculating corresponding data;
Reliability determination means for determining the reliability of the corresponding data calculated by the corresponding point search means;
When the reliability determined by the reliability determination unit is lower than a predetermined threshold value, an interpolation data generation unit that generates interpolation data for the corresponding point based on the corresponding data around the corresponding point;
Interpolation data evaluation means for evaluating the generated interpolation data;
An image processing apparatus comprising: a recalculation unit that changes a parameter used in the corresponding point search unit based on an evaluation result of the interpolation data evaluation unit when recalculating the corresponding data of the corresponding point.   The image processing apparatus according to claim 1, wherein the interpolation data evaluation unit performs evaluation based on a difference between the interpolation data for the corresponding point and the corresponding data.   When the evaluation by the interpolation data evaluation means is larger than a predetermined threshold, the recalculation means uses the generated interpolation data as an initial value and uses the same parameters as those used by the corresponding point search means. The image processing apparatus according to claim 1, wherein the corresponding data of the corresponding points is recalculated.   When the evaluation by the interpolation data evaluation unit is smaller than a predetermined threshold value, the recalculation unit uses the generated interpolation data or the corresponding data for the corresponding point as an initial value, and the corresponding point search unit The image processing apparatus according to claim 1, wherein the corresponding data of the corresponding point is recalculated with a parameter different from the used parameter.   The image processing apparatus according to claim 1, wherein the parameter includes a parameter related to a window size used in the corresponding point search unit.   The image processing apparatus according to claim 1, wherein the parameter includes a parameter related to a cut frequency of a low-pass filter used in the corresponding point search unit. The interpolation data generation means includes correspondence data evaluation means for evaluating correspondence data around the corresponding point,
The image processing apparatus according to claim 1, wherein when the evaluation by the corresponding data evaluation unit is high, interpolation data for the corresponding point is generated based on corresponding data around the corresponding point.   The image processing apparatus according to claim 7, wherein the interpolation data evaluation unit does not perform evaluation when interpolation data is not generated.   The image processing apparatus according to claim 7 or 8, wherein when interpolation data for the corresponding point is not created, a parameter used in the corresponding point search unit is changed to recalculate the corresponding data of the corresponding point. . The corresponding data evaluation means evaluates a plurality of corresponding data around the corresponding point,
The interpolation data generation means determines that the evaluation data is high when the number of pieces of data having reliability equal to or higher than a predetermined threshold is greater than a predetermined number in a plurality of corresponding data around the corresponding point. The image processing device according to claim 7, wherein the image processing device is generated.   The image processing apparatus according to claim 1, wherein the interpolation data generation unit performs interpolation using only data having reliability of a predetermined threshold value or more.   The image processing apparatus according to claim 1, wherein the interpolation data generation unit generates interpolation data using an average, a median, and other statistics.   The image processing apparatus according to claim 1, wherein the interpolation data generation unit generates interpolation data in consideration of data distribution.   The image processing apparatus according to claim 1, wherein the corresponding point search unit associates corresponding points between a standard image and a reference image using a phase-only correlation method. A corresponding point search step for associating corresponding points between the reference image and the reference image and calculating corresponding data;
A reliability determination step of determining the reliability of the corresponding data calculated in the corresponding point search step;
When the reliability determined in the reliability determination step is lower than a predetermined threshold value, an interpolation data generation step for generating interpolation data for the corresponding point based on the corresponding data around the corresponding point;
An interpolation data evaluation step for evaluating the generated interpolation data;
A recalculation step of changing a parameter used in the corresponding point search step based on an evaluation result of the interpolation data evaluation step when recalculating the corresponding data of the corresponding point. Control method. A corresponding point search step for associating corresponding points between the reference image and the reference image and calculating corresponding data;
A reliability determination step of determining the reliability of the corresponding data calculated in the corresponding point search step;
When the reliability determined in the reliability determination step is lower than a predetermined threshold value, an interpolation data generation step for generating interpolation data for the corresponding point based on the corresponding data around the corresponding point;
An interpolation data evaluation step for evaluating the generated interpolation data;
Image processing for causing a computer to execute a recalculation step for changing a parameter used in the corresponding point search step based on an evaluation result of the interpolation data evaluation step when recalculating the corresponding data of the corresponding point Device control program.

Images