JP2008217053A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor which can perform alignment in sub-pixel by decreasing processing load by performing alignment with smaller number of hierarchies. <P>SOLUTION: An image input means of the image processor inputs two sheets of an image. A computing means computes a coordinate with minimum difference level from a difference level of grids based on a layer value, covering the image input by the image input means. A control means makes the coordinate with minimum difference level computed by the computing means output if the layer value is a desired layer value. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program.

High-speed and high-precision image alignment for panoramic image generation that overlays similar textures between multiple images, higher resolution of images using Super resolution technology, and motion compensation processing in moving image compression Technology is essential.
In particular, there is a demand for high-speed processing with the accuracy of alignment being the accuracy of sub-pixels.

  For example, Patent Document 1 has an object to provide an alignment method with sub-pixel accuracy that does not decrease accuracy even when an image is deformed due to the characteristics of an imaging device, noise, or the like. , A series of pixels having a large differential value of the input image is set as a fitting range, and in the function fitting means, a unimodal function is fitted to each fitting range to obtain an edge position with sub-pixel accuracy. The pixel precision alignment means performs alignment. In the correspondence candidate curve generation means, a plurality of corresponding edge points on the other image are connected by a curve to an edge point on one image, and a correspondence candidate curve is obtained. Are found, and the coordinates are regarded as sub-pixel precision displacements from one image to the other.

Further, as a technique related to these, for example, Patent Document 2 describes a technique for performing image alignment with sub-pixel accuracy. That is, by considering the N-dimensional similarity, the multi-parameter high-precision simultaneous estimation method in image sub-pixel matching, which enables simultaneous estimation of the correspondence parameter between images with a small amount of calculation, stably and at high speed, and multiple An object is to provide a parameter high-accuracy simultaneous estimation program, and an N-dimensional similarity value between images obtained at discrete positions becomes maximum or minimum on a straight line parallel to a certain parameter axis. It is disclosed that a subsampling position is obtained, an N-dimensional hyperplane that is closest to the obtained subsampling position is obtained for each parameter axis, and a subsampling grid estimated position is obtained from the intersection of the obtained hyperplanes. .
Japanese Patent No. 2897772 Table 2004-063991

If a high-precision alignment of sub-pixels is attempted to increase the resolution of an image, it takes a huge amount of processing time to calculate the degree of difference for each pixel. Further, the hierarchical search method has a problem that the processing time increases because the calculation of the degree of difference and the interpolation processing are required for the predetermined number of layers, and the accuracy decreases when the number of layers is reduced.
The present invention has been made in the background of such a background art, and by performing alignment with a smaller number of layers, it is possible to reduce the processing load and perform alignment with sub-pixels. An object of the present invention is to provide an image processing apparatus and an image processing program.

The gist of the present invention for achieving the object lies in the inventions of the following items.
[1] Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
An image processing apparatus comprising: control means for outputting the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating means if the layer value is a desired layer value.

[2] Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is not a desired layer value, the image processing apparatus comprises: a control unit that causes the minimum dissimilarity coordinate calculating unit to calculate a minimum dissimilarity coordinate from a grid of layer values reduced to an arbitrary number .

[3] The control unit according to [1] or [2], wherein the control unit quantizes the minimum dissimilarity coordinate calculated by the minimum dissimilarity coordinate calculation unit with the next reduced layer value. Image processing device.

[4] The image processing apparatus according to [1], [2], or [3], wherein the grid is pixels separated from each other by a pixel corresponding to the layer value.

[5] The minimum dissimilarity coordinate calculating means calculates the minimum dissimilarity coordinate in a grid having a minimum dissimilarity at the center of the grid. [1], [2], [3] or [4] ] The image processing apparatus as described in.

[6] The image according to [5], wherein the control unit performs a search within a layer in order to determine a grid having a minimum difference between the centers of the grids by the minimum difference calculation unit. Processing equipment.

[7] The search of the grid in the layer performed by the control means is ended when the difference of the central pixel is the minimum among the differences of the pixels separated by the pixels of the layer value. The image processing apparatus according to [6].

[8] The search of the grid in the layer performed by the control means is performed when the dissimilarity of the pixel at the center is not the smallest among the dissimilarities of the pixels separated by the pixels of the layer value. The image processing apparatus according to [6], wherein an uncalculated dissimilarity is calculated based on a dissimilarity between pixels separated by a pixel corresponding to a layer value centering on.

[9] The control means generates an interpolated image with the phase of the minimum dissimilarity coordinate calculated by the minimum dissimilarity coordinate calculating means,
The dissimilarity minimum coordinate calculation means calculates the dissimilarity of the grid in the next layer from the interpolated image generated by the control means and the minimum dissimilarity coordinates, and calculates the minimum dissimilarity coordinates of the layer. The image processing apparatus according to [1] or [2], which is characterized.

[10] An image processing system
Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is a desired layer value, the image processing program is made to function as a control unit that outputs the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating unit.

[11] An image processing system
Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is not a desired layer value, the image processing program causes the minimum dissimilarity coordinate calculating unit to function as a control unit that calculates the minimum dissimilarity coordinate from a grid of layer values reduced to an arbitrary number. .

  According to the image processing apparatus and the image processing program according to the present invention, compared with the case where the present configuration is not provided, by performing alignment with a smaller number of layers, the processing load is reduced and sub-pixels are reduced. Can be aligned.

<Background Technology in the Embodiment>
First, in order to facilitate understanding of the embodiment, the background technology in the embodiment and the situation when the background technology is realized will be described.
Patent Document 1 described above describes a method of obtaining a position at which a quadratic curve passing through three points is drawn by using a degree of difference at three integer positions and minimizing the quadratic curve.
In other words, as a method of performing alignment between two images with subpixel accuracy, a property (see FIG. 18B) in which the relationship between the position between images and the degree of difference is a downwardly convex quadratic curve is used. In order to obtain the convex minimum dissimilarity coordinates, a method has been proposed in which approximate coordinates are obtained by inferring from the peripheral dissimilarities.
However, in order to specify the periphery where the degree of difference that becomes convex is minimum, it is necessary to calculate the degree of difference for each pixel while shifting between images, and to specify the position where the degree of difference is minimum. The degree of difference includes a least square error between images, an absolute error, a correlation coefficient, and the like. In order to calculate the degree of difference at a certain position, it is necessary to perform raster scanning for two images. That is, it is necessary to calculate the degree of difference in the scanning order of all pixels.
If two images of 512 pixels × 512 pixels are aligned (see FIG. 18A), if the degree of difference is calculated by shifting the overlap from one pixel to one pixel unit, (511 + 512 + 511) × (511 + 512 + 511) = 2,353,156 It is necessary to perform calculation between the pixels of two huge images (see FIG. 18C), which is not practical.

Further, Patent Document 2 obtains a straight line connecting points where the dissimilarity is minimum on a plurality of horizontal lines parallel to the X axis, and connects a point where the dissimilarity is minimum on the plurality of horizontal lines parallel to the Y axis. It is described that a straight line is obtained and an intersection of two straight lines is used as an estimated value, and therefore 25 dissimilarity values are required.
That is, in order to perform high-speed alignment between two images with sub-pixel accuracy, a hierarchical method has been proposed for the purpose of simplifying the number of differences calculation.
In this method, the minimum dissimilarity coordinates in a narrower range are obtained in the next hierarchy centering on the pixels having the minimum dissimilarity of 3 × 3 9 pixels in a certain hierarchy.
Therefore, since the degree of difference is calculated only on a predetermined grid, the number of hierarchies becomes large for high accuracy. In the subpixel hierarchy, an interpolated image is generated for each hierarchy.
If the position is adjusted with an accuracy of 0.1 pixel or less, it is necessary to narrow down hierarchically in 0.5 pixel steps in order to search without fail. For this reason, when starting with 4 pixels, 7 levels of hierarchy from 4 pixels to 0.0625 pixels are required (see FIG. 19). During this time, 9 times + 6 steps × 8 times = 57 times of calculation of difference and 3 times of interpolation image generation processing are required, and the processing time is still not practical. FIG. 19 shows a hierarchical situation, in which the white square is a hierarchy of four pixels. Hereinafter, the white triangle is two pixels, the black circle is one pixel, the black square is 0.5 pixel, and the black triangle. Represents 0.125 pixel and white circle represents 0.0625 pixel (note that 0.25 pixel is omitted).

<Basic architecture>
If the difference between the two images (vertical axis direction) and the position between the images (horizontal axis direction) are plotted in a graph, the graph becomes convex downward (see, for example, FIG. 18B, becomes two-dimensional). In the case of a secondary phase graph). By utilizing this property, a quadratic curved surface passing through nine dissimilarities as in S0 to S8 shown in FIG. 1A can be estimated, and the minimum coordinates of the dissimilarities can be obtained.
By calculating the nine points of this method with an n pixel width, it is possible to perform rough shake of the minimum dissimilarity coordinates even if the accuracy is poor. Here, n is a layer value. Therefore, the grid has 3 × 3 9 pixels separated from each other by the layer value.
Based on the obtained minimum dissimilarity coordinates, further calculating the minimum dissimilarity coordinates with 9 points of the nm pixel width can obtain coordinates with higher accuracy (see FIG. 1B).
By calculating the optimum values for n and m according to the image and setting the number of repetitions, the processing time can be reduced while maintaining high accuracy.
Hereinafter, the coordinates for obtaining the minimum dissimilarity coordinates are referred to as a grid. That is, the nine coordinates S0 to S8 for obtaining the minimum dissimilarity coordinates shown in FIG. Normally, the minimum difference coordinates are calculated with the accuracy of sub-pixels by setting the grid coordinates to one pixel. This n is called a layer.

<First Embodiment>
In the following, various preferred embodiments for realizing the present invention will be described with reference to the drawings.
FIG. 2 is a conceptual module configuration diagram of the first embodiment.
The module generally refers to a component such as software or hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a program but also a module in a hardware configuration. Therefore, the present embodiment also serves as an explanation of a program, a system, and a method. In addition, the modules correspond almost one-to-one with the functions. However, in mounting, one module may be composed of one program, or a plurality of modules may be composed of one program. A plurality of programs may be used. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. In the following, “connection” includes not only physical connection but also logical connection (data exchange, instruction, etc.).
The system includes a configuration in which a plurality of computers, hardware, devices, and the like are connected by communication means such as a network, and also includes a case where the system is realized by a single computer, hardware, devices, and the like.

The first embodiment includes a layer value determination module 21, a layer grid dissimilarity calculation module 22, a dissimilarity minimum coordinate calculation module 23, and a quantization module 24.
As shown in FIG. 2, the layer value determination module 21 is connected to the layer grid dissimilarity calculation module 22 and the quantization module 24, and the layer initial value (n) and transition value as layer information in advance. (M) is determined. These are transferred to the grid difference calculation module 22 and the quantization module 24 of the layer. However, the value of the layer information may be a fixed value or a dynamic and variable value.
As shown in FIG. 2, the layer grid dissimilarity calculation module 22 is connected to the layer value determination module 21, the quantization module 24, and the minimum dissimilarity coordinate calculation module 23, and inputs two images. The layer information is received from the layer value determination module 21 and the difference between the layer grids is calculated based on the layer value and the image data. The calculated difference is passed to the minimum difference coordinate calculation module 23. Note that inputting an image specifically includes inputting an image by a scanner, inputting an image from an external device via a communication line by fax, and inputting an image from a storage medium.

As shown in FIG. 2, the minimum dissimilarity coordinate calculation module 23 is connected to the difference calculation module 22 of the layer grid and the quantization module 24, and the grid calculated by the dissimilarity calculation module 22 of the layer grid is used. The minimum difference degree coordinate of the layer is calculated based on the difference degree. Then, the minimum dissimilarity coordinates are passed to the quantization module 24. Alternatively, under the control of the quantization module 24, the minimum dissimilarity coordinates are transferred to a processing unit that synthesizes an image. The processing performed by the minimum difference coordinate calculation module 23 will be described later with reference to FIGS. 5, 6, and 7.
As shown in FIG. 2, the quantization module 24 is connected to a layer value determination module 21, a layer grid dissimilarity calculation module 22, and a minimum dissimilarity coordinate calculation module 23. The layer information and the minimum dissimilarity coordinate calculated by the minimum dissimilarity coordinate calculating module 23 are received, and the quantized value of the coordinate value is passed to the dissimilarity calculating module 22 of the layer grid. That is, if the layer is not the target layer (for example, layer = 1), the minimum dissimilarity coordinate value calculated by the minimum dissimilarity coordinate calculation module 23 is quantized in the next layer, and the grid difference of the layer is again determined. Fall back to the difference calculation module 22 of the grid of the layer for calculating the degree. When the minimum dissimilarity coordinate value is calculated in the target layer, the value is output. The processing by the quantization module 24 will be described later with reference to FIG.

The processing and operation (operation) according to the first embodiment will be described with reference to the flowchart shown in FIG.
In step S302, the layer value determination module 21 determines the layer value (n). That is, the layer value is substituted for the variable n.
In step S304, the layer grid dissimilarity calculation module 22 determines the grid of the layer (n) determined in step S302 or step S310.
In step S306, the minimum dissimilarity coordinate calculation module 23 calculates the minimum dissimilarity coordinate of the layer (n) determined in step S304.

In step S308, it is determined whether the quantization module 24 is the target layer. In this determination, if it is the target layer, the process proceeds to step S314. If it is not the target layer, the process proceeds to step S310, and the processes in steps S304 and S306 are repeated.
In step S310, the quantization module 24 determines the next layer. That is, the transition value (m or the layer value change value) is subtracted from the current layer (n) to obtain the next layer (n). More specifically, the current layer (n) −transition value (m) is substituted for the variable n.
In step S312, the quantization module 24 quantizes the minimum dissimilarity coordinates in the layer (n) determined in step S310. Then, the process returns to step S304.
In step S314, since the target layer has been reached, the quantization module 24 sends the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating module 23 to a processing unit that combines images.

With reference to FIG. 4, the quantization processing of the minimum dissimilarity coordinates by the quantization module 24 and the like will be described in more detail.
For example, a procedure for quantizing the minimum dissimilarity coordinates calculated in layer n = 3 in layer n = 1 will be described below. The transition value m = 2.
The closest grid of layer n = 1 with respect to the minimum difference coordinates (black circles shown in FIG. 4) calculated using the nine differences of the grids (white circles shown in FIG. 4) obtained in layer n = 3 (Black triangle shown in FIG. 4) is calculated and sent to the grid difference calculation module 22 of the layer as quantized coordinates.
After that, by the following processing, based on the dissimilarity of the nine-point grid (white triangle shown in FIG. 4) of layer n = 1 around the quantized difference degree minimum coordinate (black triangle shown in FIG. 4), Calculate the minimum dissimilarity coordinates.

There are several methods for calculating the coordinate that minimizes the dissimilarity, which is a process performed by the minimum dissimilarity coordinate calculating module 23.
(1) Method 1 (Technique described in Patent Document 1)
As shown in FIG. 5, when the values of three points at integer positions are known, a quadratic curve passing through the three points is drawn to obtain a position (optimum value) at which the quadratic curve is minimized.
(2) Method 2 (Technique described in Patent Document 2)
Find the value of x that minimizes the dissimilarity on a horizontal line. Then, such a value of x is obtained on a plurality of horizontal lines. A straight line passing through this point (white circle in FIG. 6) is drawn.
Further, a value of y that minimizes the degree of difference on a certain vertical line is obtained. Then, such a value of y is obtained on a plurality of vertical lines. A straight line passing through this point (black circle in FIG. 6) is drawn.
The optimum value for obtaining the intersection of the two straight lines obtained in this way is used.
(3) Method 3
A quadric surface that roughly passes through nine points of the grid for obtaining coordinates with the smallest difference is estimated (indicated by a broken line in FIG. 7 is an estimated quadric surface), and this curved surface is minimized (x, y). Find the value of. That is, an input image is converted on an integer grid, a coefficient of a quadric surface that approximates a function for obtaining a difference degree is calculated, and the calculated coefficient is estimated with sub-quantization accuracy. Then, in the case of approximation, weighting for each integer grid position may be performed, and the weighting may be an area or a super volume of a Voronoi divided region centered on each integer grid.

<Second Embodiment>
A second embodiment will be described with reference to FIGS.
As shown in FIG. 9, among the grids from S0 to S8 (S0 is located at the center of 9 points), the grid having the smallest S0 is called a minimum grid. Calculation of the minimum dissimilarity coordinates using the dissimilarity of the minimum grid is an interpolation, and therefore the accuracy is further increased. In the second embodiment, the minimum grid is obtained, and the accuracy is improved by using the minimum grid.
The second embodiment includes a layer value determination module 81, a minimum grid difference calculation module 82, a minimum difference coordinate calculation module 83, and a quantization module 84.
As shown in FIG. 8, the layer value determining module 81 is connected to the layer minimum grid dissimilarity calculating module 82 and the quantizing module 84, and the layer value determining module 21 of the first embodiment is connected to the layer value determining module 81. It is the same.
As shown in FIG. 8, the minimum grid dissimilarity calculation module 82 is connected to a layer value determination module 81, a quantization module 84, and a minimum dissimilarity coordinate calculation module 83, and inputs two images. Then, the layer information is received from the layer value determination module 81, the minimum grid of the layer is determined based on the layer value and the image data, and the dissimilarity of the minimum grid is calculated. The calculated difference is passed to the minimum difference calculation module 83.

As shown in FIG. 8, the minimum dissimilarity coordinate calculation module 83 is connected to the dissimilarity calculation module 82 and the quantization module 84 of the minimum grid of the layer, and the minimum dissimilarity coordinate calculation module of the first embodiment. 23. However, the minimum dissimilarity coordinate is calculated based on the dissimilarity of the minimum grid calculated by the minimum grid dissimilarity calculation module 82 of the layer.
As shown in FIG. 8, the quantization module 84 is connected to a layer value determination module 81, a minimum grid dissimilarity calculation module 82, and a minimum dissimilarity coordinate calculation module 83, which is the first embodiment. This is the same as the quantization module 24 in FIG. However, the target grid is the minimum grid. Processing performed by the quantization module 84 will be described later with reference to FIG.

The process and operation (operation) according to the second embodiment will be described with reference to the flowchart shown in FIG.
In step S1002, the layer value determination module 81 determines the layer value (n). That is, the layer value is substituted for the variable n.
In step S1004, the difference calculation module 82 of the minimum grid of the layer searches and determines the minimum grid from the grid of the layer (n) determined in step S1002 or step S1010. The search for the minimum grid will be described later with reference to FIGS.
In step S1006, the minimum dissimilarity coordinate calculation module 83 calculates the minimum dissimilarity coordinate of the layer (n) determined in step S1004.

In step S1008, it is determined whether the quantization module 84 is the target layer. In this determination, if it is the target layer, the process proceeds to step S1014. If it is not the target layer, the process proceeds to step S1010, and the processes in steps S1004 and S1006 are repeated.
In step S1010, the quantization module 84 determines the next layer. That is, the transition value (m) is subtracted from the current layer (n) to obtain the next layer (n). More specifically, the current layer (n) −transition value (m) is substituted for the variable n.
In step S1012, the quantization module 84 quantizes the dissimilarity minimum coordinate in the layer (n) determined in step S1010. Then, the process returns to step S1004.
In step S1014, since the target layer has been reached, the quantization module 84 sends the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculation module 83 to a processing unit that synthesizes an image.

With reference to FIG. 11, the quantization processing of the minimum dissimilarity coordinates by the quantization module 84 and the like will be described in more detail.
For example, a procedure for quantizing the minimum dissimilarity coordinates calculated in layer n = 3 in layer n = 1 will be described below. The transition value m = 2.
With respect to the minimum dissimilarity coordinates (black circles shown in FIG. 11) calculated using the nine differences of the minimum grid (white circles shown in FIG. 11) obtained in layer n = 3, the closest layer n = 1 A grid (black triangle shown in FIG. 11) is calculated and sent to the difference calculation module 82 of the minimum grid of the layer as quantized coordinates.
After that, the following processing determines whether the nine-point grid (white triangle shown in FIG. 11) of layer n = 1 is the minimum grid around the quantized minimum difference coordinates (black triangle shown in FIG. 11). I do. Then, the minimum dissimilarity coordinates are calculated.

The search procedure for the minimum difference grid in a certain layer will be described in more detail with reference to the flowchart shown in FIG. 12 and FIG.
In step S1202, the difference calculation module 82 of the minimum grid of the layer calculates the difference for the initial grid.
Hereinafter, the processing is varied depending on which of the three cases shown in FIG.
In step S1204, it is determined whether or not the grid of 9 points of interest is case 1. If it is case 1, the process ends (step S1212). If it is not case 1, the process proceeds to step S1206. In other words, when the smallest value of the differences between the nine grids of interest (initially the initial grid) is the center of the nine points (gray circle in case 1 shown in FIG. 13A), Since the 9-point grid is the smallest grid, the process is terminated. That is, the process ends when the difference between the pixels at the center is the smallest among the differences between the pixels separated by the pixel of the layer value.

In step S1206, it is determined whether or not the grid of 9 points of interest is Case 2. If it is case 2, the process proceeds to step S1210. If it is not case 2, the process proceeds to step S1208. In other words, if it is not case 1, the dissimilarity of the grid at one point out of the four points near the center (gray circle of case 2 shown in FIG. 13B) in the nine-point grid is minimum (for example, FIG. 13). It is determined whether or not it is a thick gray circle in case 2 shown in (B) (whether it is case 2).
In step S1210, since it is determined in step S1206 that it is case 2, the outside of the minimum dissimilarity coordinate of case 2 is searched. An uncalculated dissimilarity is calculated based on the dissimilarity of pixels separated by a pixel corresponding to the layer value with the minimum dissimilarity coordinate as the center. That is, the dissimilarity of the grid (three black circles in case 2 shown in FIG. 13B) that is not calculated in step S1202 is calculated from the nine dissimilarities centered on the minimum dissimilarity coordinates. Then, the process returns to step S1204.

  In step S1208 (when neither case 1 nor 2), the outside of the minimum difference degree coordinate of case 3 is searched. An uncalculated dissimilarity is calculated based on the dissimilarity of pixels separated by a pixel corresponding to the layer value with the minimum dissimilarity as the center. That is, in the nine-point grid, one of the diagonal positions with respect to the center (the gray circle in case 3 shown in FIG. 13C) has the smallest difference (for example, as shown in FIG. 13C). Of the nine dissimilarities centered on the minimum dissimilarity coordinates, the grid that is not calculated in step S1202 (the five black circles of case 3 shown in FIG. 13C) is the bold gray circle in case 3). ) Is calculated. Then, the process returns to step S1204.

<Third Embodiment>
A third embodiment will be described with reference to FIGS.
The difference from the above-described embodiment is that the minimum difference degree coordinates are not quantized in the next layer, the interpolation image is generated with the calculated phase of the difference degree calculation coordinates, and the difference level of the next layer is minimized. This is the point where the coordinates are obtained. Although the third embodiment depends on the target, the processing load may be smaller than in other embodiments.
The third embodiment includes a layer value determination module 141, a layer grid dissimilarity calculation module 142, a dissimilarity minimum coordinate calculation module 143, and an interpolated image creation module 144.
As shown in FIG. 14, the layer value determination module 141 is connected to the layer grid dissimilarity calculation module 142 and the interpolated image creation module 144. The layer value determination module 141 includes the layer value determination module 21 according to the first embodiment. It is the same.
As shown in FIG. 14, the layer grid dissimilarity calculation module 142 is connected to the layer value determination module 141, the interpolated image creation module 144, and the minimum dissimilarity coordinate calculation module 143, and is described in the first embodiment. This is the same as the difference calculation module 22 of the grid of the layer or the difference calculation module 82 of the minimum grid of the layer according to the second embodiment.

As shown in FIG. 14, the minimum dissimilarity coordinate calculation module 143 is connected to the dissimilarity calculation module 142 of the layer grid and the interpolated image creation module 144, and the minimum dissimilarity coordinate calculation module of the first embodiment. 23.
As shown in FIG. 14, the interpolated image creation module 144 is connected to a layer value determination module 141, a grid difference calculation module 142, and a minimum difference coordinate calculation module 143. The minimum dissimilarity coordinate calculated by 143 is received, and the coordinate value and the interpolated image are passed to the dissimilarity calculating module 142 of the layer grid. That is, if the layer is not a target layer (for example, layer = 1), an interpolated image is generated based on the phase of the minimum dissimilarity coordinate value calculated by the minimum dissimilarity coordinate calculation module 143, and the minimum dissimilarity coordinate and the interpolation are generated. The process returns to the layer grid difference calculation module 142 that calculates the layer grid difference based on the image. When the minimum dissimilarity coordinate value is calculated in the target layer, the value is output. Interpolation image generation processing by the interpolation image creation module 144 will be described later with reference to FIG.

The processing and operation (operation) according to the third embodiment will be described with reference to the flowchart shown in FIG.
In step S1502, the layer value determination module 141 determines the layer value (n). That is, the layer value is substituted for the variable n.
In step S1504, the difference calculation module 142 of the layer grid determines the grid of the layer (n) determined in step S1502 or step S1510.
In step S1506, the minimum dissimilarity coordinate calculation module 143 calculates the minimum dissimilarity coordinate of the layer (n) determined in step S1504.

In step S1508, it is determined whether or not the target layer is the interpolation image creation module 144. In this determination, if it is the target layer, the process proceeds to step S1514. If it is not the target layer, the process proceeds to step S1510, and the processes in steps S1504 and S1506 are repeated.
In step S1510, the interpolation image creation module 144 determines the next layer. That is, the transition value (m) is subtracted from the current layer (n) to obtain the next layer (n). More specifically, the current layer (n) −transition value (m) is substituted for the variable n.
In step S1512, the interpolated image creation module 144 generates an interpolated image with the phase of the minimum dissimilarity coordinate in the layer (n) determined in step S1510. Then, the process returns to step S1504.
In step S1514, since the target layer has been reached, the interpolation image creation module 144 sends the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculation module 143 to a processing unit that synthesizes images.

An interpolation image generation process based on the phase of the minimum difference degree coordinate will be described with reference to FIG.
For example, a procedure for generating an interpolation image from the phase of the minimum difference degree coordinate in layer n = 3 and calculating the difference degree of the grid in layer n = 1 will be described below. The transition value m = 2.
Interpolation with this coordinate value as the phase for the minimum dissimilarity coordinates (black circles shown in FIG. 16) calculated using the 9 points of dissimilarity of the grid (white circles shown in FIG. 16) obtained in layer n = 3 An image is generated (see grid with thin dotted lines in FIG. 16).
Thereafter, the grid dissimilarity is calculated by using the grid of layer n = 1 (triangle shown in FIG. 16) by the following processing.

<Implementation of the embodiment>
According to the above embodiment, it is possible to align two images at high speed while maintaining the accuracy of the sub-pixel unit.
For example, in the case of performing alignment of two images of 512 pixels × 512 pixels, the background art requires calculation between 2,353 and 156 times, but for example, n = 255, m = 254 (the next n 1 is an example of 2 hierarchies), the calculation between pixels is 9 points × 2 hierarchies = 18 times, and 130,730 times higher speed can be expected.
Actually, if the initial value of n is large, the accuracy of the minimum coordinate of the dissimilarity of the next layer is lowered and the search processing of the minimum grid is increased. For this reason, in order to increase the accuracy, even if n = 4 and m = 3, the first layer is 128 × 128 points = 16,384 times (actually, the search process is less because the search reaches the shortest to the minimum grid). This is a total of 16,393 times with 9 times in the second layer, which is a speed increase of 143 times or more.
Further, assuming that the accuracy of 0.1 pixel is compared with the hierarchization method shown in the background art, the hierarchization method starts with n = 4, and the hierarchization method calculates 9 times + 6 steps × 8 times = 57 times and 3 It is necessary to generate the interpolated image twice, and in particular, the processing time is taken to generate the interpolated image. In this embodiment, n = 4 and m = 3, and the calculation ends only with 9 times + 9 times = 18 times.
Further, by controlling the layers as described above, optimal control can be performed within the trade-off position accuracy and processing time.
In the above embodiment, the center of the image may be determined as the initial grid. Further, the center of gravity of the pixel having the largest pixel value in the image may be determined as the initial grid. By doing so, it is possible to reduce the hierarchy and converge faster.

<Hardware configuration example>
With reference to FIG. 17, a hardware configuration example of the above-described embodiment will be described. The configuration shown in FIG. 17 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 1717 such as a scanner and a data output unit 1718 such as a printer.

  The CPU (Central Processing Unit) 1701 includes various modules described in the above-described embodiments, that is, the modules such as the layer grid dissimilarity calculation module 22, the dissimilarity minimum coordinate calculation module 23, and the quantization module 24. It is a control part which performs a process according to the computer program which described the execution sequence.

  A ROM (Read Only Memory) 1702 stores programs used by the CPU 1701, calculation parameters, and the like. A RAM (Random Access Memory) 1703 stores programs used in the execution of the CPU 1701, parameters that change as appropriate in the execution, and the like. These are connected to each other by a host bus 1704 including a CPU bus.

  The host bus 1704 is connected to an external bus 1706 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1705.

  A keyboard 1708 and a pointing device 1709 such as a mouse are input devices operated by an operator. The display 1710 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 1711 has a built-in hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 1701 and information. The hard disk stores input images and synthesized images. Further, various computer programs such as various other data processing programs are stored.

  The drive 1712 reads out data or a program recorded on a removable recording medium 1713 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out as an interface 1707 and an external bus 1706. , The bridge 1705, and the RAM 1703 connected via the host bus 1704. The removable recording medium 1713 can also be used as a data recording area similar to the hard disk.

  The connection port 1714 is a port for connecting the external connection device 1715 and has a connection unit such as USB or IEEE1394. The connection port 1714 is connected to the CPU 1701 and the like via an interface 1707, an external bus 1706, a bridge 1705, a host bus 1704, and the like. A communication unit 1716 is connected to the network and executes data communication processing with the outside. The data reading unit 1717 is a scanner, for example, and executes document reading processing. The data output unit 1718 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration illustrated in FIG. 17 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 17, and is a configuration capable of executing the modules described in the present embodiment. I just need it. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line Alternatively, a plurality of the systems shown in FIG. 17 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (also called a multi-function copying machine, which has functions of a scanner, a printer, a copying machine, a fax machine, etc.).

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standards such as “DVD + R, DVD + RW, etc.”, compact discs (CDs), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc. MO), flexible disk (FD), magnetic tape, hard disk, read only memory (ROM), electrically erasable and rewritable read only memory (EEPROM), flash memory, random access memory (RAM), etc. It is.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

It is a conceptual explanatory drawing of this Embodiment. It is a conceptual module block diagram about the structural example of 1st Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is a conceptual explanatory drawing about quantization of the minimum difference degree coordinate by 1st Embodiment. It is explanatory drawing which shows one of the methods of calculating a dissimilarity minimum coordinate. It is explanatory drawing which shows one of the methods of calculating a dissimilarity minimum coordinate. It is explanatory drawing which shows one of the methods of calculating a dissimilarity minimum coordinate. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is explanatory drawing which shows an example of a grid. It is a flowchart which shows the process example by 2nd Embodiment. It is a conceptual explanatory drawing about quantization of the minimum difference degree coordinate by 2nd Embodiment. It is a flowchart which shows the process example which searches the minimum grid by 2nd Embodiment. It is explanatory drawing of the example of a process which searches for a minimum grid. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is a flowchart which shows the process example by 3rd Embodiment. It is a conceptual explanatory drawing about quantization of the minimum difference degree coordinate by 3rd Embodiment. And FIG. 11 is a block diagram illustrating a hardware configuration example of a computer that implements an embodiment. It is explanatory drawing which shows background art. It is explanatory drawing which shows background art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 21 ... Layer value determination module 22 ... Layer grid difference calculation module 23 ... Difference minimum coordinate calculation module 24 ... Quantization module 81 ... Layer value determination module 82 ... Layer minimum grid difference calculation module 83 ... Difference minimum coordinate calculation module 84 ... Quantization module 141 ... Layer value determination module 142 ... Layer grid difference calculation module 143 ... Difference difference minimum coordinate calculation module 144 ... Interpolated image creation module

Claims (11)

Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
An image processing apparatus comprising: control means for outputting the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating means if the layer value is a desired layer value. Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is not a desired layer value, the image processing apparatus comprises: a control unit that causes the minimum dissimilarity coordinate calculating unit to calculate a minimum dissimilarity coordinate from a grid of layer values reduced to an arbitrary number . 3. The image processing apparatus according to claim 1, wherein the control unit quantizes the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating unit with the next reduced layer value. The image processing apparatus according to claim 1, wherein the grid includes pixels separated from each other by a pixel corresponding to a layer value. 5. The image processing apparatus according to claim 1, wherein the dissimilarity minimum coordinate calculating unit calculates a dissimilarity minimum coordinate in a grid having a minimum dissimilarity at the center of the grid. The image processing apparatus according to claim 5, wherein the control unit performs a search in a layer in order to determine a grid having a minimum difference in the center of the grid by the minimum difference coordinate calculation unit. The search of the grid in the layer performed by the control means is terminated when the difference of the central pixel is the minimum among the differences of pixels separated by the pixel of the layer value. Item 7. The image processing apparatus according to Item 6. The search of the grid in the layer performed by the control means is performed by focusing on the coordinates having the smallest difference when the difference of the central pixel is not the smallest among the differences of the pixels separated by the pixel of the layer value. The image processing apparatus according to claim 6, wherein an uncalculated difference is calculated based on a difference between pixels separated by a pixel corresponding to a layer value. The control means generates an interpolated image with the phase of the minimum dissimilarity coordinate calculated by the minimum dissimilarity coordinate calculating means,
The dissimilarity minimum coordinate calculation means calculates the dissimilarity of the grid in the next layer from the interpolated image generated by the control means and the minimum dissimilarity coordinates, and calculates the minimum dissimilarity coordinates of the layer. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Image processing system
Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is a desired layer value, the image processing program is made to function as a control unit that outputs the minimum dissimilarity coordinates calculated by the minimum dissimilarity coordinate calculating unit. Image processing system
Image input means for inputting two images;
For the image input by the image input means, a difference degree minimum coordinate calculation means for calculating a difference degree minimum coordinate from a grid difference degree based on a layer value;
If the layer value is not a desired layer value, the image processing program causes the minimum dissimilarity coordinate calculating unit to function as a control unit that calculates the minimum dissimilarity coordinate from a grid of layer values reduced to an arbitrary number. .

Images