JPH04299474A - Picture matching system - Google Patents

Abstract

PURPOSE:To correctly perfom the correspondence even between pictures having occulusion by bi-directionlly performing the matching at the time of the matching for performing the correspondence of two pictures. CONSTITUTION:The correlation of the picture elements between the blocks of two pictures A and B with each other is calculated by a correlation calculation means 13. The ordering by correlation value corresponding to each of the direction from A to B and the direction from B to A is performed by correlation factor ordering means 15. The successive sum of hte direction from A to B and the direction from B to A are determined by a correlation factor order addition means 16. The corresponding combination to what the sum of the order is integrated and a fixed penalty in accordane with the number of block having no correspondence is added becomes minimum is determined by a minimum addition order selection means 17.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image matching for stereo matching and motion vector detection.

0002

[Prior Art] Conventionally, among the techniques related to image matching, a method using block matching was developed
Publication No. 2-107386 “Image matching method”, U.S. Patent No. 4677476 “METHOD FOR
DETECTING A MOVEMENT
OF A TELEVISION SIGNA
The techniques described in "L" are known. In these methods, when performing matching processing on two pieces of image data stored in a storage device, one of the images is divided into multiple blocks. , while each block is shifted in parallel to calculate the cross-correlation with the other image, and the block where the correlation is maximum is estimated to be the block corresponding to the other image.

0003

[Problems to be Solved by the Invention] However, conventional block matching has a problem in that correct matching cannot be performed in occlusion areas. In other words, as shown in FIG. 5, if an object in area a in image A moves to area c in image B, area b marked with diagonal lines in image A will move to area c in image B. Then, it is hidden in area c. In this case, there is no region in image B that corresponds to region b. Generally, such a region b is called an occlusion region.

When image A is divided into blocks and matched with image B, the blocks included in area b originally have no counterpart. Despite this, in the conventional method, the point where the correlation is maximum is determined to be a corresponding point. For example, block (a) included in area b corresponds to block (b), which is an undesirable correspondence.

An object of the present invention is to provide a matching method that provides accurate correspondence in the sense that it does not provide correspondence to occlusion areas.

[0006]

[Means for Solving the Problems] The image matching method of the first invention has the following features: a. Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means f for ranking in ascending order, and ranking in ascending order from the blocks on the B image to the blocks on the A image in the order of the larger correlation coefficient.
．． Correlation from blocks on image A to blocks on image B, obtained by the correlation coefficient ranking means, for each correspondence between blocks on images A and B calculated by the correlation calculation means. correlation coefficient rank adding means for calculating the sum of the rank of the number and the rank of the correlation coefficient from the same block on the B image to the same block on the A image g. For all combinations of blocks on images A and B calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and The present invention is characterized in that it includes a minimum addition rank selection means that adds a certain penalty according to the number of blocks not included in the correspondence to the addition result of the combination and finds a combination that minimizes the result.

The image matching method of the second invention includes a.
Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means f for ranking in ascending order, and ranking in ascending order from the blocks on the B image to the blocks on the A image in the order of the larger correlation coefficient.
．． g. cost calculating means for converting the ranking value into a cost value using a monotonically increasing function given in advance for the ranking obtained by the correlation coefficient ranking means; g. The cost from the block on the A image to the block on the B image obtained by the cost calculation means for each correspondence between the blocks on the A and B images calculated by the correlation calculation means, and the B image Cost addition means for calculating the sum of costs from the same block on the A image to the same block on the A image h. For all combinations of correspondences between blocks on images A and B calculated by the correlation calculation means, the sum of ranks obtained by the cost addition means is added for each combination, and further the correspondences in each combination are added. The present invention is characterized in that it includes a minimum addition cost selection means that adds a certain penalty according to the number of blocks not included in the combination to the addition result of the combination and finds a combination that minimizes the result.

The image matching method of the third invention includes a.
Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the correlation coefficient values obtained by the correlation calculation means for each correspondence between blocks on the A image and blocks on the B image, the correlation coefficient is calculated from the block on the A image to each block on the B image. A and B direction correlation coefficient ranking and correspondence determining means f. sequentially ranking a certain number of blocks in ascending order, and determining that there is no possibility of correspondence between blocks that are not ranked; f. Using the correlation coefficient values obtained by the correlation calculation means for each correspondence between blocks on the A image and blocks on the B image, the correlation coefficient is calculated from the block on the B image to each block on the A image. g. BA direction correlation coefficient ranking and correspondence determining means for ranking a certain number of blocks in ascending order and determining that there is no possibility of correspondence between blocks that are not ranked; g. Among the correspondence between the blocks on the A and B images calculated by the correlation calculation means, the AB direction correlation coefficient ranking and correspondence determination means and the BA direction correlation coefficient ranking and correspondence determination means , the ranking obtained by the AB direction correlation coefficient ranking and determination means and the rank obtained by the BA direction correlation coefficient ranking and determination means, excluding correspondences for which it is determined that there is no possibility of correspondence. Correlation coefficient rank addition means for calculating the sum of h. For all combinations of corresponding blocks on images A and B calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and The AB direction correlation coefficient ranking and determination means and the BA direction correlation The present invention is characterized by comprising minimum addition rank selection means for determining from among the combinations of correspondences that are not determined to have no possibility of correspondence by the numerical ranking and determination means.

The image matching method of the fourth invention includes a.
Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing images A and B stored in the image storage means into each scanning line, and further dividing the scanning lines into segments by allowing overlap; d. Correlation calculating means e that calculates the cross-correlation of pixel values for all or part of the correspondence between the segments on the A image and the segments on the B image generated by the image dividing means, which exist on the scanning line at the same position. ．． Regarding the correspondence between the segments on the A and B images calculated by the correlation calculation means, the value of the correlation coefficient for each correspondence is used, and the correlation coefficient from the segment on the A image to the segment on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and ranking in ascending order of correlation coefficients from segments on the B image to segments on the A image, f. Regarding each correspondence between the segments on images A and B calculated by the correlation coefficient ranking means, the ranking of the correlation coefficient from the segment on image A to the segment on image B, and the correlation coefficient on image B Correlation coefficient rank adding means for calculating the sum of the ranks of correlation coefficients from the segment to the segment on the A image g. A calculated by the correlation coefficient rank addition means
, A path generating means for generating not only paths between adjacent correspondences but also paths between non-adjacent correspondences regarding the correspondence between segments on the B image h. For all the corresponding combinations expressed by connecting the paths generated by the path generation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and The present invention is characterized in that it includes a minimum addition rank selection means that adds a certain penalty according to the number of blocks not included in the correspondences and uses dynamic programming to find a combination of correspondences that minimizes the result.

0010

[Operation] In the image matching method of the present invention, occlusion areas are detected by performing bidirectional matching. As a result, accurate correspondence is achieved in the sense that blocks included in the occlusion area are not given corresponding blocks.

[0011] This will be explained using FIG. Here, image A
Suppose that area a in the image B has moved to area d in image B. Area b marked with diagonal lines in image A and area c marked with diagonal lines in image B are occlusion areas. Here, one of the blocks in b is assumed to be (a).

First, both image A and image B are divided into blocks, and the mutual relationships are calculated for every correspondence of blocks. Next, rank each block on the A image to each block on the B image in ascending order of the correlation coefficient value,
Then, each block on the B image to each block on the A image is ranked in ascending order of the correlation coefficient value. Here, it is assumed that block (B) has the largest correlation coefficient with respect to block (A), and the rank of (A) with respect to (B) is set to 1. In the conventional method, it is determined at this point that the corresponding point to (a) is (b).

[0013] In the image matching method of the present invention, the order of the correlation coefficient from a block on the A image to a block on the B image and the correlation from the same block on the B image to the same block on the A image are further determined. Calculate the sum of numbers and ranks. In the case of FIG. 6, it is assumed that the (c) block on the A image and the (b) block on the B image actually correspond. In that case,(
It is thought that the ranking from (c) to (a) and from (a) to (c) are both small, and conversely, the ranking from (a) to (a) is large. Therefore, the sum of the ranks from (c) to (b) and the ranks from (b) to (c) is the sum of the ranks from (a) to (b) and from (b) to (a). expected to be smaller.

Next, among the corresponding combinations of blocks on the A image and blocks on the B image, the sum of the ranks is added according to the combination, and the number of blocks that are not included in the correspondence in the combination is further calculated. The cost is calculated by adding a certain penalty according to the amount. Considering the example in Figure 6, it also corresponds to block (A) in the occlusion area (A) - (
If we compare the cost of the combination that provides (b) with the cost of the combination that does not provide correspondence to the block (a) in the occlusion area, it is thought that the latter will be smaller. Therefore, if we find the combination of responses that minimizes the cost, we get
It can be expected that the combination excludes blocks in the occlusion area.

[0015] Here, if there is a block that does not provide correspondence, it is important to add a certain penalty to the cost accordingly. This is because, if no penalty is given, a combination with no correspondence will have the minimum cost.

In the image matching method of the second invention, the sum of the ranks is not used as a direct cost, but a value obtained by converting the ranks using a certain monotonically increasing function is used as the cost. Generally, the importance of rank differences in lower ranks is considered to be higher than the importance of rank differences in higher ranks. This function can change the importance of the difference in rank depending on the rank.

[0017] The image matching method of the third invention ranks each correspondence in ascending order of the correlation coefficient by a certain number, and for those that are not ranked, the possibility of correspondence is I judge that there is no. Therefore, the number of combinations to be searched to find the corresponding combination that provides the minimum cost is reduced, allowing faster execution. Additionally, the cost of a correspondence that includes a block in an occlusion area increases, and the possibility of excluding a correspondence that includes a block in an occlusion area increases accordingly.

The matching method for each scanning line of the fourth invention uses dynamic programming that allows discontinuous paths.
Accurate combinations of segments are required quickly. In FIG. 7, scanning lines A and B are arranged vertically. The plane defined by axis AB is generally called a search plane. The center positions of blocks on A and B are represented by vertical lines and horizontal lines, respectively, and one correspondence between segments is represented by the intersection of the vertical line and the horizontal line. The corresponding combination between AB is P1 and P
It is indicated by a single path C on the search plane connecting the two.

[0019] If there is an occlusion region, the path becomes discontinuous. A and b in FIG. 7 are portions where the path is discontinuous. Discontinuity a indicates a region hidden by B, and conversely, discontinuity b indicates a region hidden by A.

The problem of finding the optimal path connecting p1 and p2 is formulated as an optimal path problem, to which dynamic programming can be applied. However, conventional dynamic programming only considers continuous paths, which results in incorrect responses in occlusion areas.

The image matching method of the present invention uses dynamic programming that allows discontinuous paths. FIG. 8 shows the difference from conventional dynamic programming. Here, starting from p1, p2
Suppose we want to find the optimal path to . In the conventional method, for example, the points on the lower left side of point u that may be connected to point u at coordinates (x, y) are (x-1, y), (x, y
-1), (x-1, y-1) only. This 3
The points are indicated by black circles in FIG. In dynamic programming, which allows discontinuous paths, a path is generated by taking into account that occlusion occurs at point u and its lower left point, and also taking into account the case where the white circle point directly connects to u. For this purpose, an accurate correspondence is required that does not give a pass to the occlusion area.

[0022]

Embodiment FIG. 1 is a block diagram showing an embodiment of the first invention.

[0023] This image matching method uses two images (
An image input means 11 for inputting images (A image, B image), an image storage means 12 for storing images inputted by this image input means, and an image A, B image stored in this image storage means.
Image dividing means 1 for dividing an image into blocks allowing overlap
3, and a correlation calculating means 14 for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means. , using the correlation coefficient values for each correspondence between blocks on the A image and blocks on the B image obtained by this correlation calculation means, calculate the correlation coefficient from the block on the A image to the block on the B image. In descending order,
Correlation coefficient ranking means 15 for ranking in ascending order and ranking in ascending order from blocks on B image to blocks on A image in ascending order of correlation coefficient; and correlation calculation means 14.
The ranking of the correlation coefficient from the block on the A image to the block on the B image obtained by the correlation coefficient ranking means 15 for each correspondence between the blocks on the A and B images calculated in , correlation coefficient rank addition means 16 for calculating the sum of the ranks of correlation coefficients from the same block on image B to the same block on image A, and the correlation calculation means 14, The sum of the ranks obtained by the correlation coefficient rank adding means 16 is added for each combination of blocks corresponding to each other, and a certain sum is added for each combination of blocks that are not included in the correspondence in each combination. penalty,
In addition to the addition results of the combinations, the minimum addition rank selection means 17 calculates the combination that gives the minimum result.

Next, the operation of this image matching method will be explained.

Two images (A
image, B image) is input and stored by the image storage means 12. Next, the image dividing means 13 divides the A image and the B image into blocks while allowing overlap. The correlation calculation means 14 calculates the cross-correlation of the pixel values included in the blocks for all or some combinations of the blocks on the A image and the blocks on the B image.

For the set of correlation coefficients obtained here, the correlation coefficient ranking means 1 ranks each block on the A image to each block on the B image in ascending order according to the value of the correlation coefficient.
5. That is, for a certain block x on the A image, the order of the block on the B image with the highest correlation coefficient value is set to 1, and then the order of the correlation coefficient values is 2, 2,
3,... and the block y on the B image are ranked OR [A(x)B
(y)].

Similarly, from each block on the B image to each block on the A image, the ascending order (O
R[B(y)A(x)] and correlation coefficient ranking means 1
Do it in 5.

FIG. 2 is a block diagram showing an embodiment of the second invention.

In this image matching method, a certain block x on image A is
For a certain block y on B image, the correlation coefficient rank O from block x on A image to block y on B image
R[A(x)B(y)] and the rank OR[B(y) of the correlation coefficient from block y on B image to block x on A image
)A(x)] sum W(x,y)=OR[A(x)B(y)]+OR[B(
y)A(x)]. Furthermore, the sum of the ranks W(x, y) obtained here is added according to all correspondence combinations, and if there is a block that is not included in the correspondence among the combinations, a certain penalty is applied according to the number. , and the minimum addition rank selection means 17 determines the combination of correspondences that minimizes the cost.

This image matching method uses two images (
An image input means 21 for inputting images (A image, B image), an image storage means 22 for storing images inputted by this image input means, and A image, B image stored in this image storage means.
Image dividing means 2 that divides an image into blocks allowing overlap
3, and a correlation calculating means 24 for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means. , using the correlation coefficient values for each correspondence between blocks on the A image and blocks on the B image obtained by this correlation calculation means, calculate the correlation coefficient from the block on the A image to the block on the B image. In descending order,
Correlation coefficient ranking means 25 for ranking in ascending order and ranking in ascending order from blocks on the B image to blocks on the A image in ascending order, and this correlation coefficient ranking means For the obtained ranking, a cost calculation means 26 converts the ranking value into a cost value using a monotonically increasing function given in advance, and a cost calculation means 26 for converting the ranking value into a cost value using a monotonically increasing function given in advance. Regarding the correspondence, the cost of calculating the sum of the cost from the block on the A image to the block on the B image and the cost from the same block on the B image to the same block on the A image, obtained by the cost calculation means 26. For all the combinations of corresponding blocks on images A and B calculated by the addition means 27 and the correlation calculation means 24, the sum of the ranks obtained by the cost addition means 27 is added for each combination, and It consists of a minimum addition cost selection means 28 that adds a certain penalty according to the number of blocks not included in the correspondence in the combination to the addition result of the combination and finds the combination that minimizes the result.

Next, the operation of this image matching method will be explained.

Image input means 21, image storage means 22, image division means 23, correlation calculation means 24, and correlation coefficient ranking means 25 operate in the same manner as each means shown in FIG.

The cost calculating means 26 converts the ranking value into a cost value using a monotonically increasing function f given in advance for the ranking obtained by the correlation coefficient ranking means. The rank values are OR[A(x)B(y)] and OR[B(y)
)A(x)], each of them becomes f(
OR[A(x)B(y)]), f(OR[B(y)A(
x)]).

The cost addition means 27 calculates the cost from the block x on the A image to the block y on the B image obtained by the cost calculation means 26 for each correspondence between the blocks on the A and B images, and the cost of the B image. Calculate the sum W(x, y) of costs from the same block y above to the same block x on the A image. W(x,y)=OR[A(x)B(y)]+OR[B(
y)A(x)] Furthermore, the sum of the ranks obtained here W(x
. The minimum additional cost selection means 28 determines the corresponding combination.

FIG. 3 is a block diagram showing an embodiment of the third invention.

This image matching method uses two images (
An image input means 31 for inputting images (A image, B image), an image storage means 32 for storing images input by this image input means, and A image, B image stored in this image storage means.
Image dividing means 3 for dividing an image into blocks allowing overlap
3, and a correlation calculating means 34 for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means. , using the correlation coefficient values for each correspondence between the blocks on the A image and the blocks on the B image obtained by this correlation calculation means, the correlation coefficient is calculated from the block on the A image to each block on the B image. In descending order,
AB direction correlation coefficient ranking and correspondence determining means 35 that ranks a certain number of blocks in ascending order and determines that there is no possibility of correspondence between the blocks for blocks that are not ranked; and correlation calculating means. A obtained by 34
Using the correlation coefficient value for each correspondence between blocks on the image and blocks on the B image, rank the blocks on the B image to each block on the A image in ascending order by a certain number of blocks in descending order of correlation coefficient. B. If the blocks are not ranked, it is determined that there is no possibility of correspondence between the blocks.
Among the respective correspondences of blocks on the A and B images calculated by the A direction correlation coefficient ranking and correspondence determination means 36 and the correlation calculation means 34, the AB direction correlation coefficient ranking and correspondence determination means 35 and the BA direction correlation coefficient ranking and correspondence determination means 36, excluding correspondences determined to have no possibility of correspondence, the ranks obtained by the AB direction correlation coefficient ranking and determination means 35, and the BA Correlation coefficient rank addition means 37 calculates the sum with the rank obtained by directional correlation coefficient ranking and determination means 36, and correlation calculation means 34 calculates the correspondence between blocks on images A and B. For all combinations, correlation coefficient rank adding means 37
Add the sum of the ranks found in each combination for each combination, add a certain penalty according to the number of blocks that are not included in the correspondence in each combination, and select the combination of correspondences that minimizes the result. , AB direction correlation coefficient ranking and determination means 35
and a minimum addition rank selection means 38 for determining from among the combinations of correspondences that are not determined to have no possibility of correspondence by the BA direction correlation coefficient ranking and determination means 36.

Next, the operation of this image matching method will be explained.

The image input means 31, the image storage means 32, the image division means 33, the correlation calculation means 34, and the correlation coefficient rank addition means 36 operate in the same manner as each means shown in FIG.

The AB direction correlation coefficient ranking and correspondence determination means 35 uses the correlation coefficient value for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means 34, and From blocks on the image to each block on the B image, the blocks are ranked in ascending order by a certain number of OL in descending order of correlation coefficient, and for blocks that are not ranked, there is no possibility of correspondence between the blocks. It is determined that

That is, for a certain block x on the A image, the order of the block on the B image with the highest correlation coefficient value is set to 1, and then the order of the correlation coefficient values is 2, 3, etc.
,..., the blocks on the OL and B images are ranked OR [A(x)
B(y)]. Let the set of blocks on the B image that were not ranked at this time be NCb(x), and the set of correspondences NC that is not included in the combination of correspondences is:

[Math 1]

[0041]

Calculate as follows.

The BA direction correlation coefficient ranking and correspondence determination means 36 calculates the correlation coefficient value for each correspondence between the blocks on the A image and the blocks on the B image obtained by the correlation calculation means 34. A from the block on the B image
The blocks on the image are ranked in ascending order by a certain number OL in descending order of their correlation coefficients, and for blocks that are not ranked, it is determined that there is no possibility of correspondence between the blocks.

That is, for a certain block x on the B image, the order of the block on the A image with the highest correlation coefficient value is set to 1, and then the order of the correlation coefficient values is 2, 3, etc.
,..., rank OR [B(y)
A(x)]. Let the set of blocks on image A that were not ranked at this time be NCa(x), and the set of correspondences NC that is not included in the combination of correspondences is:

[Math 2]

[0045]

Calculate as follows.

[0047] The correlation coefficient rank addition means 37 selects the correspondence NC for which it is determined that there is no possibility of correspondence among the correspondences of the blocks on the A and B images calculated by the correlation calculation means 34. except for the rank OR [A(x)B(y)] obtained by the AB direction correlation coefficient ranking and determination means 35.
and the rank OR [B(y)A(x)] obtained by the BA direction correlation coefficient ranking and determination means 36
, y) using the following formula: W(x, y)=OR[A(x)B(y)]+OR[B(
y)A(x)] The minimum addition rank selection means 38 calculates the sum of ranks W(x
. Find responses from those excluded from NC.

FIG. 4 is a block diagram showing an embodiment of the fourth invention.

This image matching method uses two images (
An image input means 41 for inputting images (A image, B image), an image storage means 42 for storing images inputted by this image input means, and A image, B image stored in this image storage means.
An image dividing means 43 that divides an image into each scanning line and further divides each scanning line into segments by allowing overlap, and a segment on the A image generated by the image dividing means 43 that exists on the scanning line at the same position. Correlation calculation means 44 that calculates the cross-correlation of pixel values for all or part of the correspondence between segments on images A and B, and the correspondence between the segments on images A and B calculated by this correlation calculation means. , using the correlation coefficient values for each correspondence, from the segment on the A image to the segment on the B image in descending order of the correlation coefficient,
Correlation coefficient ranking means 45 for ranking in ascending order and ranking in ascending order from segments on the B image to segments on the A image in ascending order; For each correspondence between the calculated segments on images A and B, rank the correlation coefficient from the segment on image A to the segment on image B, and from the segment on image B to the segment on image A. Correlation coefficient rank addition means 46 calculates the sum of the ranks of the correlation coefficients of A path generating means 47 that generates not only paths but also paths between non-adjacent correspondences, and a correlation coefficient rank adding means for all combinations of correspondences expressed by connecting paths generated by this path generating means. Add the sum of the ranks obtained in step 46 for each combination, add a certain penalty according to the number of blocks that are not included in the correspondence in each combination, and move the combination of correspondences that minimizes the result. and a minimum addition rank selection means 48 which is determined using a systematic programming method.

Next, the operation of this image matching method will be explained.

The image input means 41, the image storage means 42, the correlation calculation means 44, the correlation coefficient ranking means 45, and the correlation coefficient rank addition means 46 operate in the same manner as each means shown in FIG. The image dividing means 43 divides the image into each scanning line, and further divides each scanning line into blocks.

The minimum addition rank selection means 48 using dynamic programming that allows discontinuous paths will be explained with reference to FIG. This figure shows a search plane composed of an axis x on scanning line A and an axis y on scanning line B. The range of correspondence calculated by the correlation calculating means 44 within the search plane is represented by S. In addition, each grid point on the search plane has the sum W of ranks from both directions.
(x, y) are given. The minimum addition rank selection means selects the path that minimizes W(x, y) given to the lattice points on the path from among the paths connecting p1 and p2 that allow discontinuity.

The minimum addition rank selection means considers a straight line L such that x+y=t, and by increasing t by 1, L is set to p.
Move from position 1 to position p2 in order. At each stage of moving L, the minimum cost of reaching the lattice point on L that is included in S is calculated. For example, the minimum cost r to reach the grid point u(x,y)
Find c(x,y) as follows. rc(x,y)=min{rc(x-1,y-1)+W
(x,y),
rc(x-1,y)+W(x,y),
rc(x
−2,y)+W(x,y)+α,
rc(x-3,y)+W
(x,y)+2α,
rc(x-4,y)+W(x,y)+
3α,
・

・

・
rc(x,y-1)+W(x,y),
rc(
x,y-2)+W(x,y)+α,
rc(x,y-3)+
W(x,y)+2α,
rc(x,y-4)+W(x,y)
+3α,
・

・

・
}However, α is the penalty for creating one block that does not provide correspondence. In the figure, the correspondence (x, y) is a black square, the correspondence (x-1, y-1
), (x-1, y), (x-2, y), (x-3, y)
, ..., and the correspondence (x, y-1), ((x, y-2)
, (x, y-3), . . . are indicated by black circles. At the point where L arrives at p2, the path that gives the minimum arrival cost at p2 is the combination of correspondences given by the fourth invention.

[0054]

As described above, the image matching method of the present invention reduces the possibility of providing correspondence to occlusion areas. Therefore, there is less need to force a match where matching is not possible, and the accuracy of matching is improved.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of the first invention.

FIG. 2 is a block diagram showing an embodiment of the second invention.

FIG. 3 is a block diagram showing an embodiment of the third invention.

FIG. 4 is a block diagram showing an embodiment of the fourth invention.

FIG. 5 is a diagram for explaining an example in which a conventional method gives an incorrect response when there is an occlusion area.

FIG. 6 is a diagram for explaining an example showing that the image matching method of the present invention provides accurate correspondence.

FIG. 7 is a diagram for explaining an example in which occlusion areas are shown as discontinuous paths in one-dimensional image matching.

FIG. 8 is a diagram showing the difference between conventional dynamic programming and dynamic programming that allows discontinuous paths.

FIG. 9 is an explanatory diagram of a dynamic programming method that allows discontinuous paths.

[Explanation of symbols]

11 Image input means 12 Image storage means 13 Image division means 14 Correlation calculation means 15 Correlation coefficient ranking means 16 Correlation coefficient rank addition means 17 Minimum addition rank selection means 21 Image input means 22 Image storage means 23 Image division means 24 Correlation Calculation means 25 Correlation coefficient ranking means 26 Cost calculation means 27 Cost addition means 28 Minimum added cost selection means 31 Image input means 32 Image storage means 33 Image division means 34 Correlation calculation means 35 AB direction correlation coefficient ranking and correspondence determination Means 3
6 BA direction correlation coefficient ranking and correspondence determination means 37
Correlation coefficient rank addition means 38 Minimum addition rank selection means 41 Image input means 42 Image storage means 43 Image division means 44 Correlation calculation means 45 Correlation coefficient ranking means 46 Correlation coefficient rank addition means 47 Path generation means 48 Minimum addition rank selection means

Claims (4)

[Claims] Claim 1: An image matching method consisting of the following a to g. a. Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means f for ranking in ascending order, and ranking in ascending order from the blocks on the B image to the blocks on the A image in the order of the larger correlation coefficient.
．． Correlation from blocks on image A to blocks on image B, obtained by the correlation coefficient ranking means, for each correspondence between blocks on images A and B calculated by the correlation calculation means. correlation coefficient rank adding means for calculating the sum of the rank of the number and the rank of the correlation coefficient from the same block on the B image to the same block on the A image g. For all combinations of blocks on images A and B calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and Minimum addition rank selection means that adds a certain penalty according to the number of blocks not included in the correspondence to the addition result of the combination to find the combination that minimizes the result. 2. An image matching method consisting of the following a to h. a. Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means f for ranking in ascending order, and ranking in ascending order from the blocks on the B image to the blocks on the A image in the order of the larger correlation coefficient.
．． g. cost calculating means for converting the ranking value into a cost value using a monotonically increasing function given in advance for the ranking obtained by the correlation coefficient ranking means; g. The cost from the block on the A image to the block on the B image obtained by the cost calculation means for each correspondence between the blocks on the A and B images calculated by the correlation calculation means, and the B image Cost addition means for calculating the sum of costs from the same block on the A image to the same block on the A image h. For all combinations of correspondences between blocks on images A and B calculated by the correlation calculation means, the sum of ranks obtained by the cost addition means is added for each combination, and further the correspondences in each combination are added. Minimum addition cost selection means for adding a certain penalty according to the number of blocks not included in the combination to the addition result of the combination and finding the combination that minimizes the result. 3. An image matching method consisting of the following a to h. a. Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing the A image and B image stored in the image storage means into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of pixel values included in the blocks for all or part of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the correlation coefficient values obtained by the correlation calculation means for each correspondence between blocks on the A image and blocks on the B image, the correlation coefficient is calculated from the block on the A image to each block on the B image. A and B direction correlation coefficient ranking and correspondence determining means f. sequentially ranking a certain number of blocks in ascending order, and determining that there is no possibility of correspondence between blocks that are not ranked; f. Using the correlation coefficient values obtained by the correlation calculation means for each correspondence between blocks on the A image and blocks on the B image, the correlation coefficient is calculated from the block on the B image to each block on the A image. g. BA direction correlation coefficient ranking and correspondence determining means for ranking a certain number of blocks in ascending order and determining that there is no possibility of correspondence between blocks that are not ranked; g. Among the correspondence between the blocks on the A and B images calculated by the correlation calculation means, the AB direction correlation coefficient ranking and correspondence determination means and the BA direction correlation coefficient ranking and correspondence determination means , the ranking obtained by the AB direction correlation coefficient ranking and determination means and the rank obtained by the BA direction correlation coefficient ranking and determination means, excluding correspondences for which it is determined that there is no possibility of correspondence. Correlation coefficient rank addition means for calculating the sum of h. For all combinations of corresponding blocks on images A and B calculated by the correlation calculation means, the sum of ranks obtained by the correlation coefficient rank addition means is added for each combination, and The AB direction correlation coefficient ranking and determination means and the BA direction correlation Minimum addition rank selection means for determining from among the combinations of correspondences that are not determined to have no possibility of correspondence by the numerical ranking and determination means. 4. An image matching method consisting of the following a to h. a. Image input means for inputting two images (image A, image B) b. image storage means for storing the image input by the image input means c. image dividing means for dividing images A and B stored in the image storage means into each scanning line, and further dividing the scanning lines into segments by allowing overlap; d. Correlation calculating means e that calculates the cross-correlation of pixel values for all or part of the correspondence between the segments on the A image and the segments on the B image generated by the image dividing means, which exist on the scanning line at the same position. ．． Regarding the correspondence between the segments on the A and B images calculated by the correlation calculation means, the value of the correlation coefficient for each correspondence is used, and the correlation coefficient from the segment on the A image to the segment on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and ranking in ascending order of correlation coefficients from segments on the B image to segments on the A image, f. Regarding each correspondence between the segments on images A and B calculated by the correlation coefficient ranking means, the ranking of the correlation coefficient from the segment on image A to the segment on image B, and the correlation coefficient on image B Correlation coefficient rank adding means for calculating the sum of the ranks of correlation coefficients from the segment to the segment on the A image g. A calculated by the correlation coefficient rank addition means
, A path generating means for generating not only paths between adjacent correspondences but also paths between non-adjacent correspondences regarding the correspondence between segments on the B image h. For all the corresponding combinations expressed by connecting the paths generated by the path generation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and Minimum addition rank selection means that adds a certain penalty according to the number of blocks that are not included in the correspondences, and uses dynamic programming to find a combination of correspondences that minimizes the result.