JP3027995B2 - Image matching method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to stereo matching and image matching for detecting a motion vector.

[0002]

2. Description of the Related Art Among conventional image matching techniques, a system using block matching is known as Japanese Patent Publication No.
2-107386, "Image matching method", U.S. Pat. No. 4,677,476, "METHOD FOR"
DETECTING A MOVEMENT OF A
A technique described in "Television Signal" is known. In these systems, when performing a matching process of two image data stored in a storage device, one of an image is divided into a plurality of blocks. And
Cross-correlation is performed with the other image while shifting each block in parallel, and it is estimated that the portion where the correlation is maximum is the block corresponding to the other image.

[0003]

However, the conventional block matching has a problem that correct matching cannot be performed in the occlusion area. That is, as shown in FIG. 5, if the object in the area a in the image A moves to the area c in the image B, the area b indicated by the oblique lines in the image A becomes the image B Is hidden in the area c. In this case, there is no area in the image B corresponding to the area b. Generally, such an area b is called an occlusion area.

When the image A is divided into blocks and matched with the image B, the blocks included in the area b do not originally have a corresponding partner. Nevertheless, in the conventional method, the point where the correlation becomes maximum is determined to be the corresponding point. For example, the block (a) included in the area b gives an undesired correspondence, such as corresponding to (a).

[0005] An object of the present invention is to provide a matching method that gives an accurate correspondence in the sense that no correspondence is given to an occlusion area.

[0006]

According to the first aspect of the present invention, there is provided an image matching system comprising: a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculation means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image division means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from a block on the B image to a block on the A image in descending order of the correlation coefficient f. For each correspondence between the blocks on the A and B images calculated by the correlation calculating means, the block B on the A image obtained by the correlation coefficient ranking means is used.
Correlation coefficient rank adding means for calculating the sum of the rank of the correlation coefficient to the block on the image and the rank of the correlation coefficient from the same block on the B image to the same block on the A image g. For all combinations of blocks on the A and B images calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and further, Number of blocks that do not give a correspondence
, A penalty is added to the addition result of the combination, and a minimum addition order selecting means for obtaining a combination that minimizes the result is provided.

[0007] The image matching method according to the second invention comprises the following steps: a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculating means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from a block on the B image to a block on the A image in descending order of the correlation coefficient f. Cost calculating means for converting the rank value into a cost value by a predetermined monotone increasing function with respect to the rank obtained by the correlation coefficient ranking means; g. For each correspondence between the blocks on the A and B images calculated by the correlation calculation unit, the cost from the block on the A image to the block on the B image, obtained by the cost calculation unit, and the B image Cost adding means for calculating the sum of costs from the same block above to the same block on the A image h. For all the combinations of the blocks on the A and B images calculated by the correlation calculating means, the sum of the ranks obtained by the cost adding means is added for each combination, and the correspondence in each combination is further added. Is proportional to the number of blocks that do not give
A minimum addition cost selecting means for obtaining a combination which obtains a minimum number, adds the result as a penalty to the addition result of the combination, and obtains a combination which minimizes the result.

[0008] An image matching method according to a third aspect of the present invention comprises: a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculation means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image division means e. The value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculating means is used to increase the correlation coefficient from the block on the A image to each block on the B image. AB order correlation coefficient ranking and correspondence determination means for sequentially determining a certain number in ascending order and determining that there is no possibility of correspondence between the blocks for the unordered ones. F. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the B image to each block on the A image is large. BA order correlation coefficient ranking and correspondence judging means for judging in ascending order by a certain number in order and determining that there is no possibility of correspondence between the blocks for those not ranked g. In the correspondence of the blocks on the A and B images calculated by the correlation calculation means, by the AB direction correlation coefficient ranking and correspondence determination means and the BA direction correlation coefficient ranking and correspondence determination means And the order obtained by the AB direction correlation coefficient ranking and determination means, and the order obtained by the BA direction correlation coefficient ranking and determination means, except for the correspondence determined to be impossible. Correlation coefficient rank adding means for calculating the sum of h. For all combinations of the blocks on the A and B images calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and further, Number of unassigned blocks in
Is calculated as a penalty, and the union
In addition to the summation result of the set, there is no possibility that the AB direction correlation coefficient ranking and determination means and the BA direction correlation coefficient ranking and determination means will correspond in the correspondence combinations that minimize the result. It is characterized by comprising a minimum addition order selecting means for obtaining from the corresponding combinations which have not been determined to be.

An image matching method according to a fourth aspect of the present invention comprises: a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. Image dividing means for dividing the A image and the B image stored in the image storage means for each scanning line, and further dividing the scanning lines into segments while allowing overlaps. D. Correlation calculating means e for calculating the cross-correlation of pixel values for all or some correspondences between segments on the A image and segments on the B image generated by the image dividing means, which are present on the scanning line at the same position. . For the correspondence between the segments on the A and B images calculated by the correlation calculation means, the value of the correlation coefficient for each correspondence is used, and the correlation coefficient from the segment on the A image to the segment on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from the segment on the B image to the segment on the A image in descending order of the correlation coefficient f. For each correspondence between the segments on the A and B images calculated by the correlation coefficient ranking means, the order of the correlation coefficient from the segment on the A image to the segment on the B image, Correlation coefficient rank adding means for calculating the sum of the rank of the correlation coefficient from the segment to the segment on the A image g. The calculation was performed by the correlation coefficient rank adding means,
Path generation means for generating not only paths between adjacent correspondences but also paths between non-adjacent correspondences regarding the correspondence between segments on the A and B images h. For all the corresponding combinations represented by connecting the paths generated by the path generating means, the sum of the ranks determined by the correlation coefficient rank adding means is added for each combination, and further, Don't give me a medium response
Find a number proportional to the number of new blocks and penalize it.
In addition to the addition result of the combination, there is provided a minimum addition order selecting means for obtaining a corresponding combination that minimizes the result by using the dynamic programming method.

[0010]

According to the image matching method of the present invention, the occlusion area is detected by performing bidirectional matching. As a result, the blocks included in the occlusion area are correctly associated in the sense that no corresponding block is given.

This will be described with reference to FIG. Here, image A
It is assumed that the area a in the middle has moved to the area d in the image B. An area b indicated by oblique lines in the image A and an area c indicated by oblique lines in the image B are occlusion areas. Here, one of the blocks in b is (a).

First, both the image A and the image B are divided into blocks, and the mutual relation is calculated for every correspondence of the blocks. Next, the ascending order of the correlation coefficient values from each block on the A image to each block on the B image is set in ascending order, and the phase from each block on the B image to each block on the A image is determined. The order of ascending order of the value of the relation number is ascending. Here, it is assumed that the correlation coefficient of the block (a) is the largest for the block (a), and (a) of (a)
And rank 1 for. In the conventional method, at this point, the corresponding point to (A) is determined to be (A).

In the image matching method of the present invention, the order of the correlation coefficient from the block on the A image to the block on the B image and the correlation between the same block on the B image and the same block on the A image are further determined. Calculate the sum of a number and its rank. FIG.
In the case of (1), it is assumed that the (c) block on the A image and the (b) block on the B image actually correspond to each other. In this case, it is considered that both the rank from (c) to (b) and the rank from (b) to (c) are small, and conversely, the rank from (b) to (a) is large. Therefore, from (U) to (A)
It is expected that the sum of the ranks of (a) and (c) will be smaller than the sum of the ranks of (a) and (b) and the ranks of (a) and (a).

Next, among the combinations of the correspondence between the block on the A picture and the block on the B picture, the sum of the ranks is added according to the combination, and the correspondence in the combination is not given.
Find a number proportional to the number of new blocks and penalize it.
Obtaining a cost plus a. Considering the example of FIG. 6, the cost of a combination giving (a)-(b) to the block (a) in the occlusion area and the cost of a combination giving no correspondence to the block (a) in the occlusion area are considered. When comparing costs, the latter is considered to be smaller. Therefore, if a corresponding combination that minimizes the cost is obtained, it can be expected that the combination is a combination excluding the blocks in the occlusion area.

Here, if there is a block to which no correspondence is given, it is important to add a certain penalty to the cost accordingly. This is because if no penalty is given, a combination having no correspondence will have the minimum cost.

In the image matching method according to the second aspect of the present invention, a value obtained by converting a rank by a certain monotonically increasing function is used as a cost instead of directly using a sum of the ranks as a cost. In general, the significance of the rank difference at lower ranks is considered to be higher than the significance of the rank difference at higher ranks. The significance of such a rank difference depending on the rank can be changed by this function.

According to the image matching method of the third invention, for each correspondence, a fixed number of ranks are assigned in ascending order in descending order of the correlation coefficient. Judge that there is no. Therefore, the number of combinations to be searched to find the corresponding combination that gives the minimum cost is reduced, and the execution can be speeded up. Further, the cost provided by the correspondence including the block in the occlusion area increases, and the possibility of eliminating the correspondence including the block in the occlusion area increases accordingly.

In the matching method for each scanning line of the fourth invention, an accurate combination of segments can be obtained at high speed by using a dynamic programming method that allows discontinuous paths. FIG. 7 shows scanning lines A and B arranged vertically. Axis A
The plane formed by B is generally called a search plane.
The center positions of the blocks on A and B are represented by vertical and horizontal lines, respectively, and one correspondence between segments is represented by the intersection of the vertical and horizontal lines. The correspondence combination between AB is P1 and P
2 is indicated by one path C on the search plane connecting the two.

If there is an occlusion area, the path becomes discontinuous. 7A and 7B are portions where the paths are discontinuous. The discontinuity a indicates an area hidden by B, and the discontinuity b indicates an area hidden by A.

The problem of finding an optimal path connecting p1 and p2 is formulated as an optimal course problem, and dynamic programming can be applied. However, in the conventional dynamic programming, only continuous paths are considered, so that only an incorrect response is required in the occlusion area.

In the image matching method of the present invention, a dynamic programming method that allows discontinuous paths is used. FIG. 8 shows a difference from the conventional dynamic programming. Here, starting from p1, p2
Find the optimal path to In the conventional method, for example, a point on the lower left side of the point u which may be connected to the point u of the coordinates (x, y) is (x-1, y), (x, y).
y-1) and (x-1, y-1). These three points are indicated by black circles in FIG. In the dynamic programming method that allows a discontinuous path, a path is generated in consideration of the fact that occlusion occurs at a point u and a point on the lower left side thereof, and also in a case where the point of a white circle is directly connected to u. For this reason, an accurate response without giving a path to the occlusion area is required.

[0022]

FIG. 1 is a block diagram showing an embodiment of the first invention.

This image matching method includes an image input means 11 for inputting two images (A image and B image), an image storage means 12 for storing an image input by the image input means, and an image storage means. A image stored in
An image dividing unit 13 that divides the B image into blocks while allowing overlap, and all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image dividing unit are included in the blocks. Calculation means 14 for calculating the cross-correlation of the pixel values to be obtained, and the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by this correlation calculation means. From the block on the B image to the block on the B image in ascending order, and from the block on the B image to the block on the A image in ascending order. The correlation coefficient ranking means 15 and the correlation calculation means 14 calculate the correspondence between the blocks on the A and B images. And ranking of correlation coefficients from block to block in the B picture, A from the same block on the B image
Correlation coefficient rank adding means 16 for calculating the sum of the ranks of the correlation coefficients for the same block on the image, and all combinations of the correspondence of the blocks on the A and B images calculated by the correlation calculation means 14 , The sum of the ranks obtained by the correlation coefficient rank adding means 16 is added for each combination, and a number proportional to the number of unassigned blocks in each combination is obtained.
A minimum addition rank selecting means 17 for adding a result added as a penalty to an addition result of the combination and obtaining a combination that minimizes the result.

Next, the operation of the image matching method will be described.

The image input means 11 outputs two images (A
Image, B image), and stores the image by the image storage unit 12. Next, the A image and the B image are divided into blocks by the image dividing means 13 while allowing overlap. The correlation calculator 14 calculates the cross-correlation of the pixel values included in the block for all or some combinations of the block on the A image and the block on the B image.

For the set of correlation coefficients obtained here, the correlation coefficient ranking means 1 ranks each block on the A image from each block on the B image in ascending order according to the value of the correlation coefficient.
5 is performed. That is, for a certain block x on the A image, the rank of the block on the B image having the highest correlation coefficient value is set to 1, and thereafter, the order of the correlation coefficient value is 2,
, And the order OR [A (x) B
(Y)].

Similarly, an ascending order (O) based on the value of the correlation coefficient from each block on the B image to each block on the A image.
R [B (y) A (x)] and correlation coefficient ranking means 1
Step 5

Next, the correlation coefficient rank adding means 16 calculates
For a certain block x on the A image and a certain block y on the B image, the rank OR [A (x) B] of the correlation coefficient from the block x on the A image to the block y on the B image
(Y)] and the order OR [B (y) A (x)] of the correlation coefficient from the block y on the B image to the block x on the A image W (x, y) = OR [A ( x) B (y)] + OR [B (y) A (x)] is calculated. Further, the sum W (x,
y) is added in accordance with any corresponding combination, and if there is a block in the combination that does not provide a correspondence ,
Cost plus penalty proportional to the number of
The corresponding combination that minimizes the cost is obtained by the minimum addition order selecting means 17.

FIG . 2 is a block diagram showing an embodiment of the second invention.
FIG.

The image matching method includes an image input means 21 for inputting two images (A image and B image), an image storage means 22 for storing an image input by the image input means, and an image storage means. A image stored in
An image dividing unit 23 that divides the B image into blocks while permitting duplication, and all or some of the correspondence between the block on the A image and the block on the B image generated by the image dividing unit are included in the blocks. Calculating means 24 for calculating the cross-correlation of the pixel values to be obtained, and the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculating means. From the block on the B image to the block on the B image in ascending order, and from the block on the B image to the block on the A image in ascending order. And a cost calculator for converting the rank value into a cost value by a monotonically increasing function given in advance to the rank obtained by the correlation coefficient ranking means. 26 and the cost of the block on the A image from the block on the B image obtained by the cost calculation unit 26 for each correspondence of the blocks on the A and B images calculated by the correlation calculation unit. , B
For all combinations of the blocks on the A and B images calculated by the cost adding means 27 for calculating the sum of the costs from the same block on the image to the same block on the A image and the correlation calculating means 24 , The sum of the ranks obtained by the cost adding means 27 is added for each combination, and a number proportional to the number of unassigned blocks in each combination is obtained.
A minimum addition cost selecting means 28 for adding the result of the addition as a penalty to the addition result of the combination and obtaining a combination that minimizes the result is provided.

Next, the operation of the image matching method will be described.

The image input means 21, the image storage means 22, the image dividing means 23, the correlation calculating means 24, and the correlation coefficient ranking means 25 operate in the same manner as the respective means in FIG.

The cost calculating means 26 converts the rank value obtained by the correlation coefficient ranking means into a cost value by a predetermined monotone increasing function f. Rank values are represented by OR [A (x) B (y)] and OR [B
(Y) A (x)], this function causes f (OR [A (x) B (y)]) and f (OR [B (y)
A (x)]).

The cost adding means 27 calculates the cost from the block x on the A image to the block y on the B image obtained by the cost calculating means 26 for each block on the A and B images, The sum W (x, y) of costs from the same block y above to the same block x on the A image is calculated. W (x, y) = OR [A (x) B (y)] + OR [B (y) A (x)] Further, the sum of the ranks obtained here is represented by If there is a block that is not included in the correspondence among the combinations in addition according to the combination, a cost obtained by adding a certain penalty according to the number is used as the cost, and the corresponding combination that minimizes the cost is the minimum addition cost It is determined by the selection means 28.

FIG. 3 is a block diagram showing an embodiment of the third invention.

This image matching method comprises an image input means 31 for inputting two images (A image and B image), an image storage means 32 for storing an image input by the image input means, and an image storage means A image stored in
An image dividing unit 33 that divides the B image into blocks while allowing overlap, and a block that includes all or a part of the correspondence between the block on the A image and the block on the B image generated by the image dividing unit. A correlation calculating means 34 for calculating a cross-correlation of pixel values to be obtained, and a value of a correlation coefficient for each correspondence between a block on the A image and a block on the B image obtained by the correlation calculating means. From the block of the B image to each block on the B image in ascending order by a fixed number in the order of larger correlation coefficient, and if there is no ranking, it is determined that there is no possibility of correspondence between the blocks. Using the correlation coefficient ranking and correspondence determination means 35 in the AB direction and the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means 34 The blocks on the B image are ranked in ascending order from the block on the A image to the blocks on the A image by a certain number in descending order of the correlation coefficient. BA direction correlation coefficient ranking and correspondence determination means 36
In the correspondence of the blocks on the A and B images calculated by the correlation calculating means 34, the AB direction correlation coefficient ranking and correspondence determination means 35 and the BA direction correlation coefficient ranking and correspondence determination The order obtained by the AB direction correlation coefficient ranking and determination unit 35 and the order obtained by the BA direction correlation coefficient ranking and determination unit 36 are excluded except for the correspondence determined to be impossible by the unit 36. The correlation coefficient rank adding means 37 for calculating the sum of the ranks obtained and the correlation coefficient rank adding means 37 for all the combinations of the blocks on the A and B images calculated by the correlation calculating means 34. Add the sum of the ranks obtained in step 2 for each combination, and furthermore, it is proportional to the number of unassigned blocks in each combination.
The AB combination correlation coefficient ranking and determination means 3
5 and a minimum addition order selecting means 38 which is determined from combinations of correspondences which are not judged to have no possibility of correspondence by the correlation coefficient ranking and judging means 36 in the BA direction.

Next, the operation of the image matching method will be described.

The image input means 31, image storage means 32, image division means 33, correlation calculation means 34, and correlation coefficient rank addition means 36 operate in the same manner as the respective means in FIG.

The AB direction correlation coefficient ranking and correspondence judging means 35 uses the correlation coefficient value for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculating means 34. From the block on the image to each block on the B image, ranking is performed in ascending order by a certain number OL in ascending order of the correlation coefficient, and if there is no ranking, there is no possibility of correspondence between the blocks. Is determined.

That is, for a certain block x on the A picture, the rank of the block on the B picture having the highest correlation coefficient value is set to 1;
3, ..., OL and the block on the B image are ranked OR [A
(X) B (y)]. At this time, a set of blocks on the B image that has not been ranked is denoted as NCb (x), and a corresponding set NC that is not included in the corresponding combination.
To

(Equation 1)

[0041]

Is calculated.

The BA direction correlation coefficient ranking and correspondence judging means 36 calculates the correlation coefficient value for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculating means 34. Using A from block on B image
The blocks are ranked in ascending order by a certain number OL in the descending order of the correlation coefficient to each block on the image, and it is determined that there is no possibility of correspondence between the blocks if the blocks are not ranked.

That is, with respect to a certain block x on the B image, the rank of the block on the A image having the highest correlation coefficient value is set to 1;
3, ..., OL and the block on the B image are ranked OR [B
(Y) A (x)]. At this time, a set of blocks on the A image that have not been ranked is denoted as NCa (x), and a corresponding set NC that is not included in the corresponding combination.
To

(Equation 2)

[0045]

Is calculated.

The correlation coefficient rank adding means 37 calculates the correspondence NC determined to have no possibility of correspondence among the correspondences of the blocks on the A and B images calculated by the correlation calculation means 34. Excluding the order OR [A (x) B (y)] obtained by the AB direction correlation coefficient ranking and determination means 35
W of the OR and the rank OR [B (y) A (x)] obtained by the BA direction correlation coefficient ranking and determination means 36
Calculate (x, y) by the following formula: W (x, y) = OR [A (x) B (y)] + OR [B (y) A (x)] Number order adding means 3
For all combinations of the blocks on the A and B images calculated in step 7, the sum of the ranks W (x, y) is added for each combination, and further, a block that does not give a correspondence in each combination is added.
A penalty proportional to the number of locks is added, and the combinations of the correspondence that minimize the result are obtained from the correspondences excluded from the correspondence set NC.

FIG. 4 is a block diagram showing an embodiment of the fourth invention.

The image matching method comprises an image input means 41 for inputting two images (A image and B image), an image storage means 42 for storing an image input by the image input means, and an image storage means A image stored in
An image dividing unit 43 that divides the B image for each scanning line, and further divides the scanning line into segments while allowing overlap, and an image on the A image generated by the image dividing unit 43 that exists on the scanning line at the same position. Correlation calculating means 44 for calculating the cross-correlation of the pixel values for all or some of the correspondence between the segments and the segments on the B image, and the correspondence between the segments on the A and B images calculated by the correlation calculating means. , Using the value of the correlation coefficient for each correspondence, assigning the ascending order of the correlation coefficient from the segment on the A image to the segment on the B image in ascending order;
A correlation coefficient ranking means 45 for ranking the segments on the image in ascending order of the correlation coefficient, and a segment between the A and B images calculated by the correlation coefficient ranking means. For each correspondence, the rank of the correlation coefficient from the segment on the A image to the segment on the B image, and B
Correlation coefficient rank adding means 46 for calculating the sum of the rank of the correlation coefficient from the segment on the image to the segment on the A image
And this correlation coefficient rank addition means,
For the correspondence between the segments on the A and B images, a path generation unit 47 that generates not only a path between adjacent correspondences but also a path between non-adjacent correspondences and a path generated by the path generation means. For all corresponding combinations represented by
6 is added for each combination, and a penalty proportional to the number of unassigned blocks in each combination is added, and the corresponding combination that minimizes the result is added. And a minimum addition rank selecting means 48 for determining the sum using dynamic programming.

Next, the operation of the image matching method will be described.

The image input means 41, the image storage means 42, the correlation calculation means 44, the correlation coefficient ranking means 45, and the correlation coefficient rank addition means 46 operate in the same manner as the respective means in FIG.
The image dividing means 43 divides the image for each scanning line, and further divides each scanning line into blocks.

A description will now be given, with reference to FIG. 9, of the minimum addition order selecting means 48 using a dynamic programming method that allows discontinuous paths. This figure shows a search plane composed of an axis x on the scanning line A and an axis y on the scanning line B. In the search plane, the corresponding range calculated by the correlation calculation unit 44 is represented by S. In addition, each grid point on the search plane has a sum W of ranks in both directions.
(X, y) is given. The minimum addition order selecting means selects a path which allows discontinuity between p1 and p2 and which minimizes W (x, y) given to a grid point on the path.

The minimum addition order selecting means considers a straight line L such that x + y = t, and increases L by 1 to make L equal to p.
Move from position 1 to position p2 in order. At each stage of moving L, for the lattice points on L included in S, the minimum arrival cost up to the lattice point is calculated. For example, the minimum arrival cost r up to the grid point u (x, y)
c (x, y) is obtained as follows. rc (x, y) = min ｛rc (x-1, y-1) + W (x, y), rc (x-1, y) + W (x, y), rc (x-2, y) + W (X, y) + α, rc (x−3, y) + W (x, y) + 2α, rc (x−4, y) + W (x, y) + 3α,..., Rc (x, y−1) + W (x, y), rc (x, y−2) + W (x, y) + α, rc (x, y−3) + W (x, y) + 2α, rc (x, y−4) + W (x , Y) + 3α,...｝ Where α is a penalty for creating one uncorrelated block. In the figure, the correspondence (x, y) is a black square, the correspondences (x-1, y-1), (x-1, y),
(X−2, y), (x−3, y),... And corresponding (x, y−1), ((x, y−2), (x, y−3),
.. Are indicated by black circles. At the point where L arrives at p2, the path giving the minimum arrival cost at p2 is the corresponding combination provided by the fourth invention.

[0054]

As described above, according to the image matching method of the present invention, the possibility of providing a correspondence to the occlusion area is reduced. For this reason, it is less likely to forcibly correspond to places where matching cannot be performed originally, and the accuracy of matching is improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the first invention.

FIG. 2 is a block diagram showing an embodiment of the second invention.

FIG. 3 is a block diagram showing an embodiment of the third invention.

FIG. 4 is a block diagram showing an embodiment of the fourth invention.

FIG. 5 is a diagram for explaining an example in which an erroneous response is given in the conventional method when there is an occlusion area.

FIG. 6 is a diagram for explaining an example showing that accurate correspondence is given in the image matching method of the present invention.

FIG. 7 is a diagram for explaining an example in which an occlusion region is indicated by discontinuous paths in one-dimensional image matching.

FIG. 8 is a diagram showing a difference between a conventional dynamic programming and a dynamic programming which allows discontinuous paths.

FIG. 9 is an explanatory diagram of a dynamic programming method that allows discontinuous paths.

[Explanation of symbols]

 Reference Signs List 11 image input means 12 image storage means 13 image division means 14 correlation calculation means 15 correlation coefficient ranking means 16 correlation coefficient rank addition means 17 minimum addition rank selection means 21 image input means 22 image storage means 23 image division means 24 correlation Calculation means 25 Correlation coefficient ranking means 26 Cost calculation means 27 Cost addition means 28 Minimum addition cost selection means 31 Image input means 32 Image storage means 33 Image division means 34 Correlation calculation means 35 AB direction correlation coefficient ranking and correspondence judgment Means 36 BA direction correlation coefficient ranking and correspondence determination means 37 Correlation coefficient rank addition means 38 Minimum addition rank selection means 41 Image input means 42 Image storage means 43 Image division means 44 Correlation calculation means 45 Correlation coefficient ranking means 46 Correlation coefficient rank adding means 47 Path generating means 48 Minimum addition Order selection means

Claims (4)

(57) [Claims] An image matching method comprising the following a to g. a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculation means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image division means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from a block on the B image to a block on the A image in descending order of the correlation coefficient f. For each correspondence between the blocks on the A and B images calculated by the correlation calculating means, the block B on the A image obtained by the correlation coefficient ranking means is used.
Correlation coefficient rank adding means for calculating the sum of the rank of the correlation coefficient to the block on the image and the rank of the correlation coefficient from the same block on the B image to the same block on the A image g. For all combinations of blocks on the A and B images calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and further, Number of blocks that do not give a correspondence
Minimum addition order selecting means for obtaining a number proportional to the above , adding that as a penalty to the addition result of the combination, and obtaining a combination that minimizes the result. 2. An image matching method comprising the following a to h. a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculation means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image division means e. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the A image to the block on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from a block on the B image to a block on the A image in descending order of the correlation coefficient f. Cost calculating means for converting the rank value into a cost value by a predetermined monotone increasing function with respect to the rank obtained by the correlation coefficient ranking means; g. For each correspondence between the blocks on the A and B images calculated by the correlation calculating means, the cost from the block on the A image to the block on the B image, obtained by the cost calculating means, and the B image Cost adding means for calculating the sum of costs from the same block above to the same block on the A image h. For all combinations of blocks on the A and B images calculated by the correlation calculating means, the sum of the ranks obtained by the cost adding means is added for each combination, and the correspondence in each combination is further added. Is proportional to the number of blocks that do not give
Means for calculating the number of the combination, adding the result as a penalty to the addition result of the combination, and obtaining the combination that minimizes the result. 3. An image matching method comprising the following a to h. a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. An image dividing unit that divides the A image and the B image stored in the image storage unit into blocks while allowing overlap; d. Correlation calculation means for calculating the cross-correlation of the pixel values included in the blocks for all or some of the correspondence between the blocks on the A image and the blocks on the B image generated by the image division means e. The value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculating means is used to increase the correlation coefficient from the block on the A image to each block on the B image. AB order correlation coefficient ranking and correspondence determination means for sequentially determining a certain number in ascending order and determining that there is no possibility of correspondence between the blocks for the unordered ones. F. Using the value of the correlation coefficient for each correspondence between the block on the A image and the block on the B image obtained by the correlation calculation means, the correlation coefficient from the block on the B image to each block on the A image is large. BA order correlation coefficient ranking and correspondence judging means for judging in ascending order by a certain number in order and determining that there is no possibility of correspondence between the blocks for those not ranked g. In the correspondence of the blocks on the A and B images calculated by the correlation calculation means, by the AB direction correlation coefficient ranking and correspondence determination means and the BA direction correlation coefficient ranking and correspondence determination means And the order obtained by the AB direction correlation coefficient ranking and determination means, and the order obtained by the BA direction correlation coefficient ranking and determination means, except for the correspondence determined to be impossible. Correlation coefficient rank adding means for calculating the sum of h. For all combinations of the blocks on the A and B images calculated by the correlation calculation means, the sum of the ranks obtained by the correlation coefficient rank addition means is added for each combination, and further, Number of unassigned blocks in
Is calculated as a penalty, and the union
In addition to the summation result, there is no possibility that the AB direction correlation coefficient ranking and judging means and the BA direction correlation coefficient ordering and judging means will correspond in the correspondence combinations that minimize the result. Means for selecting the minimum addition order from the corresponding combinations that have not been determined. 4. An image matching method comprising the following a to h. a. Image input means for inputting two images (A image and B image) b. Image storage means for storing the image input by the image input means c. Image dividing means for dividing the A image and the B image stored in the image storage means for each scanning line, and further dividing the scanning lines into segments while allowing overlaps. D. Correlation calculating means e for calculating the cross-correlation of pixel values for all or some correspondences between segments on the A image and segments on the B image generated by the image dividing means, which are present on the scanning line at the same position. . For the correspondence between the segments on the A and B images calculated by the correlation calculation means, the value of the correlation coefficient for each correspondence is used, and the correlation coefficient from the segment on the A image to the segment on the B image is large. Correlation coefficient ranking means for ranking in ascending order, and in ascending order from the segment on the B image to the segment on the A image in descending order of the correlation coefficient f. For each correspondence between the segments on the A and B images calculated by the correlation coefficient ranking means, the order of the correlation coefficient from the segment on the A image to the segment on the B image, Correlation coefficient rank adding means for calculating the sum of the rank of the correlation coefficient from the segment to the segment on the A image g. The calculation was performed by the correlation coefficient rank adding means,
Path generation means for generating not only paths between adjacent correspondences but also paths between non-adjacent correspondences regarding the correspondence between segments on the A and B images h. For all the corresponding combinations represented by connecting the paths generated by the path generating means, the sum of the ranks determined by the correlation coefficient rank adding means is added for each combination, and further, Don't give me a medium response
Find a number proportional to the number of new blocks and penalize it.
Means for selecting the minimum addition order by using a dynamic programming method to determine the corresponding combination that minimizes the result in addition to the addition result of the combination.