JPH09305760A - Method and device for matching form - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform stable and highly accurate form matching. SOLUTION: An affine transformation component to be applied to the respective points of an input pattern is determined by the method of weighted least square method while using a Gaussian function type weight coefficient based on an inter-point distance between the respective constitutive points of an input pattern and a standard pattern. Between a deformed input pattern generated by using the affine transformation component and the standard pattern, the average value of a distance between most adjacent points is calculated as an inter-pattern distance D1 and similarly, an inter-pattern distance between the input pattern and the standard pattern is also calculated as an inter-pattern distance D0 . When the D1 is not decreased from the D0 , the paired most adjacent points between the deformed input pattern and the standard pattern are outputted as the form matching result. When the D1 is decreased from the D0 , the deformed input pattern is regarded as the input pattern, the affine transforming operation is repeated and when the D1 is not decreased from the D0 , the paired most adjacent points are outputted as the form matching result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention determines the displacement / deformation of a pattern in the fields of computer-based pattern recognition, moving image analysis, stereovision, etc., by associating points between two point sets on a two-dimensional plane. Pattern matching method and device.

[0002]

2. Description of the Related Art Conventionally, shape matching techniques are roughly classified into two types: (1) combination search type and (2) energy minimization type.

The former has an inherent problem of processing amount divergence because it searches for an optimal solution from the total number of combinations of point factorial orders of points. In order to avoid the divergence of this processing amount, pruning of the search tree using an appropriate constraint is essential. Under these constraints, so far, the problem of determining whether two sets of points match under the congruential transformation (rotation / translation) and the similarity transformation (rotation / expansion / translation) Regarding the problem of determining whether or not to do it, a solution algorithm with a throughput of a power of points has been reported (for example, Shinji Umeyama, “Point Pattern Matching Algorithm,” Theoretical theory (D-II), vol. J72-
D-II, no. 2, pp. 218-228, Feb.
1989). However, there are problems such as it is difficult to give versatility to the constraint condition for pruning, there is still a large amount of processing, and a solution algorithm has not yet been found for the affine transformation that further distorts the similarity transformation. there were.

On the other hand, the latter is formulated as an energy minimization problem of a dynamic system, and the variational method, the relaxation method and the dynamic programming method are used to determine locally continuous matching by repeatedly applying a small displacement. For these conventional techniques, see, for example, Sakagami, Yokoya, "Relative Method and Regularization," Information Processing, vol. 3
0, no. 9, pp. 1047-1057, Sept.
1989, and Ohta, Yamada, "Pattern Matching by Dynamic Programming," Information Processing, vol. 30, no. 9,
pp. 1058-1066, Sept. 1989, see. These techniques have the advantage of being able to handle the matching problem analytically or algebraically, but it has been difficult in principle to find a global solution for optimum matching by stacking small displacements for finite displacements. On the other hand, in order to handle the finite displacement, a method of algebraically determining the optimum local affine transformation for each neighborhood of each point of the pattern (Japanese Patent Application No. 63-
281985) was proposed. But with this technique,
There are problems that trial and error are required to control the size parameter in the neighborhood, and the amount of processing is enormous because the affine transformation is determined for each point.

[0005]

As described above, as a shape matching technique for determining displacement / deformation of a pattern by associating points between two point sets, a combination search type search tree with appropriate constraint conditions is used. There have been considered methods for pruning, and energy-minimization type to realize local continuous matching by stacking small displacements or use local affine transformation.

However, each of them has drawbacks, and a method capable of handling a finite and wide range of displacement / deformation that is not minute with a practical processing amount has not been proposed.

An object of the present invention is to handle a wide range of displacements / deformations expressed by arbitrary affine transformations (rotation / expansion / contraction / translation) between patterns, and iterating weighted least squares method for the affine transformations. A shape matching method and apparatus for realizing a stable and highly accurate shape matching based on point correspondence between a deformed input pattern deformed by the affine transformation and a standard pattern, which is strictly determined by a practical processing amount by application. To provide.

[0008]

The shape matching method of the present invention is an input pattern represented by a set of points on a two-dimensional plane.

[0009]

[Outside 39] And standard patterns

[0010]

[Outside 40] Between

[0011]

[Outside 41] Affine transformation is applied to the transformed input pattern

[0012]

[Outside 42] Introduce a conversion operation to generate a standard pattern

[0013]

[Outside 43] Input pattern that maximizes the overlap with

[0014]

[Outside 44] By generating the transformed input pattern

[0015]

[Outside 45] And standard patterns

[0016]

[Outside 46] The nearest neighbor point pair between is output as a matching result.

According to the embodiment of the present invention, the weighting coefficient is calculated based on the point-to-point distance between the constituent points of the input pattern and the standard pattern, and the affine transformation is performed by the weighted least squares method using the weighting coefficient. Determine the ingredients.

Further, the shape matching device of the present invention has two types.
Input pattern represented by a set of points on a dimensional plane

[0019]

[Outside 47] And standard patterns

[0020]

[Outside 48] Is the set of position vectors

[0021]

[Outside 49] As an input pattern

[0022]

[Outside 50] Affine transformation to each point of

[0023]

[Outside 51] (A is a matrix with 2 rows and 2 columns.
Represents distortion,

[0024]

[Outside 52] Is a two-dimensional vector, which means parallel movement)

[0025]

[Outside 53] Input pattern that maximizes the overlap with

[0026]

[Outside 54] A shape matching device that generates points to associate points between two patterns,

[0027]

[Outside 55] And standard patterns

[0028]

[Outside 56] Point-to-point distance between each constituent point of

[0029]

[Outside 57] Point weight coefficient calculating means for calculating a Gaussian function type weight coefficient based on the above, and an input pattern by a weighted least squares method using the weight coefficient.

[0030]

[Outside 58] Optimal affine transformation component for

[0031]

[Outside 59] Affine transformation component simultaneous linear equation generation means for generating simultaneous linear equations to be satisfied, affine transformation component determination means for solving the simultaneous linear equations and determining affine transformation components, and input using the determined affine transformation components pattern

[0032]

[Outside 60] Affine transformation to each point of

[0033]

[Outside 61] Input pattern

[0034]

[Outside 62] Transform and transform input pattern

[0035]

[Outside 63] Input pattern generating means for generating

[0036]

[Outside 64] And standard patterns

[0037]

[Outside 65] The average value of the distances between the nearest neighbor points from each point of one pattern to the nearest neighbor point in the other pattern between the two patterns

[0038]

[Outside 66] Input pattern before transformation

[0039]

[Outside 67] And standard patterns

[0040]

[Outside 68] The distance between the patterns between

[0041]

[Outside 69] Inter-pattern distance calculation means for calculating as

[0042]

[Outside 70] Compare

[0043]

[Outside 71] Input pattern if not decreased from

[0044]

[Outside 72] And standard patterns

[0045]

[Outside 73] And output the shape matching result by associating the closest point in the opponent pattern with each point between

[0046]

[Outside 74] Deformation input pattern when decreasing from

[0047]

[Outside 75] Input pattern again

[0048]

[Outside 76] And an input to the point-to-point weighting factor calculation means again, and a convergence determination means for repeating the affine transformation operation.

The first feature of the present invention is to handle arbitrary affine transformation as a range of displacement / deformation. The allowable range of displacement / deformation is much wider than that of the congruential conversion and the similarity conversion, which are the targets of the prior art.

The second feature of the present invention resides in that the affine transformation component is strictly determined by iterative application of the weighted least squares method. In particular, this is achieved with a practical throughput. The processing is a general-purpose method that does not include any control parameters.

As described above, the present invention is a general-purpose shape matching method for determining the optimum point correspondence with a practical processing amount for a wide range of displacement / deformation such as arbitrary affine transformation, and is described above. It solves the problems of the prior art.

[0052]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of a shape matching device according to an embodiment of the present invention.

This shape matching apparatus includes an input pattern constituent point storage unit 1, a standard pattern constituent point storage unit 2, an inter-point weighting coefficient calculation unit 3, and an affine transformation component simultaneous linear equation generation unit 4.
It is composed of an affine transformation component determination unit 5, a modified input pattern generation unit 6, an inter-pattern distance calculation unit 7, and a convergence determination unit 8.

The functions of the respective parts will be specifically described below.

The input pattern constituent point storage unit 1 is an input pattern represented by a set of points on a two-dimensional plane.

[0057]

[Outside 77] Set of position vectors of each point

[0058]

[Outside 78] Is stored. here,

[0059]

[Outside 79] Represents the position vector of the i-th point of X, and the X and Y coordinate values are stored. However, the order of the points is arbitrary. m is the input pattern

[0060]

[Outside 80] Represents the total score of.

The standard pattern constituent point storage unit 2 has the
Standard pattern represented by a set of points on a dimensional plane

[0062]

[Outside 81] Set of position vectors of each point

[0063]

[Outer 82] Is stored. here,

[0064]

[Outside 83] Represents the position vector of the j-th point of, and the X and Y coordinate values are stored. However, the order of the points is arbitrary. n is a standard pattern

[0065]

[Outside 84] Represents the total score of.

The point-to-point weighting coefficient calculation unit 3

[0067]

[Outside 85] Point-to-point distance between each constituent point of

[0068]

[Outside 86] Gaussian function type weighting factor

[0069]

[Outside 87] Is calculated by the following equation. However

[0070]

[Outside 88] Represents the norm of the vector, and for example, the Euclidean norm may be used.

[0071]

[Equation 1] here,

[0072]

[Outside 89] Is the input pattern

[0073]

[Outside 90] And standard patterns

[0074]

[Outside 91] Is the average value of the distances between the closest points from each point of one pattern to the closest point in the other pattern, and gives the distance between patterns before affine transformation. Weighting factor calculated by equation (1)

[0075]

[Outside 92] Is sent to the affine transformation component simultaneous equation generation unit 4.

The affine transformation component simultaneous equation generation unit 4
Affine transformation component for the input pattern that maximizes the overlap with the standard pattern

[0077]

[Outside 93] Generate a system of linear equations for determining. here,
A is a matrix with 2 rows and 2 columns, and represents rotation, expansion, and distortion operations of the position vector,

[0078]

[Outside 94] Represents the translation of the position vector, and the input pattern

[0079]

[Outside 95] Points that make up

[0080]

[Outside 96] Is converted to Here, the affine transformation component for the input pattern that maximizes the overlap with the standard pattern

[0081]

[Outside 97] Is to minimize the objective function Ψ of the following equation. Clearly, Ψ is the weighting factor

[0082]

[Outside 98] It is a weighted least squares method using.

[0083]

[Equation 2] Optimal affine transformation component that minimizes Ψ in equation (3)

[0084]

[Outside 99] Is Ψ

[0085]

[Outside 100] It is determined from the condition that the value of partial differentiation of each component becomes zero.

[0086]

(Equation 3) Unknown affine transformation component

[0087]

[Outside 101] A six-dimensional system of linear equations for is obtained. However, t represents the transposition of the vector. Here, it is a great advantage that the equation (4) does not include any control parameters. The affine transformation component simultaneous equations generated by the above equation (4) are sent to the affine transformation component determination unit 5.

The affine transformation component determination unit 5 uses a known numerical solution method, for example, the Gaussian elimination method (Iwanami Course, Applied Mathematics, "Linear Computation" Chapter 1, Iwanami Shoten, 19
Input pattern solved by (see 94)

[0089]

[Outside 102] Optimal affine transformation component for

[0090]

[Outside 103] To determine. The affine transformation component thus obtained is sent to the modified input pattern generation unit 6.

The modified input pattern generator 6 receives the affine transformation component sent from the affine transformation component determiner 5.

[0092]

[Outside 104] Input pattern using

[0093]

[Outside 105] By

[0094]

[Outside 106] Deform to.

[0095]

(Equation 4) Deformation input pattern by the above equation (5)

[0096]

[Outside 107] Is generated and transformed input pattern

[0097]

[Outside 108] Is sent to the inter-pattern distance calculation unit 7.

The inter-pattern distance calculation unit 7 uses the modified input pattern.

[0099]

[Outside 109] And standard patterns

[0100]

[Outside 110] The average value of the distances of the nearest neighbors between

[0101]

[Outside 111] Is calculated by the following formula.

[0102]

(Equation 5) Similarly, the input pattern before affine transformation

[0103]

[Outside 112] And standard patterns

[0104]

[Outside 113] The average value of the distances between the nearest neighbor points between

[0105]

[Outside 114] Is calculated by the above equation (2).

The value of the inter-pattern distance before and after these affine transformation operations

[0107]

[Outside 115] Is sent to the convergence determination unit 8.

The convergence determination unit 8 is the inter-pattern distance calculation unit 7
Distance between two patterns sent from

[0109]

[Outside 116] Compare

[0110]

[Outside 117] Input pattern if not decreased from

[0111]

[Outside 118] And standard patterns

[0112]

[Outside 119] The pair of nearest neighbor points is determined and output as the shape matching result. on the other hand,

[0113]

[Outside 120] Deformation input pattern when decreasing from

[0114]

[Outside 121] Input pattern again

[0115]

[Outside 122] And the affine transformation operation determined by the weighted least squares method is performed again, and the distance between patterns after transformation is changed.

[0116]

[Outside 123] Is the distance between patterns before conversion

[0117]

[Outside 124] This operation is repeated until the value does not decrease. Thus the distance between patterns

[0118]

[Outside 125] Decreases monotonically, and when it converges, the pair of nearest neighbor points between the two patterns is output as the shape matching result, and the operation ends.

FIG. 2 shows a character pattern "type" on a two-dimensional plane.
As an example, the input pattern (composition points are marked with a circle) in FIG. 7A and the standard pattern (composition points are marked with a circle) in FIG.
And the input pattern is shown. However, ● indicates an overlapping point between the input pattern and the standard pattern. FIGS. 7C to 7F are diagrams in which the deformed input pattern and the standard pattern when the affine transformation operation is performed for the number of iterations = 1, 3, 7, and 15 are superimposed on the input pattern. In the figure,
“•” indicates the position of the original constituent point of the input pattern. However, in this example, when the number of iterations = 15, the decrease in the inter-pattern distance converges, and the iteration of the affine transformation operation ends.

From FIG. 2, as the affine transformation is repeated, the number of overlapping points (indicated by ●) increases from the state of FIG. 2C toward the final state of FIG.
In the final state of (f), most of the constituent points of the modified input pattern overlap the standard pattern. In this way, the most adjacent point pair in the final state (FIG. 2 (f)) is output as the shape matching result. It can be seen that even when the input pattern includes a considerably large deformation as shown in this figure, correct shape matching is realized by repeating the affine transformation operation.

[0121]

As described above, according to the present invention,
For an input pattern including an arbitrary affine transformation component (rotation, expansion, distortion, translation), the affine transformation is rigorously calculated with a practical processing amount by iterative application of the weighted least squares method, and the affine transformation is performed. By determining the nearest neighbor point pair between the transformed input pattern transformed by the transformation and the standard pattern, stable and highly accurate shape matching can be performed. In particular, since the weighted least squares method can be reduced to the solution of simultaneous linear equations, the procedure is simple and the processing amount is small. Further, since the affine transformation calculation process does not include any control parameters, it is an extremely general-purpose shape matching technique.

As described above, since it is considered that the extraction of the displacement / deformation between the patterns widely corresponds to the point correspondence between the two point sets, the present invention is a small process for a wide range of displacement / deformation such as arbitrary affine transformation. We provide a general-purpose shape matching technology that enables highly accurate point correspondence by quantity. Therefore, this is extremely advantageous when applied to the field of character / figure deformation extraction in pattern recognition, displacement / deformation extraction in moving image understanding, and stereovision corresponding point detection in three-dimensional depth perception.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a shape matching device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a specific example for explaining an operation of point association by repeating an affine transformation operation. 1 Input pattern constituent point storage unit 2 Standard pattern constituent point storage unit 3 Point weight coefficient calculation unit 4 Affine transformation component simultaneous linear equation generation unit 5 Affine transformation component determination unit 6 Modified input pattern generation unit 7 Pattern distance calculation unit 8 Convergence Judgment part

Claims (3)

[Claims] 1. An input pattern represented by a set of points on a two-dimensional plane. And standard pattern [Outside 2] Input pattern [External 3] Affine transformation is applied to the transformed input pattern [External 4] Introduce the conversion operation to generate the standard pattern Input pattern that maximizes the overlap with By generating the transformed input pattern And standard pattern [8] A shape matching method that outputs the pair of points closest to each other as a shape matching result. 2. A weighting factor is calculated based on a point-to-point distance between each of the constituent points of the input pattern and the standard pattern,
The shape matching method according to claim 1, wherein the affine transformation component is determined by a weighted least squares method using the weighting factor. 3. An input pattern represented by a set of points on a two-dimensional plane. And standard pattern [Outside 10] Is a set of position vectors of constituent points Stored in the storage device as Affine transformation to each point of (A is a matrix with 2 rows and 2 columns.
Represents distortion, [External 14] Is a two-dimensional vector, and represents a parallel movement) Input pattern that maximizes the overlap with A shape matching device for generating points and associating points between two patterns, the input pattern And standard pattern [Outside 18] Inter-point distance between each constituent point of Point-to-point weighting coefficient calculating means for calculating a Gaussian function-type weighting coefficient based on, and an input pattern by a weighted least squares method using the weighting coefficient Optimal affine transformation component for Affine transformation component simultaneous linear equation generation means for generating simultaneous linear equations to be satisfied, affine transformation component determination means for determining the affine transformation component by solving the simultaneous linear equations, and inputting using the determined affine transformation component Pattern [outside 22] Affine transformation to each point of Input pattern by applying [External 24] Transform and transform input pattern [External 25] A modified input pattern generating means for generating And standard pattern [External 27] The average value of the distances between the nearest neighbor points from each point of one pattern to the nearest neighbor point in the other pattern between And the input pattern before transformation And standard pattern [30] The distance between patterns is also [External 31] Inter-pattern distance calculation means for calculating as Comparing If there is no decrease from the modified input pattern [Ex. 34] And standard patterns [35] And output the shape matching result by associating the nearest neighbor point in the partner pattern with each point between If it is decreasing from the Input pattern again [outside 38] A shape matching device having a convergence determining unit that repeats the affine transformation operation by inputting again to the point-to-point weighting coefficient calculating unit.