JP2007188329A - Image matching device, image matching method, and image matching program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make the global retrieval of an optimum point compatible with the convergence to conduct precise image matching. <P>SOLUTION: This image matching device is provided with an image energy force calculating part 103 for calculating image energy force received by the second lattice point, by a gradient of potential energy based on a position of the each second lattice point of a reference image and a gradient of potential energy based on a position of the each first lattice point of an objective image corresponding to the second lattice point, an elastic energy force calculating part 104 for calculating elastic energy force received from elastic energy between the each second lattice point and the other second lattice point adjacent to the second lattice point, a friction coefficient control part 102 for variable-controlling a friction coefficient, a friction force calculating part 105 for calculating friction force acting on the each second lattice point, using the friction coefficient, an analytical processing part 101 for analytical-processing an equation of motion as to the each second lattice point due to the image energy force, the elastic energy force and the friction force, and a mapping processing part 107 for finding a correspondence relation between the objective image and the reference image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image matching apparatus, an image matching method, and an image matching program that detect and associate corresponding points from two images of a target image and a reference image.

  Image matching technology that detects the corresponding point of each pixel of the other reference image from one target image and obtains the corresponding relationship includes, for example, motion detection in moving images, stereo matching, image morphing, image recognition, and moving image coding Are used in the field of image processing technology.

  Such image matching techniques can be classified into four main types: an optical flow method, a block-based method, a gradient method, and a Bayesian method (see, for example, Non-Patent Document 1).

  The optical flow method derives an optical flow equation that “a change in luminance is constant” and obtains a flow using the optical flow equation as a constraint. The block-based method is a method in which an image is divided into predetermined blocks and a motion is obtained by template matching for each block. The gradient method is a method for performing matching in a direction in which the luminance gradient of an image decreases. The Bayesian method is a technique for obtaining matching that is probabilistically plausible.

  In addition, as a conventional image matching technique, a plurality of multi-resolution image pyramids are generated using a plurality of multi-resolution filters, and the generated image pyramids are sequentially matched from the upper layer to the lower layer, thereby performing a large process. There is an image matching technique with high robustness capable of associating images from movement to small movement (see, for example, Patent Document 1).

A. Murat Tekalp, “Digital Video Processing”, Prentice Hall, 1995 Japanese Patent No. 2927350

  However, such a conventional image matching technique has the following problems. The optical flow method has a problem that it is sensitive to noise and it is inherently difficult to cope with fast movement. In the block-based method, image matching is performed for each block in the image. However, there is a problem that it is difficult to essentially cope with the movement of the object in the image such as deformation or rotation. In the gradient method, since matching is performed in a direction in which the luminance gradient of the image decreases, there is a problem that it is difficult to stably search for the movement of the object. The Bayesian method has a problem that it is difficult to obtain a global optimum.

  On the other hand, in the technique disclosed in Patent Document 1, in principle, a plurality of multi-resolution filters are used to perform matching processing from the highest layer to the lowest layer of the multi-resolution image pyramid. It is a typical optimization process and is not necessarily optimized globally.

  The present invention has been made in view of the above, and is an image matching apparatus, an image matching method, and an image that can perform more accurate image matching while achieving both search for a global optimum point and convergence. The purpose is to provide a matching program.

  In order to solve the above-described problems and achieve the object, the present invention is an image matching device for obtaining a correspondence relationship between a target image and a reference image, and a plurality of first lattice points on the target image. And a potential energy based on the correlation of the image between each of the second grid points corresponding to the first grid point on the reference image on a one-to-one basis, The image energy for calculating the image energy force received by the second lattice point by the gradient of the potential energy based on the position of the lattice point and the position of the first lattice point corresponding to the second lattice point A force calculator, an elastic energy force calculator that calculates an elastic energy force received from elastic energy between each second lattice point and another second lattice point adjacent to the second lattice point; Each A friction coefficient control unit that variably controls a friction coefficient acting on a frictional force acting on two grid points, and a friction that computes a frictional force acting on each second lattice point using the friction coefficient From the result of the solution processing unit by the force calculation unit, the solution processing unit that solves the equation of motion related to each second lattice point by the image energy force, the elastic energy force, and the friction force, and the solution processing by the solution processing unit A mapping processing unit for obtaining a correspondence relationship between the target image and the reference image.

  The present invention also provides an image matching method and an image matching program that can be executed by the image matching apparatus.

  According to the present invention, both the search for the global optimum point and the convergence are achieved by calculating the friction coefficient that is variably controlled and calculating the friction force acting on each second grid point on the reference image. Thus, there is an effect that more accurate image matching can be performed.

  Exemplary embodiments of an image matching apparatus, an image matching method, and an image matching program according to the present invention are explained in detail below with reference to the accompanying drawings.

  Note that the image matching device according to the following embodiment is applied to a motion compensation device that generates a compensated image by performing motion compensation of a moving image, as well as stereo matching, image morphing, image recognition, moving image coding, and the like. It can also be applied to image processing technology.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration of the image matching apparatus according to the first embodiment. As shown in FIG. 1, the image matching apparatus 100 according to the present embodiment includes a solution processing unit 101, an image energy force calculation unit 103, an elastic energy force calculation unit 104, a friction force calculation unit 105, and a friction coefficient. A control unit 102, a mapping processing unit 107, and a frame memory 106 are mainly provided.

  In the image matching device 100 according to the first embodiment, each lattice point obtained by dividing each of the target image and the reference image into a predetermined lattice shape is treated as a dynamic mass point, and the first lattice point and the first lattice point of the target image are processed. Considering the potential energy based on the correlation between the second grid points of the reference image corresponding to the grid points one-to-one, the gradient of the potential energy based on the positions of the first and second grid points The second energy due to the image energy force received by the second lattice point by the elastic energy force received from the elastic energy between the lattice points adjacent to the second lattice point and the frictional force generated at the second lattice point. The equation of motion related to the lattice point is used as an image matching state transition model, and the solution equation is processed to obtain the equilibrium state of the second lattice point, and the first lattice point and the first lattice point are obtained. And performing image matching by seeking the mapping relationship between the lattice points.

  In the image matching apparatus 100 according to the first embodiment, when the friction force is calculated, the optimization process can be performed globally by variably controlling the friction coefficient of the friction force.

  The solution processing unit 101 stores an image between each of the plurality of first grid points on the target image and each of the plurality of second grid points corresponding to the first grid points on the reference image. The image energy force received by the second lattice point due to the potential energy gradient based on the correlation, the elastic energy force received from the elastic energy between the lattice points adjacent to the second lattice point, and the second lattice point It is a processing unit that solves the equation of motion related to the second lattice point by the generated frictional force by numerical analysis processing by the Euler method as a discrete variable method.

  The mapping processing unit 107 obtains the equilibrium state of the second lattice point from the result of the solution processing by the solution processing unit 101, thereby matching the first lattice point on the target image with the second lattice point on the reference image. It is a processing unit for obtaining a mapping relationship that is a relationship.

  The image energy force calculation unit 103 is a processing unit that calculates the image energy force received by the second lattice point based on the gradient of the potential energy based on the positions of the first lattice point and the second lattice point.

  The elastic energy force calculation unit 104 is a processing unit that calculates the elastic energy force received from the elastic energy between the lattice points adjacent to the second lattice point.

  The friction coefficient control unit 102 is a processing unit that variably controls the friction coefficient used for calculating the friction force acting on each second lattice point. Specifically, the friction coefficient control unit 102 variably controls the friction coefficient by calculating the friction coefficient so as to increase according to a linear function of the time as time passes in the equation of motion.

  The frictional force calculation unit 105 is a processing unit that calculates the frictional force generated at the second grid point using the friction coefficient.

  The frame memory 106 is a memory for storing the input target image and reference image. Detailed processing of each unit will be described later.

  Next, an image matching state transition model introduced by the image matching apparatus 100 according to the present embodiment will be described.

The pixel values of the target image and the reference image at the point x∈E ² on the two-dimensional Euclidean space are shown by the formula (1).

（１）式は実数空間における連続画像モデルであり、本実施の形態ではデジタル画像を扱うため、２次元格子空間上の点ｎ∈Ｎ²について（２）式に示す離散画像モデルを考える。

Equation (1) is a continuous image model in real number space. In this embodiment, in order to handle digital images, a discrete image model shown in Equation (2) is considered for a point n∈N ² on a two-dimensional lattice space.

従って、連続画像モデルの画素値はデータ補間関数ｇを用いて、（３）式に示すように定義される。ここで、データ補間関数ｇとしては、例えば双曲形補間関数を使用することができる。

Therefore, the pixel value of the continuous image model is defined as shown in the equation (3) using the data interpolation function g. Here, as the data interpolation function g, for example, a hyperbolic interpolation function can be used.

本実施の形態では、対象画像から参照画像への画像マッチング問題を、対象画像上の曲面Ｘ⊂Ｅ²から参照画像上の曲面Ｙ⊂Ｅ²への写像として求めることにする。図２は、対象画像上の曲面から参照画像上の曲面への写像を示す説明図である。図３は、対象画像上の曲面Ｘの点ｘから参照画像上の曲面Ｙの点ｙへの写像を示す説明図である。すなわち、図２および３に示すように、画像マッチングは、対象画像上の曲面Ｘの点が参照画像上の曲面Ｙのどの点に対応しているのかを表す写像を求める問題として扱う。

In this embodiment, to obtaining an image matching problem from the target image to the reference image, as a mapping from the curved X⊂E ² on the target image to the curved surface Y⊂E ² on the reference image. FIG. 2 is an explanatory diagram showing mapping from the curved surface on the target image to the curved surface on the reference image. FIG. 3 is an explanatory diagram showing a mapping from a point x on the curved surface X on the target image to a point y on the curved surface Y on the reference image. That is, as shown in FIGS. 2 and 3, image matching is handled as a problem of obtaining a mapping that indicates which point of the curved surface Y on the reference image corresponds to the point of the curved surface X on the target image.

  Deriving such a mapping condition is as follows. Using the parameter (u, v), the point on the curved surface X is represented as x (u, v) εX, the point on the curved surface Y is represented as y (u, v) εY, and the parameter (u, v) and the point are represented. When x has the same scale, the point x (u, v) on the curved surface X can be expressed by the equation (4) by the parameter (u, v).

ここで、曲面Ｘから曲面Ｙへの求めるべき写像を（５）式で定義する。写像の対応関係は曲面Ｘ上の点ｘが一意に曲面Ｙに対応づけることができればよいので、写像ｇはすべての点が一意に決定する全単射とする。よって同一パラメータ（ｕ，ｖ）を有する点ｘ（ｕ，ｖ）、ｙ（ｕ，ｖ）は，（６）式を満たすことになる。

Here, the mapping to be obtained from the curved surface X to the curved surface Y is defined by equation (5). Since the mapping relationship is sufficient if the point x on the curved surface X can be uniquely associated with the curved surface Y, the mapping g is bijection in which all points are uniquely determined. Therefore, the points x (u, v) and y (u, v) having the same parameter (u, v) satisfy the expression (6).

このことは、すなわち、写像ｇについて考えることは、曲面Ｙそのものを考えることと同じである。一つの対象物が対象画像と参照画像に射影されて映っていると考えると、射影された対象画像上の像と参照画像上の像は、画素値が同じであると考えることができる。従って、対応づけるべき同一点の画素値が等しいと仮定すると、写像ｇに関して（７）式の条件式が導出される。かかる（７）式をパラメータｕ＝（ｕ，ｖ）を用いて表示すると、（８）式のようになる。

This means that thinking about the mapping g is the same as considering the curved surface Y itself. If one object is projected on the target image and the reference image, it can be considered that the projected image on the target image and the image on the reference image have the same pixel value. Therefore, assuming that the pixel values of the same point to be matched are equal, the conditional expression (7) is derived for the mapping g. When the equation (7) is displayed using the parameter u = (u, v), the equation (8) is obtained.

従って、画像マッチング問題は、（８）式の条件式を満たす写像ｇを求めるという問題に帰着することになる。

Therefore, the image matching problem results in a problem of obtaining a mapping g that satisfies the conditional expression (8).

  Next, a solution process for the image matching problem will be described. Expression (8) can be modified like Expression (9).

しかしながら、（９）式は、画像に含まれるノイズなどにより厳密には成立しないため、かかる式から直接写像ｇを求めることは困難である。このため、本実施の形態では、最小二乗法を利用している。最小二乗法によれば、（１０）式に示すエネルギー関数を考え、かかるエネルギー関数の最小化を行うことによりノイズが含まれていても条件を満足する写像ｇを求めることができる。

However, since the expression (9) is not strictly established due to noise included in the image, it is difficult to directly obtain the mapping g from such an expression. For this reason, the least square method is used in the present embodiment. According to the least square method, the energy function shown in the equation (10) is considered, and by minimizing the energy function, a mapping g that satisfies the condition can be obtained even if noise is included.

大数の法則が成立する程度の標本数があれば、最小二乗法により解が安定に求まると考えられるが、本実施の形態では、標本数が一つであるため、やはりノイズの影響を受けてしまう。解を安定して求めるためには、ベイズの定理により、近傍では滑らかに変化するという事前知識により事後分布の精度を高めるアプローチ、例えば、写像ｇに対して構造を規定して力学的なアプローチを考える。

If the number of samples is sufficient to satisfy the law of large numbers, the solution can be obtained stably by the least squares method. However, in this embodiment, since there is only one sample, it is still affected by noise. End up. In order to obtain a stable solution, an approach that improves the accuracy of the posterior distribution based on Bayes' theorem based on prior knowledge that it changes smoothly in the vicinity, for example, a dynamic approach by defining the structure for the mapping g Think.

  Since the mapping g is bijective, thinking about the mapping g is synonymous with considering the curved surface Y itself. As the mechanical structure, for example, a structure like a rigid body is conceivable, but in the case of an image, deformation or the like is conceivable, so that the structure of the rigid body is not appropriate. Therefore, in the present embodiment, an elastic body structure that can flexibly cope with image deformation or the like is introduced into the curved surface Y.

  The energy of the elastic body is expressed by equation (11).

また、（１０）式で示される画像エネルギーを曲面Ｙ上の点を用いて表すと、（１２）式で表される。

Further, when the image energy represented by the equation (10) is expressed using points on the curved surface Y, it is represented by the equation (12).

ポテンシャルエネルギーは、（１２）式の画像エネルギーと（１１）式の弾性エネルギーの線形結合により（１３）式のように定義される。

The potential energy is defined as in equation (13) by linear combination of image energy in equation (12) and elastic energy in equation (11).

このポテンシャルエネルギーを最小にすることは、画像マッチングの条件式である（７）式をできるだけ満たすように、かつ弾性体としての安定形態が取れるような曲面Ｙを探索することを意味する。このことをエネルギー最小化問題として定式化すると（１４）式で表される。

Minimizing this potential energy means searching for a curved surface Y that satisfies Equation (7), which is a conditional expression for image matching, as much as possible and that can take a stable form as an elastic body. When this is formulated as an energy minimization problem, it is expressed by equation (14).

従って、条件式（７）式を満たす写像ｇを求める画像マッチング問題は、エネルギー最小化問題の（１４）式を解法することに帰着する。

Therefore, the image matching problem for obtaining the mapping g that satisfies the conditional expression (7) results in solving the expression (14) of the energy minimization problem.

  Further, the necessary condition for optimality can be obtained from the equation (15) from the equation (14).

従来の手法では、この最適性必要条件の（１５）式を満たすような曲面Ｙ上の点ｙを求めることにより解法している。これらは静的なエネルギーの最小化であり、静的な釣り合いの位置を求める静的な最適化である。図４は、ポテンシャルエネルギーの最小化の点を探索する場合の静的な最適点の説明をするための模式図である。図４に示すように、静的な最適化では初期値に最も近い位置で質点のポテンシャルエネルギーの局所最適点（釣り合い位置）が求まる。従って、この局所最適点が大域的な最適点になるかどうかは初期値に依存することになる。言い換えれば、初期値の選定が好ましくない場合には、大域的に最適な点が得られない場合がある。

In the conventional method, the solution is obtained by obtaining a point y on the curved surface Y that satisfies the equation (15) of the optimality requirement. These are static energy minimizations and static optimizations that determine the position of the static balance. FIG. 4 is a schematic diagram for explaining a static optimum point when searching for a potential energy minimizing point. As shown in FIG. 4, in the static optimization, the local optimum point (balance position) of the potential energy of the mass point is obtained at the position closest to the initial value. Therefore, whether or not this local optimum point becomes a global optimum point depends on the initial value. In other words, when the selection of the initial value is not preferable, a globally optimum point may not be obtained.

  Therefore, in order to find a global optimum point and solve the equation (14), in this embodiment, each lattice point obtained by dividing the target image and the curved surface on the reference image into a lattice shape is considered as a mass point. The kinetic energy at the lattice point (mass point) of the curved surface Y on the reference image is introduced to perform global optimization. In other words, a time axis is introduced to the curved surface Y so that the curved surface Y can be deformed according to time, and a structure of motion is applied to the curved surface. FIG. 5 is a schematic diagram for explaining a global optimum point when searching for a potential energy minimizing point. As shown in FIG. 5, since the initial potential energy is converted into kinetic energy by the introduction of kinetic energy, the global optimality satisfying Eq. (15) in a wider range from the local optimal point to the lattice point (mass point). You can search for points. As a result, a wider range can be searched than in the static optimization, so that the dependency on the initial value is lowered, and there is a possibility that it is more robust against the influence of noise and the like.

Next, introduction of an equation of motion for global optimization of equation (15) will be described. First, the point y on the curved surface Y is expanded as shown in Equation (16) using the time τ introduced into the equation of motion. The total differentiation of the point y on the curved surface Y expressed by the equation (16) is the equation (17), but assuming that the parameter (u, v) ^T is independent of the time τ, the equation (18) is Since this holds, equation (17) is eventually expressed by equation (19). Further, the second derivative of the point y on the curved surface Y from the equation (19) is expressed by the equation (20). The total differentiation of the point y on the curved surface Y shown by the equation (19) and the second order differentiation shown by the equation (20) are expressed by the equation (21), respectively.

（１９）式は、曲面Ｙ上の点ｙの速度を表し、（２０）式は、曲面Ｙ上の点ｙの加速度を表している。従って、曲面Ｙ上の点ｙの運動エネルギーは、（２２）式で定義することができる。

Equation (19) represents the speed of the point y on the curved surface Y, and Equation (20) represents the acceleration of the point y on the curved surface Y. Therefore, the kinetic energy of the point y on the curved surface Y can be defined by equation (22).

このような運動エネルギーを導入したことは、曲面Ｙ上の点ｙを質点と考えて、曲面上の運動を定義したことになり、曲面Ｙ上の点ｙの運動という力学的な構造を規定したことを意味する。かかる曲面Ｙ上の点ｙの運動は運動方程式によって記述される。

The introduction of such kinetic energy means that the point y on the curved surface Y is considered as a mass point and the motion on the curved surface is defined, and the dynamic structure of the movement of the point y on the curved surface Y is defined. Means that. The motion of the point y on the curved surface Y is described by a motion equation.

  Next, the equation of motion of the point y on the curved surface Y is derived as follows according to the Lagrange method. Since the curved surface Y follows the equation (14) for energy minimization, the curved surface Y needs to move in a direction that minimizes the potential energy U. Lagrangian is defined by equation (23).

  Equation (23) means that the object falls so as to minimize the gravitational field, similar to the free fall of the object in the gravitational field that is the potential energy of the position. Since the Lagrangian equation is expressed by the equation (24), the equation of motion of the point y on the curved surface Y is expressed by the equation (25) from the equations (23) and (24).

（２５）式の運動方程式に従った運動は保存系であり、エネルギー保存則により解は収束しない。このため、系を非保存系にするために（２６）式で示される摩擦エネルギーを導入する。

The motion according to the equation of motion of Equation (25) is a conservation system, and the solution does not converge due to the energy conservation law. For this reason, in order to make the system non-conservative, the frictional energy represented by the equation (26) is introduced.

（２６）式で示される摩擦エネルギーを導入した場合のラグランジュ方程式は、（２７）式で表される。

The Lagrangian equation when the friction energy represented by the equation (26) is introduced is represented by the equation (27).

この（２７）式を解法することによって、摩擦エネルギーを導入した運動方程式が（２８）式で得られる。

By solving the equation (27), an equation of motion in which friction energy is introduced is obtained by the equation (28).

次に、動的な探索、（２８）式の運動方程式に従った解がエネルギー最小化の（１４）式に対する最適性必要条件である（１５）式を満足することについて説明する。

Next, it will be described that the dynamic search and the solution according to the equation of motion of the equation (28) satisfy the equation (15) which is the optimum requirement for the energy minimization (14).

  First, consider the case where the search is completed. In this case, since the point y stops completely, both the velocity (total derivative of y) and acceleration (second derivative of y) of the point y are both 0, so that the equation (29) is obtained from the equation of motion (28). Thus, equation (30) is established.

（３０）式が成立することは、最適性必要条件の（１５）式を満足することを意味している。従って、探索が完了した場合、そのときの点ｙは最適点であることがわかる。

The fact that the expression (30) is satisfied means that the expression (15) of the optimality requirement is satisfied. Therefore, when the search is completed, it can be seen that the point y at that time is the optimum point.

  Next, consider a case where the point y on the curved surface Y is stationary. In this case, the velocity at the point y (the total derivative of y) is 0, but the acceleration (the second derivative of y) is not 0, so that equation (31) is obtained from the equation of motion of equation (28).

ここで、点ｙが最適点であると仮定すると、最適性条件の（１５）式により、点ｙの加速度（ｙの２階微分）は０となり、最適点の探索が完了する。一方、点ｙが最適点でない場合には、点ｙの加速度（ｙの２階微分）が０とならないため、最適点の探索は完了していないことになる。すなわち、点ｙが静止した位置が最適点の位置である場合に（３１）式により最適性必要条件の（１５）式が満足することを意味している。

Assuming that the point y is the optimum point, the acceleration of the point y (second derivative of y) becomes 0 according to the equation (15) of the optimum condition, and the search for the optimum point is completed. On the other hand, when the point y is not the optimum point, the acceleration of the point y (second-order derivative of y) does not become 0, and the optimum point search is not completed. That is, when the position where the point y is stationary is the position of the optimum point, it means that the expression (15) of the optimality requirement is satisfied by the expression (31).

  Next, consider the case where frictional energy is acting on the point y. Equation (32) is derived from the nature of the frictional energy, which means that the movement of the point y stops and the search for the optimum point is completed over time.

このように、最適点の探索が完了した場合、点ｙが静止した場合、点ｙに摩擦エネルギーが作用している場合の上記説明により、（２８）式の運動方程式に従った点ｙの運動が、（１５）式の最適性必要条件を満足することがわかる。

As described above, when the search for the optimum point is completed, when the point y is stationary, or when the friction energy is acting on the point y, the motion of the point y according to the equation of motion of the equation (28) However, it is understood that the optimality requirement of the equation (15) is satisfied.

  On the other hand, equation (15) is in the form of first derivative, but the optimality requirement for second derivative is satisfied because the search may be completed at the peak of the potential energy peak shown in FIG. May not be. In this case, the optimality requirement for the second floor can be satisfied by assuming the equation (33) as a condition for the search for the optimum point not to be completed at the apex of the peak of potential energy. Specifically, the assumption of equation (33) can be achieved by realizing the condition of equation (34) with a search algorithm.

  From the above, it can be seen that the search for the optimum point according to the equation of motion of Equation (28) gives the optimum solution for the optimization problem shown by Equation (14). Such a search is more robust than a static search.

Here, in the equation of motion shown in the equation (28), since the partial differential with respect to the point y and the parameters (u, v) is included in the elastic energy E _k , the partial differential equation is converted into the differential equation by the finite difference method. Replace. When the right side of the elastic energy E _k is discretized by the finite difference method, the equation (35) is obtained.

ここで、ｎは離散化されたパラメータ（ｕ，ｖ）の格子点、ｙ_nはｎに対応する離散化されたＹ上の点、ｙ_n+(-1,0)，ｙ_n+(1,0)，ｙ_n+(0,-1)，ｙ_n+(0,1)は点ｙ_nの偏微分に関連する上下左右の格子点を示している。

Here, n is a lattice point of the discretized parameter (u, v), y _n is a point on Y that is discretized corresponding to _n , y _{n + (− 1,0)} , y _{n + (1,0 )} , Y _{n + (0, -1)} , y _{n + (0,1)} indicate the upper, lower, left, and right lattice points related to the partial differentiation of the point y _n .

The equation (35) when partially differentiated by y _n (36) below is obtained and such expression shows elastic energy force is the force generated by the elastic energy.

この弾性エネルギー力を（３７）式で示し、離散パラメータΔｕ，Δｖをそれぞれ１とすると、弾性エネルギー力は（３８）式で表されることになる。

When this elastic energy force is expressed by the equation (37) and the discrete parameters Δu and Δv are respectively 1, the elastic energy force is expressed by the equation (38).

ここで、画像エネルギーＥ_iは、パラメータ（ｕ，ｖ）に関して独立と考え、（２８）式の運動方程式もそのまま離散可能となり、（３９）式で表される。

Here, the image energy E _i is considered to be independent with respect to the parameters (u, v), and the equation of motion of the equation (28) can be discrete as it is, and is expressed by the equation (39).

この（３９）式の右辺第１項は、点ｙ_nにおける画像エネルギーの勾配の逆向きの力である画像エネルギー力であり、（４０）式のようにおく。

The first term on the right side of the equation (39) is an image energy force that is the force opposite to the gradient of the image energy at the point y _n , and is set as in the equation (40).

次に、この画像エネルギー力Ｆ_i（ｎ）、すなわち画像エネルギーの勾配の計算について説明する。本実施の形態では、画像エネルギーの勾配を、局所最適化の手法によって求めている。図６および図７は、画像エネルギー力の概念を示す説明図である。ここで、画像モデルＳ_cは連続画像モデルであり、実際にはサンプリングされた画像モデルＳ_pを利用しているため、局所最適化においてもサンプリングされた画像モデルに基づいておこなっている。

Next, calculation of the image energy force F _i (n), that is, the gradient of the image energy will be described. In this embodiment, the gradient of image energy is obtained by a local optimization technique. 6 and 7 are explanatory diagrams showing the concept of image energy force. Here, the image model S _c is a continuous image model, in practice because it uses the image model S _p sampled, is performed also based on the sampled image model in local optimization.

The sampling point closest to y _n (τ) is set as the local space center y _c , and y _c is obtained by equation (41).

そして、隣接空間Ｌを（４２）式で定義する。このとき、局所探索集合Ωは、（４３）式のように定義することができる。

And the adjacent space L is defined by (42) Formula. At this time, the local search set Ω can be defined as in equation (43).

そして、局所最適化の後、その方向へのベクトルを求め、求めたベクトルを正規化し勾配の大きさを乗算すると、画像エネルギー力として（４４）〜（４６）式が得られる。本実施の形態では、画像エネルギー力として（４４）〜（４６）式を使用している。

Then, after local optimization, a vector in that direction is obtained, and when the obtained vector is normalized and multiplied by the magnitude of the gradient, equations (44) to (46) are obtained as image energy forces. In the present embodiment, the equations (44) to (46) are used as the image energy force.

なお、実装上の扱いやすさ等を考慮してエネルギー関数を（４７）式のように定義して、（４７）式により導かれる画像エネルギー力を用いることもできる。

It is also possible to define an energy function as shown in Equation (47) in consideration of ease of handling in mounting and the like, and use the image energy force derived from Equation (47).

以上説明した画像エネルギーＦ_i（ｎ）、弾性エネルギー力Ｆ_k（ｎ）、摩擦力を使用して、格子点ｙ_n（τ）に対する運動方程式を画像マッチング状態遷移モデルとして生成すると、運動方程式は、（４８）式で表される。また、ｙ_n（０）が参照画像に等しいとし、運動エネルギーは０、すなわちｙ_n（０）の速度が０として考えると、運動方程式の初期条件は、（４９）式となる。

Using the image energy F _i (n), elastic energy force F _k (n), and frictional force described above to generate the equation of motion for the lattice point y _n (τ) as an image matching state transition model, the equation of motion is , (48). Further, _assuming that y _n (0) is equal to the reference image and the kinetic energy is 0, that is, the velocity of y _n (0) is 0, the initial condition of the equation of motion is the equation (49).

本実施の形態では、画像エネルギー力計算部１０３によって、（４４）〜（４６）式で示される画像エネルギー力Ｆ_iを計算し、弾性エネルギー力計算部１０４によって（３８）式で示される弾性エネルギー力Ｆ_kを計算し、摩擦力計算部１０５によって（４８）式の右辺第３項で示される摩擦力を計算する。そして、解法処理部１０１によって（４８）、（４９）式で示される運動方程式を数値解析処理によって解法している。

In the present embodiment, the image energy force calculation unit 103 calculates the image energy force F _i expressed by the equations (44) to (46), and the elastic energy force calculation unit 104 calculates the elastic energy expressed by the equation (38). The force _Fk is calculated, and the frictional force calculation unit 105 calculates the frictional force indicated by the third term on the right side of the equation (48). Then, the solution processing unit 101 solves the equations of motion represented by equations (48) and (49) by numerical analysis processing.

  Next, a method of solving the equation of motion by the solution processing unit 101 will be described. Since the equation of motion represented by the equations (48) and (49), that is, the ordinary differential equation, cannot generally be solved analytically, the time T sufficient for the system of equations of motion to converge is considered, The convergence state of the equation of motion is estimated by calculating the t = (0, T) interval by numerical analysis.

  In the present embodiment, numerical analysis processing by the Euler method is performed using the fact that the ordinary differential equation is uniquely determined by the discrete variable method when the initial value is determined.

  In the present embodiment, numerical analysis processing by the Euler method is performed, but the present invention is not limited to this. In addition to the Euler method, the discrete variable method includes various methods such as the Runge-Kutta method, the Brillesh-Store method, the predictor / corrector method, and the hidden Runge-Kutta method. A technique may be used. In the following, a numerical analysis process using the Euler method will be described as an example.

  Since the Euler method is a numerical solution to the first-order ordinary differential equation, the equations (48) and (49) are converted by applying the variable transformation of the equation (50) to the equations (48) and (49). Convert to first-order ordinary differential equations. Thereby, formulas (51) and (52) are obtained.

一方、常微分方程式（５３）式をオイラー法で数値解析処理を行って解法する場合には、（５４）式のようにｔ^(m)からｔ^{(m +1)}=ｔ^(m)+ｈへ解を進展させる。

On the other hand, when solving the ordinary differential equation (53) by performing numerical analysis processing by the Euler method, t ^(m) to t ^{(m +1)} = t ^(m) + h as shown in equation (54). Develop the solution.

ここで、ｘ^(m)はｍステップであることを示し、ｈはステップ幅である。オイラー法のスキームを（５１）、（５２）式に適用すると（５５）、（５６）式の更新式が得られる。

Here, x ^(m) indicates m steps, and h is a step width. When the Euler scheme is applied to the equations (51) and (52), the updated equations (55) and (56) are obtained.

本実施の形態では、解法処理部１０１によって、（５１）、（５２）式の更新式を繰り返して解法することによって運動方程式（４８）、（４９）式の系の収束状態を求めている。そして、マッピング処理部１０７では、この収束状態から、曲面Ｙをｙ_n＝Ψ_1,n ^(T)のように求め、さらに、（６）式よりｙ_n＝ｇ（ｎ）の関係から、写像ｇを求めている。

In the present embodiment, the solution processing unit 101 obtains the convergence state of the system of equations of motion (48) and (49) by repeatedly solving the updating equations of equations (51) and (52). Then, the mapping processing unit 107 obtains the curved surface Y from this convergence state as y _n = Ψ _{1, n} ^(T) , and further from the relationship of y _n = g (n) from the equation (6). g.

  Next, details of the friction coefficient control unit 102 will be described. When the frictional force control unit 105 calculates the frictional force in the third term on the right side of the equation (48) by the frictional force calculating unit 105, the frictional coefficient μ is variably controlled and calculated.

  The friction coefficient μ is a parameter that determines the search method. For example, when μ → ∞, all kinetic energy is absorbed as friction energy, so the search for the optimum point is not a dynamic search. The static search as shown in FIG. On the other hand, when μ = 0, a dynamic search as shown in FIG. 5 can be performed and a wide range search of the optimum point can be performed, but there is a problem that the solution does not converge due to the energy conservation law.

  For example, the convergence of the potential energy according to the value of the friction coefficient μ was obtained by experiments using the test image of the target image and the reference image as shown in FIG. FIG. 8 is an explanatory diagram illustrating an example of a target image and a reference image in which the entire screen is moved 8 pixels from left to right from the target image. By using the target image and the reference image shown in FIG. 8 and resolving the updating equations (51) and (52) described above with the friction coefficient μ constant, the equations of motion (48), (49 The convergence state of the system of equation (4) was obtained, and an attempt was made to obtain the mapping g from this convergence state.

  FIG. 9A is an explanatory diagram illustrating a mapping relationship between the target image and the reference image when the friction coefficient μ = 5. FIG. 9-2 is an explanatory diagram illustrating a mapping relationship between the target image and the reference image when the friction coefficient μ = 0. In FIGS. 9-1 and 9-2, the gray line indicates the mapping relationship, and the white line indicates the direction.

  In FIG. 8, since the reference image is the entire image moved 8 pixels from the left to the right from the target image, the mapping-related lines appearing in the images of FIGS. 9-1 and 9-2 are linear. Judge that the mapping of the entire screen is accurate.

  As shown in FIG. 9A, it can be seen that when the friction coefficient μ is large, the mapping of the entire screen cannot be obtained correctly. On the other hand, as shown in FIG. 9-2, when the friction coefficient μ is small, it can be seen that the directivity of the entire screen is almost correct, but the mapping relationship does not appear accurately.

  This can be thought of as follows. FIG. 10 is an explanatory diagram showing the relationship between image energy force and time due to the difference in friction coefficient. FIG. 11 is an explanatory diagram showing the relationship between the elastic energy force and the time due to the difference in friction coefficient. FIG. 12 is an explanatory diagram showing the relationship between potential and time due to the difference in friction coefficient.

  When the time change of the potential energy shown in FIG. 12 is seen, it can be seen that when μ = 0, the potential energy does not converge due to the energy conservation law. On the other hand, when μ = 5, it can be seen that the potential energy changes very slowly. Considering the change in the image energy shown in FIG. 10 and the conversion of the elastic energy shown in FIG. I guess that.

  Therefore, the search for the optimum point itself should have a small friction coefficient. However, since it does not converge as it is, the friction coefficient control unit 102 of the present embodiment variably controls the friction coefficient, and the time in the equation of motion is early. Reduces the friction coefficient and increases the friction coefficient at the time of convergence in the second half. Thereby, low energy can be calculated | required robustly.

  Specifically, the friction coefficient control unit 102 increases the friction coefficient μ (τ) as the time τ elapses by a two-stage linear function of the time τ in the equation of motion, as shown in the equation (57). It is calculated as follows.

図１３は、摩擦係数μの時間τによる変化の状態を示すグラフである。図１４は、（５７）式により摩擦係数μを計算して、画像マッチング処理を行った場合におけるポテンシャルエネルギーの時間的変化の状態を示す説明図である。μ制御の線が、摩擦係数を（５７）式により計算した場合である。なお、図１４において、μ₁＝０、μ_max＝０．５、Ｔ＝１００としている。図１４に示すように、ポテンシャルエネルギーが低い状態で収束していることがわかる。

FIG. 13 is a graph showing a change state of the friction coefficient μ with time τ. FIG. 14 is an explanatory diagram showing a temporal change state of potential energy when the friction coefficient μ is calculated by the equation (57) and image matching processing is performed. The μ control line is the case where the friction coefficient is calculated by the equation (57). In FIG. 14, μ ₁ = 0, μ _max = 0.5, and T = 100. As shown in FIG. 14, it is understood that the potential energy is converged in a low state.

  Next, image matching processing by the image matching apparatus 100 according to the present embodiment configured as described above will be described. FIG. 15 is a flowchart showing a procedure of image matching processing by the image matching apparatus 100. Specifically, the image matching processing is realized by solving the equations (51) and (52) shown above by numerical analysis.

First, the solution processing unit 101 sets time τ ⁽⁰⁾ = 0 (step S1501), and sets initial values Ψ _{1, n} (0) = n and Ψ _{2, n} (0) = 0 (step S1502). ). Thereby, the equation (52) is executed.

Next, the image energy force calculation unit 103 calculates the image correlation potential force F _i ^(m) (n) in m steps for all n (step S1503). The calculation processing of the image energy force F _i ^(m) (n) will be described later.

Next, the elastic energy force calculation unit 104 calculates the elastic energy force F _k ^(m) (n) in m steps for all n by the equation (38) (step S1504). Then, the frictional force calculation unit 105 calculates the frictional force [−μ (τ ^(m) ) ψ2 _{, n} ^(m) ] in m steps for all n (step S1505).

Next, the solution processing unit 101 uses the image energy force F _i ^(m) (n), the elastic energy force F _k ^(m) (n), the friction obtained from the updating formulas (55) in steps S1503 to S1505. Update with the force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] (step S1506).

Next, the solution processing unit 101 stores the value of Ψ _{1, n} ^{(m) in} the frame memory 106 (step S1507). Then, the solution processing unit 101 updates τ ^{(m + 1)} = τ ^(m) + h (step S1508), and increases m by 1 (step S1509). Then, it is determined whether or not τ ^{(m + 1)} has exceeded a predetermined time T (step S1510). If not, step S1503 to S1509 are repeatedly executed. On the other hand, τ ^{(m + 1)} is the case beyond the T is the solution processing unit 101, for all n, set to ^{_{y n Ψ 1, n (T}} ) ( step S1511). Then, the mapping processing unit 107 obtains a correspondence relationship between the target image and the reference image, that is, a mapping (step S1512).

  Next, the image energy force calculation process in step S1503 will be described. FIG. 16 is a flowchart showing a procedure of image energy force calculation processing by the image energy force calculation unit 103.

First, the image energy force calculation unit 103 calculates the sampling point closest to y _n (τ) as the local space center y _{c using} equation (41) (step S1601). Next, the image energy force calculation unit 103 sets the adjacent space L defined by the equation (42) (step S1602), and calculates the local search set Ω by the equation (43) (step S1603).

Next, the image energy force calculation unit 103 calculates y _min as a local optimization calculation using the equation (46) (step S1604), and uses the local optimization y _min obtained in step S1604 for normalization. Then, d = y _min −Ψ _{1, n} ^(m) is calculated by the equation (45) (step S1605). Then, the image energy force calculation unit 103 calculates the image energy force F _i ^(m) (n) using equation (44) using y _min , d, and the like (step S1606).

Next, the calculation process of the frictional force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] in step S1505 will be described. FIG. 17 is a flowchart showing the procedure of the frictional force calculation process. First, whether to determine in accordance with the friction coefficient at the current time τ ^(m) μ a ^{(τ (m)) (57} ) wherein the friction coefficient control unit 102, the current time tau ^(m) is 0 or more T / 2 or less Is determined (step S1701).

When the current time τ ^(m) is 0 or more and T / 2 or less (step S1701: Yes), the friction coefficient control unit 102 calculates the friction coefficient μ (τ ^(m) ) according to the equation (57). It is calculated by (2 μ ₁ / T) τ ^(m) (step S1702).

On the other hand, when the current time τ ^(m) has passed T / 2 (step S1701: No), the friction coefficient control unit 102 calculates the friction coefficient μ (τ ^(m) ) according to the equation (57). [2 (μ _max −μ ₁ ) / T] τ ^(m) + 2μ ₁ −μ _{max is} calculated (step S1703). Then, the frictional force calculation unit 105 calculates the frictional force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] using the friction coefficient calculated in step S1702 or S1703 (step S1704).

From the image energy force F _i ^(m) (n), elastic energy force F _k ^(m) (n), and friction force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] obtained in this way. As described above, the updating formula (55) is updated in step S1506.

  FIG. 18 is an explanatory diagram showing the mapping relationship between the target image and the reference image shown in FIG. 8 when the friction coefficient is variably controlled by equation (57). As shown in FIG. 18, the mapping is accurately obtained over the entire screen. In this case, it is possible to converge with a low potential energy U as in FIG.

  As described above, in the image matching apparatus 100 according to the first embodiment, the friction coefficient control unit 102 variably controls the friction coefficient when calculating the friction force so as to increase according to the linear function according to the time in the equation of motion. Therefore, it is possible to perform image matching with higher accuracy by satisfying both search for a global optimum point and convergence.

  In the first embodiment, the friction coefficient is variably controlled so as to increase according to the linear function according to the time in the equation of motion. However, the present invention is not limited to this. It may be variably controlled so as to increase step by step. For example, the coefficient of friction μ can be controlled to increase stepwise according to the equation (58). FIG. 19 is an explanatory diagram showing temporal changes in the friction coefficient μ when the friction coefficient is increased stepwise according to the equation (58) according to the time in the equation of motion.

また、摩擦係数制御部１０２によって、摩擦係数を、運動方程式における時間に応じて指数関数に従って増加するように可変制御してもよい。例えば、摩擦係数μを（５９）式で示す指数関数に従って増加するように制御することができる。図２０は、摩擦係数を、運動方程式における時間に応じて（５９）式の指数関数に従って増加する場合における摩擦係数μの時間的変化を示す説明図である。

Further, the friction coefficient control unit 102 may variably control the friction coefficient so as to increase according to an exponential function according to the time in the equation of motion. For example, the friction coefficient μ can be controlled to increase according to an exponential function expressed by the equation (59). FIG. 20 is an explanatory diagram showing a temporal change of the friction coefficient μ when the friction coefficient is increased according to the exponential function of the equation (59) according to the time in the equation of motion.

(Embodiment 2)
In the image matching apparatus 100 according to the second embodiment, a negative value can be set as the initial value of the friction coefficient μ, and the friction coefficient is variably controlled so as to increase from the initial value according to the time of the equation of motion. .

  The configuration of the image matching apparatus 100 according to the second embodiment is the same as that of the first embodiment. The entire image matching process, the image energy force calculation process, and the elastic energy force calculation process are the same as those in the first embodiment. In the second embodiment, the frictional force calculation process is different from that in the first embodiment.

  Normally, the friction coefficient μ takes a positive value, but the friction coefficient control unit 102 according to the second embodiment takes a negative value as the initial value of the friction coefficient μ and increases from the initial value according to the time of the equation of motion. So that it is variably controlled. As a result, an even wider range of optimum points can be searched. However, in this case, if the coefficient of friction is fixed to a negative value, acceleration and divergence occur, so the condition of variable control becomes important. Specifically, the friction coefficient μ is increased from a negative value according to the time τ according to the equation (60) according to a linear function. When the friction coefficient μ is negative, it can be regarded as acceleration energy.

図２１は、実施の形態２における摩擦係数μの時間τによる変化の状態を示すグラフである。ここで、μ₀は負値となっており、このため、時間τ＝０における摩擦係数μは負値の初期値をとることになる。

FIG. 21 is a graph showing a change state of the friction coefficient μ according to the time τ in the second embodiment. Here, μ ₀ is a negative value. Therefore, the friction coefficient μ at time τ = 0 takes a negative initial value.

FIG. 22 is a flowchart showing the procedure of the frictional force calculation process in the second embodiment. First, whether to determine in accordance with the friction coefficient at the current time τ ^(m) μ a ^{(τ (m)) (57} ) wherein the friction coefficient control unit 102, the current time tau ^(m) is 0 or more T / 2 or less Is determined (step S2201).

If the current time τ ^(m) is 0 or more and T / 2 or less (step S2201: Yes), the friction coefficient control unit 102 calculates the friction coefficient μ (τ ^(m) ) according to the equation (60). Calculation is performed by (2 μ ₁ / T) τ ^(m) + μ ₀ (step S2202).

On the other hand, when the current time τ ^(m) has passed T / 2 (step S2201: No), the friction coefficient control unit 102 calculates the friction coefficient μ (τ ^(m) ) according to the equation (60). [2 (μ _max −μ ₁ ) / T] τ ^(m) + 2μ ₁ −μ _{max is} calculated (step S2203). Then, the frictional force calculation unit 105 calculates the frictional force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] using the friction coefficient calculated in step S2202 or S2203 (step S2204).

  FIG. 23 is an explanatory diagram illustrating an example of a target image and a reference image in which the entire screen is moved from the target image by 12 pixels from left to right. 23. Using the target image and the reference image shown in FIG. 23, when the initial value of the negative value is not used for the friction coefficient μ and when the negative value is used, the updating formulas (51) and (52) are repeated to solve Thus, the convergence state of the system of equations of motion (48) and (49) was obtained, and an attempt was made to obtain the mapping g from this convergence state.

  FIG. 24-1 is an explanatory diagram illustrating a mapping relationship between a target image and a reference image when control is performed to increase according to time using a non-negative value (value of 0 or more) as an initial value of the friction coefficient. is there. FIG. 24-2 is an explanatory diagram illustrating the mapping relationship between the target image and the reference image when control is performed to increase according to time using a negative value as the initial value of the friction coefficient according to the equation (60). In FIGS. 24-1 and 24-2, similar to FIGS. 9-1 and 9-2, the gray line indicates the mapping relationship, and the white line indicates the direction.

  As shown in FIG. 24A, it is understood that when the initial value of the friction coefficient is controlled to increase according to time using a non-negative value, the mapping of the half screen cannot be obtained accurately. . On the other hand, as shown in FIG. 24A, it is understood that when the control is performed by using a negative value as the initial value of the friction coefficient and increasing according to time, the mapping of the entire screen is accurately obtained. .

  FIG. 25 is an explanatory diagram showing the temporal change state of potential energy when the friction coefficient μ is calculated by the equation (57) and image matching processing is performed.

  FIG. 25 shows a case where the initial value of the friction coefficient is increased according to time using a non-negative value and a case where the initial value of the friction coefficient is increased according to time using a negative value. It is explanatory drawing which shows the state of the time change of the potential energy in a case. As shown in FIG. 25, it can be seen that when the friction coefficient is variably controlled using a negative value as the initial value, the global optimum point is reached and the energy is low.

  As described above, in the image matching apparatus according to the second embodiment, a negative value can be set as the initial value of the friction coefficient μ, and the friction coefficient is variably controlled so as to increase from the initial value according to the time of the equation of motion. Therefore, it is possible to perform image matching with higher accuracy while achieving both a search for a global optimum point and convergence.

  In the second embodiment, a negative value is used as the initial value of the friction coefficient and is increased according to a linear function. However, the present invention is not limited to this, and the initial value of the negative value is gradually increased according to time. It is also possible to variably control so as to increase. For example, the friction coefficient μ can be controlled to increase stepwise from the negative initial value according to the equation (61). FIG. 26 is an explanatory diagram showing the temporal change of the friction coefficient μ when the friction coefficient increases stepwise from the initial negative value according to the equation (61) according to the time in the equation of motion.

(Embodiment 3)
In the image matching apparatus according to the first and second embodiments, the friction coefficient is variably controlled so as to increase according to the time in the equation of motion. However, in the image matching apparatus according to the third embodiment, an equilibrium state of potential energy is obtained. The friction coefficient is variably controlled according to the above.

  FIG. 27 is a block diagram of a configuration of the image matching apparatus according to the third embodiment. As shown in FIG. 27, the image matching apparatus 2700 according to the present embodiment includes a solution processing unit 101, an image energy force calculation unit 103, an elastic energy force calculation unit 104, a friction force calculation unit 105, and a friction coefficient. It mainly includes a control unit 2702, a mapping processing unit 107, an equilibrium state determination unit 2703, and a frame memory 106.

  Here, the solution processing unit 101, the image energy force calculation unit 103, the elastic energy force calculation unit 104, the friction force calculation unit 105, and the mapping processing unit 107 are the same as those in the first embodiment.

  The equilibrium state determination unit 2703 is a processing unit that determines whether or not the potential energy is in an equilibrium state. From FIG. 12, it can be seen that when the friction coefficient μ is 0, the oscillation of the potential energy does not stop due to the energy conservation law, but it vibrates around a certain state. This can be regarded as a motion in which the potential energy is oscillating around the equilibrium point. When the potential energy has shifted to such a state, the equilibrium state has already been reached, and it can be determined that the global search for the optimum point has been completed. For this reason, in the present embodiment, the equilibrium state determination unit 2703 determines whether or not the potential energy is in an equilibrium state. If the potential energy is in the equilibrium state, the friction coefficient is set larger than before the transition to the equilibrium state. It is set.

  Whether or not the equilibrium state has been reached can be determined, for example, from the distribution state of potential energy. FIG. 28 is an explanatory diagram showing a potential energy distribution state when the potential energy has reached an equilibrium state. FIG. 29 is an explanatory diagram showing the distribution state of potential energy when the potential energy has not reached the equilibrium state.

  When the equilibrium state is not reached and the state is dispersed, a distribution as shown in FIG. 29 is shown. On the other hand, when the equilibrium state is reached, the state is grouped to some extent, and thus a distribution as shown in FIG. 28 is shown. These differences are whether the distribution of the distribution is large or small. Therefore, the equilibrium state determination unit 2703 determines that the equilibrium state has been reached when there is no significant change in the distribution of potential energy distribution.

  Further, when the equilibrium state is reached, a certain point y does not move to a new moving area, and is moving in the same area. For this reason, the equilibrium state determination unit 2703 may obtain the rate of change of the moving region of the point y, and may determine that the state is in an equilibrium state when the rate of change becomes small. Specifically, when the variance of the distribution range of the coordinates of the point y is within a certain range, it may be determined that the equilibrium state has been reached, assuming that the rate of change of the moving region of the point y has decreased.

  FIG. 30 is a flowchart illustrating a procedure of frictional force calculation processing according to the third embodiment. First, the equilibrium state determination unit 2703 obtains the sample variance of the potential energy U (n) for all n (step S3001), and obtains the population variance from the obtained sample variance (step S3002). Here, the sample variance of the potential energy U (n) is obtained by calculating the sample average by the equation (62) and calculating the sample variance using the sample average by the equation (63).

次に、平衡状態判定部２７０３は、ｍステップについて推定値がｍ−１ステップについての推定値より所定値以上変化しているか否かを調べる（ステップＳ３００３）。そして、所定値以上変化している場合には、まだ、平衡状態に達していないと判断して、摩擦係数μ（τ^(m)）にμ₀を設定する（ステップＳ３００４）。一方、ステップｍについて推定値がステップｍ−１についての推定値より所定値以上変化していない場合には、平衡状態に達したと判断して、摩擦係数μ（τ^(m)）にμ₁を設定する（ステップＳ３００５）。ここで、μ₀＜μ₁である。このため、平衡状態に達した場合には摩擦係数が大きく設定されることになる。

Next, the equilibrium state determination unit 2703 checks whether or not the estimated value for m steps has changed by a predetermined value or more from the estimated value for m−1 steps (step S3003). If it has changed more than a predetermined value, it is determined that the equilibrium state has not yet been reached, and μ ₀ is set to the friction coefficient μ (τ ^(m) ) (step S3004). On the other hand, if the estimated value for step m does not change by more than a predetermined value from the estimated value for step m−1, it is determined that an equilibrium state has been reached and the friction coefficient μ (τ ^(m) ) is set to μ ₁ Is set (step S3005). Here, μ ₀ <μ ₁ . For this reason, when the equilibrium state is reached, the friction coefficient is set large.

Then, the frictional force calculation unit 105 calculates the frictional force [−μ (τ ^(m) ) Ψ _{2, n} ^(m) ] using the friction coefficient set in step S3004 or S3005 (step S3006).

  As described above, in the image matching apparatus according to the third embodiment, since the friction coefficient is variably controlled according to the balance state of the potential energy, it is possible to realize an image matching process adapted to the image.

  The image matching apparatus according to the first to third embodiments includes a control device such as a CPU, a storage device such as a ROM (Read Only Memory) or a RAM, an external storage device such as an HDD or a CD drive device, and a display such as a display device. The apparatus includes an input device such as a keyboard and a mouse, and has a hardware configuration using a normal computer.

  The image matching program executed by the image matching apparatus according to the first to third embodiments is a file in an installable format or an executable format, and is a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). And the like recorded on a computer-readable recording medium.

  Further, the image matching program executed by the image matching apparatus according to the first to third embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. good. Further, the image matching program executed by the image matching apparatus of the present embodiment may be provided or distributed via a network such as the Internet.

  Further, the image matching program of the present embodiment may be configured to be provided by being incorporated in advance in a ROM or the like.

  The image matching program executed by the image matching apparatus according to the first to third embodiments includes the above-described units (solution processing unit, image energy force calculation unit, elastic energy force calculation unit, friction force calculation unit, friction coefficient control unit, balance). The state determination unit) is a module configuration, and as actual hardware, the CPU (processor) reads and executes the image matching program from the storage medium, so that the respective units are loaded on the main storage device, Each unit (solution processing unit, image energy force calculation unit, elastic energy force calculation unit, friction force calculation unit, friction coefficient control unit, equilibrium state determination unit) is generated on the main storage device.

  It should be noted that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram illustrating a configuration of an image matching apparatus according to a first embodiment. It is explanatory drawing which shows the mapping from the curved surface on a target image to the curved surface on a reference image. It is explanatory drawing which shows the mapping from the point x of the curved surface X on a target image to the point y of the curved surface Y on a reference image. It is a schematic diagram for demonstrating the static optimal point in the case of searching the point of potential energy minimization. It is a schematic diagram for demonstrating the global optimal point in the case of searching for the point of potential energy minimization. It is explanatory drawing which shows the concept of image energy force. It is explanatory drawing which shows the concept of image energy force. It is explanatory drawing which shows the example of the reference image which the example of the target image and the whole screen moved 8 pixels from this target image from the left to the right. It is explanatory drawing which shows the mapping relationship of the object image and reference image in friction coefficient (micro | micron | mu) = 5. It is explanatory drawing which shows the mapping relationship of the target image in case of friction coefficient (micro | micron | mu) = 0, and a reference image. It is explanatory drawing which shows the relationship between the image energy force by the difference in a friction coefficient, and time. It is explanatory drawing which shows the relationship between the elastic energy force by the difference in a friction coefficient, and time. It is explanatory drawing which shows the relationship between the potential by the difference in a friction coefficient, and time. It is a graph which shows the state of change with time τ of friction coefficient μ. It is explanatory drawing which shows the state of the time change of the potential energy at the time of calculating a friction coefficient (micro | micron | mu) by Formula (57) and performing an image matching process. 4 is a flowchart illustrating a procedure of image matching processing by the image matching apparatus 100. 5 is a flowchart illustrating a procedure of image energy force calculation processing by an image energy force calculation unit 103. It is a flowchart which shows the procedure of the calculation process of frictional force. It is explanatory drawing which shows the mapping relationship of the object image shown in FIG. 8, and a reference image in the case of controlling a friction coefficient variably by (57) Formula. It is explanatory drawing which shows the time change of the friction coefficient (micro | micron | mu) in case a friction coefficient is increased in steps according to (58) Formula according to the time in a motion equation. It is explanatory drawing which shows the time change of the friction coefficient (micro | micron | mu) when increasing a friction coefficient according to the exponential function of (59) Formula according to the time in a motion equation. 6 is a graph showing a state of change of friction coefficient μ with time τ in the second embodiment. 10 is a flowchart showing a procedure of frictional force calculation processing in the second embodiment. It is explanatory drawing which shows the example of the reference image which the example of the target image and the whole screen moved 12 pixels from this target image from the left to the right. It is explanatory drawing which shows the mapping relationship of the target image and the reference image at the time of performing the control which increases according to time using the non-negative value for the initial value of a friction coefficient. It is explanatory drawing which shows the mapping relationship of the object image and a reference image in the case of performing control which increases according to time according to (60) Formula using a negative value for the initial value of a friction coefficient. It is explanatory drawing which shows the state of the time change of the potential energy at the time of calculating a friction coefficient (micro | micron | mu) by (57) Formula and performing an image matching process. It is explanatory drawing which shows the time change of the friction coefficient (micro | micron | mu) in case a friction coefficient increases in steps from the initial value of a negative value according to (61) Formula according to the time in a motion equation. FIG. 6 is a block diagram illustrating a configuration of an image matching apparatus according to a third embodiment. It is explanatory drawing which shows the distribution state of potential energy in case the potential energy has reached the equilibrium state. It is explanatory drawing which shows the distribution state of potential energy in case the potential energy has not reached the equilibrium state. 10 is a flowchart showing a procedure of frictional force calculation processing in the third embodiment.

Explanation of symbols

100, 2700 Image matching device 101 Solution processing unit 102, 2702 Friction coefficient control unit 103 Image energy force calculation unit 104 Elastic energy force calculation unit 105 Friction force calculation unit 106 Frame memory 107 Mapping processing unit 2703 Equilibrium state determination unit

Claims (13)

An image matching device for obtaining a correspondence between a target image and a reference image,
There is an image correlation between each of the plurality of first grid points on the target image and each of the plurality of second grid points corresponding to the first grid points on the reference image. Based on the potential energy gradient based on the position of each second grid point and the position of each first grid point corresponding to the second grid point. An image energy force calculator for calculating the image energy force received by the grid points;
An elastic energy force calculation unit that calculates elastic energy force received from elastic energy between each second lattice point and another second lattice point adjacent to the second lattice point;
A friction coefficient control unit that variably controls the friction coefficient of the friction force acting on each second grid point;
A frictional force calculating unit that calculates the frictional force acting on each of the second lattice points using the friction coefficient;
A solution processing unit that performs a solution process of the equation of motion related to each of the second lattice points by the image energy force, the elastic energy force, and the friction force;
A mapping processing unit for obtaining a correspondence relationship between the target image and the reference image from a result of the solution processing by the solution processing unit;
An image matching apparatus comprising:   The image matching apparatus according to claim 1, wherein the friction coefficient control unit calculates the friction coefficient so as to increase based on the passage of time in the equation of motion.   The image matching apparatus according to claim 2, wherein the friction coefficient control unit calculates the friction coefficient so as to increase according to a plurality of linear functions with respect to time in the equation of motion.   The image matching apparatus according to claim 2, wherein the friction coefficient control unit calculates the friction coefficient so as to increase stepwise as the time elapses.   The image matching apparatus according to claim 2, wherein the friction coefficient control unit calculates the friction coefficient so as to increase according to an exponential function with respect to the time.   3. The friction coefficient control unit according to claim 2, wherein the friction coefficient control unit calculates the friction coefficient so that the friction coefficient becomes a negative value at the start of time in the equation of motion and increases as the time elapses. Image matching device.   The friction coefficient control unit calculates the friction coefficient so that the friction coefficient is negative at the start of time in the equation of motion and increases according to a linear function with respect to the time. The image matching apparatus described.   7. The friction coefficient control unit calculates the friction coefficient so that the friction coefficient is a negative value at the start of time in the equation of motion and increases stepwise as the time elapses. The image matching apparatus described in 1. An equilibrium state determination unit for determining whether or not the potential energy has reached an equilibrium state;
The image matching apparatus according to claim 1, wherein the friction coefficient control unit calculates the friction coefficient based on a determination result by the equilibrium state determination unit.   When the potential energy has reached an equilibrium state, the friction coefficient control unit calculates the friction coefficient to be a value larger than the value of the friction coefficient when the potential energy has not reached the equilibrium state. The image matching apparatus according to claim 9, wherein the apparatus is an image matching apparatus.   The equilibrium state determination unit determines that the potential energy has reached an equilibrium state when the distribution of the distribution range of the coordinates of the second lattice point is within a predetermined range. The image matching apparatus according to claim 10. An image matching method for obtaining a correspondence between a target image and a reference image,
There is an image correlation between each of the plurality of first grid points on the target image and each of the plurality of second grid points corresponding to the first grid points on the reference image. Based on the potential energy gradient based on the position of each second grid point and the position of each first grid point corresponding to the second grid point. An image energy force calculation step for calculating the image energy force received by the grid points;
An elastic energy force calculation step for calculating an elastic energy force received from elastic energy between each second lattice point and another second lattice point adjacent to the second lattice point;
A friction coefficient control step for variably controlling the friction coefficient for the friction force acting on each second lattice point;
A frictional force calculating step of calculating a frictional force acting on each of the second lattice points using the friction coefficient;
A solution processing step for performing a solution processing of a motion equation relating to each of the second lattice points by the image energy force, the elastic energy force, and the friction force;
A mapping processing step for obtaining a correspondence relationship between the target image and the reference image from a result of the solution processing by the solution processing step;
An image matching method comprising: An image matching method for obtaining a correspondence between a target image and a reference image,
There is an image correlation between each of the plurality of first grid points on the target image and each of the plurality of second grid points corresponding to the first grid points on the reference image. Based on the potential energy gradient based on the position of each second grid point and the position of each first grid point corresponding to the second grid point. An image energy force calculation step for calculating the image energy force received by the grid points;
An elastic energy force calculation step for calculating an elastic energy force received from elastic energy between each second lattice point and another second lattice point adjacent to the second lattice point;
A friction coefficient control step for variably controlling the friction coefficient for the friction force acting on each second lattice point;
A frictional force calculating step of calculating a frictional force acting on each of the second lattice points using the friction coefficient;
A solution processing step for performing a solution processing of a motion equation relating to each of the second lattice points by the image energy force, the elastic energy force, and the friction force;
A mapping processing step for obtaining a correspondence relationship between the target image and the reference image from a result of the solution processing by the solution processing step;
An image matching program that causes a computer to execute.

Images