JP3077237B2 - Signal matching method and matching device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal matching method suitable for pattern matching used in, for example, image processing, image recognition, voice recognition, and the like, and a signal matching apparatus used for implementing the method. About.

[0002]

2. Description of the Related Art The object of image recognition is shifting from a still image to a moving image such as a hand gesture or a gesture. In order to recognize such a moving image, it is necessary to detect a region that has moved based on a time difference. However, to detect a region that has moved more accurately, it is conceivable to use an optical flow.
The optical flow is a trajectory following a pixel having invariant luminance in the time direction, and corresponds to a motion vector of an object in a two-dimensional image.

That is, the luminance in an image of one frame is represented by I
(X, y, t) When a time difference between two frames is defined as Δt, and a locus (x, y) satisfying ΔI (x, y, t) / Δt = 0 is obtained,
The vector (u, v) indicating the optical flow is defined as follows. u = Δx / Δt, v = Δy / Δt Using the optical flow, a motion vector <d> can be defined as follows. <D> = (uΔt, vΔt) It is known that the calculation of the optical flow (u, v) is equivalent to the calculation of the vector (u, v) that minimizes the following equation. The second term of the following equation is u,
is a constraint condition for smoothing v, and λ is a constant indicating the strength of the constraint condition.

(Equation 1)

[0004] Such a method of calculating an optical flow can be applied to, for example, matching on a time axis shift between two audio signals and matching of two image data. In order to generalize the condition of Equation 1, 2 defined on an n-dimensional Euclidean space <R> ⁿ
Consider two functions f1 (<x>) and f2 (<x>).
Assuming that n elements of a vector <x> on a space <R> ⁿ which are variables of these functions are (x1, x2, ‥‥, xn), a motion vector on the space <R> ⁿ is <d> Then, the motion vector <d> can be expressed as a vector <d (<x>)> as a function of the vector <x>. In this case, the condition corresponding to Equation 1 is equivalent to the condition that minimizes the function A (<d>) defined by the sum of the squares of the norms in the following equation.

[0005]

(Equation 2) The function P (<d>) of the second term of Expression 2 is an incidental condition corresponding to the second term of Expression 1, and the behavior of the motion vector <d> is restricted by the incidental condition. Λ is a constant indicating the strength of the incidental condition. Although the original derivation target is the motion vector <d> that minimizes the first term of Equation 2, the motion vector <d> may be obtained as it is. The additional conditions are added for the purpose of obtaining the information. Then, by taking the square of the norm of Equation 2 as the square integral of norm or the sum of the squares of the norm, the minimum value problem of Equation 2 becomes equivalent to the least square method. Is the integral expression of Equation 1.

Although an additional condition is added to the equation (2), an object to be obtained under the condition for minimizing the equation (2) is a motion vector <d that minimizes the first term of the equation (2). >, The matching error ε (<x>, <d>) in Equation 2 is defined as follows.

(Equation 3) The following approximate expression is used to derive Equation 3.

(Equation 4)

The first term of the equation (2) becomes a form obtained by square integration of the equation (3), and becomes a quadratic equation with respect to the motion vector <d>.
When the second term of the equation 2 is also represented by a quadratic equation relating to the motion vector <d>, the minimum solution becomes equal to the minimum solution, and the minimum solution is converted into a linear equation by the least square method and the variation method. It was possible to solve. For example, by using the approximation of Equation 3 in Equation 1 for calculating the optical flow, Equation 1 can be modified as follows.

(Equation 5)

Since equation (5) is a quadratic equation with respect to u and v, it is converted to an Euler equation to obtain a minimum solution, a linear differential equation is obtained, and a motion vector <d as a solution (u, v) is obtained. > Can be uniquely obtained. In other words, the approximation of Expression 3 expresses a matching error in a minute area by approximating the function f2 (<x>) by a tangent plane in <x>.

[0009]

However, when the absolute value of the motion vector <d> is large, the first-order approximation (tangent plane approximation) of the Taylor expansion of the function f2 (<x>) expressed by Equation 4 is sufficient. f2 (<x>) could not be approximated, and the estimation of the motion vector <d> did not go well.
To improve this, a method of improving the approximation accuracy by increasing the order of Taylor expansion has been considered.
Ε (<x>, <d>) has a quadratic or higher-order component with respect to <d>. In doing so, it is necessary to solve a nonlinear equation, and there is no assurance that the minimum solution becomes the minimum solution, so that <d> cannot be easily obtained. In view of the above, the present invention is used for a signal matching method capable of easily obtaining a vector <d> that minimizes the above-described Expression 2 even when the vector is long, and is used for implementing this method. It is an object to provide a matching device.

[0010]

According to the signal matching method of the present invention, for example, as shown in FIG. 1, a first function f1 of a certain vector ((x1, x2,..., Xn) = <x>) is used. In a signal matching method for obtaining a motion vector <d> for matching a first signal that can be expressed by the following equation and a second signal that can be expressed by a second function f2 of the vector, the first function and the second function , The partial differential functions of various orders with respect to each element xi of the certain vector of the certain function are obtained, and the first order of the partial differential function of the first function and the partial differential function of the second function with respect to the test motion vector are obtained. The matching error consisting of the square of the absolute value of the difference from the Taylor expansion of the various orders is obtained for these various orders, and the weighted sum of the matching errors of the various orders is added to the test motion vector <d>. Evaluation function B by adding the condition (<d>) asking, control unit the evaluation function (6)
And the control unit (6) reduces the value of the evaluation function to a minimum.
Determine the value of the above experimental motion vector so that
The test motion vector when the value of the evaluation function is minimized
Match the first signal with the second signal
The motion vector is output as a motion vector <d> .

A signal matching apparatus according to the present invention, as shown in FIG. 1, for example, has a certain vector ((x1,
(x2, ‥‥, xn) = <x>) a motion vector <1 for matching a first signal that can be expressed by a first function f1 and a second signal that can be expressed by a second function f2 of the vector. In a signal matching device for determining d>
A partial differential filter circuit (3-0 to 3-K) for obtaining a partial differential function of various orders with respect to each element xi of the certain vector of the first function and the second function; An error detection circuit for obtaining a matching error consisting of a square of an absolute value of a difference between a partial differential function and a first-order Taylor expansion of a partial differential function of the second function with respect to a test motion vector <d> for various orders thereof (4-0 to 4-K)
And an evaluation function operation circuit (5, 7-0 to 7 ) for obtaining an evaluation function B (<d>) by adding an additional conditional expression relating to the test motion vector to the weighted sum of the matching errors of various orders. −K, 8, 9) and the test motion vector <d> so that the value of the evaluation function is minimized.
And a control circuit (6) for determining the value of the evaluation signal. The motion vector for matching the test motion vector when the value of the evaluation function is minimized with the first signal and the second signal. It is to be regarded as.

[0012]

According to the signal matching method of the present invention, the partial differential function of the first function and the partial Taylor expansion of the first-order Taylor expansion with respect to the test motion vector <d> of the partial differential function of the second function are obtained. Matching errors consisting of the square of the absolute value of the difference are obtained for various orders of each element xi, and an evaluation function B
(<D>) is Ru is formed. This evaluation function is provided by the control unit (6)
Control unit (6), these weighted sum and incidental
The motion vector <d> is selected so that the conditional expression is minimized.
Output. Therefore, the first function and the second function
Compared with the case where the square of the absolute value of the difference from the first-order Taylor expansion regarding the test motion vector <d> of the function Can be estimated.

In this case, the evaluation function B (<d>) is
Since it is quadratic with respect to the vector <d>,
As in the conventional example, it can be solved by converting it into a linear equation by the least squares method and the variation method. Further, according to the signal matching apparatus of the present invention, the signal matching method can be directly implemented.

[0014]

An embodiment of the present invention will be described below with reference to the drawings. In this example, the absolute value of the function f2 (<x> + <d>)-f1 (<x>) is obtained by finding the motion vector <d> that minimizes the function A (<d>) of the above-described equation 2. The present invention is applied to a case where a motion vector <d> that minimizes is obtained. Also in this example, the matching error ε
(<X>, <d>) can be expressed by Equation 3, but in this example, since the matching error is expressed linearly with respect to the motion vector <d> and substantially up to higher-order terms of Taylor expansion, Equation 3 is extended to an error for partial differentiation of each element of the vector <x> of the function f2 (<x>).

For this reason, the n-dimensional Euclidean space <R>
^{Let n} elements of a vector <x> on n be x1, x2, ‥
要素, xn, the elements x1, x2 of the function f (<x>)
The m, m2, ‥‥, mn-order partial derivatives f ^{(m1, m2,..., Mn)} (<x>) regarding ‥‥ and xn are expressed as follows.

(Equation 6)

Then, the matching error ε of the equation (3)
(<X>, <d>) is expanded to a matching error _{εm1, m2,..., Mn} (<x>, <d>) defined by the following equation. This matching error is obtained by subtracting the partial differential function of the function f1 (<x>) from the first-order Taylor expansion of the partial differential function of the function f2 (<x>) with respect to the motion vector <d>.

(Equation 7)

Using the matching error of Equation 7, the problem of Equation 2 is solved as a problem of minimizing the following function B (<d>) with respect to the vector <d>. C _{m1, m2, ...
, mn} are constants.

(Equation 8)

For example, when this is applied to the formula for calculating the optical flow of Equation 1, the following is obtained. That is, the vector <x> = (x, y) and the vector <d> = (uΔt, vΔ
As t), if three matching errors of the zero-order derivative and each primary derivative with respect to x and y are evaluated, I (x, y, t)
= I1 (x, y), I (x, y, t + Δt) = I2
In (x, y), the problem of minimizing Equation 1 can be solved as the problem of minimizing the following integral.

[0019]

(Equation 9) Thus, not only the function I (x, y, t) but its x,
Even if each partial derivative of y is included in the error evaluation,
Is represented by a quadratic function with respect to u and v, so that it can be solved as a linear equation by the variational principle.

Next, a matching apparatus for minimizing the function B (<d>) in Equation 8 will be described with reference to FIG. In FIG. 1, f1 and f2 denote signals f1 (<x>) and f2 (<x>), respectively, and these signals f1 and f2 are passed through bus lines 1 and 2, respectively, to the partial differential filter circuits 3-0 to 3-0. Supply 3-K. These partial differential filter circuits 3-0 to 3-K obtain partial differentials defined by Equation 6 of the functions f1 and f2, respectively.

Specifically, the elements x1 and x of the vector <x>
Assuming that the highest order of the partial differentiation with respect to 2, 2, ‥‥, xn is M1, M2, Ｍｎ, Mn, respectively, the partial differentiation filter circuit 3-0 outputs a 0th-order partial differential function for all elements of the functions f1 and f2. f1 ^{(0,0, ... , 0)} (<x>) (that is, f1
Itself) and f2 ^{(0,0, ... , 0)} (<x>) (ie, f
2 itself), and the partial differential filter circuit 3-1 outputs f1 which is a first-order partial differential function for the element x1.
^{(1,0, ... , 0)} (<x>) and f2
^(1, x ^{, ... , 0)} (<x>) is output from the partial differential filter circuit 3-2, and f1 ^{(0, 1, ...} ) Which is a primary partial differential function related to the element x2. ^{. , 0)} (<x>) and f2
^{(0,1, ... , 0)} (<x>) is output, and the partial differential function f1 ^{(m1, m2,
... , mn)} (<x>) and f2
^{(m1, m2, ... , mn)} (<x>) is output. in this case,
The following equation holds. i = m1 + m2 (M1 + 1) + ‥‥ mn (M1 + 1)
(M2 + 1) ‥‥ (M (n-1) +1)

The last partial differential filter circuit 3-K
From f1 ^{(M1, M2, ...
, Mn)} (<x>) and f2 ^{(M1, M2, ... , Mn)} (<x>)
Is output, and the following equation holds. K = M1 + M2 (M1 + 1) + {Mn (M1 + 1) (M2 + 1) } (M ( n-1) +1) = (M1 + 1) (M2 + 1) ‥‥ (Mn + 1) -1

The partial differential filter circuits 3-i (i = 0)
ＫK) are supplied to the error detection circuit 4-i. These error detection circuits 4-0 to 0-0
4- is also supplied with a motion vector <d> output from the control circuit 6 described later. FIG. 2 shows the error detection circuit 4-i. Assuming that two partial differential functions supplied to the error detection circuit 4-i are w1 (<x>) and w2 (<x>),
The error detection circuit 4-i calculates and outputs the square of the norm of the matching error ε (<x>, <d>) calculated by the following equation.

(Equation 10)

This equation (10) is equivalent to the equation (7). From each error detection circuit 4-i, the square of the norm of the matching error defined by the equation (7) is expressed as ‖ε _{m1, m2} ,. (<X>,
<D>) ‖ ² is output, and supplies the squared norm of the matching error, each multiplication circuit 7-i (i = 0~K) . Each of these multiplication circuits 7-i multiplies the input data by the coefficient _{Cm1, m2,..., Mn} used in _Equation 8, and supplies the obtained product to the addition circuit 9.

[0025] 5 represents the incidental conditions applying circuit, this also supplies motion vector <d> from the control circuit 6 in the supplementary conditions applying circuit 5, the side conditions applying circuit 5, a function of the number 8 B
The additional condition {P (<d (<x>)>)} ² , which is the second term of (<d>) (excluding the constant multiple), is calculated, and the additional condition is supplied to the multiplication circuit 8. The multiplication circuit 8 supplies data obtained by multiplying the incidental condition by a constant λ to the addition circuit 9. The addition circuit 9 supplies the control circuit 6 with a function B (<d>) of Expression 8 obtained by adding all the supplied data. The control circuit 6 uses the function B
The value of the vector <d> is increased or decreased so that (<d>) is minimized and output.

The method of setting the vector <d> includes a method of sequentially changing <d> by the steepest gradient method while observing an input value obtained by changing the output vector <d> by a small amount, and a method of detecting an error. From the feedback system including the circuits 4-0 to 4-K and the control circuit 6, a linear equation in the case where the square of the norm is a square integral or a sum of squares is obtained (linear Euler equation by the least square method and the variation method). There is a method of finding a solution by solving. In the latter case, a unique solution can be obtained.

The partial differential filter circuits 3-0 to 3-K
Can be extended to any filter. For example, when handling an image signal, a Gabor (Gabor) filter or the like that is strong in the detection of a texture or the like is connected instead of the partial differential filter circuit, or by adding in parallel to the partial differential filter circuit, The matching solution <d> of the functions f1 and f2 including the feature amount can be obtained. As described above, the present invention is not limited to the above-described embodiment, but can take various configurations without departing from the gist of the present invention.

[0028]

According to the present invention, there is an advantage that even when the motion vector <d> is long, the motion vector <d> can be obtained accurately.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a signal matching device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining the operation of an error detection circuit in the example of FIG. 1;

[Explanation of symbols]

 3-0 to 3-K partial differential filter circuit 4-0 to 4-K error detection circuit 7-0 to 7-K multiplication circuit 5 incidental condition provision circuit 6 control circuit 8 multiplication circuit 9 addition circuit

Claims (2)

(57) [Claims] 1. A signal matching method for obtaining a motion vector for matching a first signal that can be expressed by a first function of a certain vector with a second signal that can be expressed by a second function of the vector. Calculating partial differential functions of various orders with respect to each element of the certain vector of the first function and the second function,
The matching error consisting of the square of the absolute value of the difference between the partial differential function of the function of the second function and the first-order Taylor expansion of the partial differential function of the second function with respect to the test motion vector is obtained for the various orders. An evaluation function is obtained by adding the incidental conditional expression relating to the test motion vector to the weighted sum of the matching errors of the following order, and the evaluation function is supplied to the control unit. On to minimize
The value of the test motion vector is determined, and the value of the evaluation function is
The test motion vector at the time of minimization is
Movement for matching the first signal and the second signal
A signal matching method characterized by outputting as a vector . 2. A signal matching apparatus for obtaining a motion vector for matching a first signal that can be expressed by a first function of a certain vector with a second signal that can be expressed by a second function of the vector. A partial differential filter circuit for obtaining partial differential functions of various orders with respect to each element of the certain vector of the first function and the second function; a partial differential function of the first function; and a second function An error detection circuit for obtaining a matching error consisting of the square of the absolute value of the difference between the partial differential function and the first-order Taylor expansion of the test motion vector for the various orders, and a weighted sum of the matching errors of the various orders An evaluation function calculating circuit for obtaining an evaluation function by adding an additional conditional expression relating to the test motion vector to the test function; A control circuit for determining the value of the motion vector, and a motion vector for matching the test motion vector when the value of the evaluation function is minimized with the first signal and the second signal. A signal matching device characterized in that it is regarded as a signal matching device.