JPH04321187A - Method and device for matching of signal - Google Patents

Abstract

PURPOSE:To accurately obtain even a long active vector <d> that functions to secure the matching between the 1st and 2nd functions f1 and f2 of a vector <x>. CONSTITUTION:The partial differentialtion filter circuits 3-0-3-K obtain the partial differential functions of various degrees in regard of each element xi of both functions f1 and f2. The error detecting circuits 4-0-4-K obtain the matching error equal to a square of the absolute value of the difference from the primary Taylor series expansion in regard of the active vector <d> of the partial differential function computing circuits 5, 7-0-7-K, 8 and 9 obtain the evaluation function B(<x>, <d>) by adding an additional condition formula to the weighted sum of the matching errors of various degrees. Then a control circuit 6 decides the valve of the active vector <d> so that the value of the function B is minimized.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to a signal matching method suitable for application to pattern matching used in image processing, image recognition, voice recognition, etc., and a signal matching device used to carry out this method. Regarding.

0002

2. Description of the Related Art The subject of image recognition is shifting from still images to moving images such as hand gestures and gestures. In order to recognize such moving images, it is necessary to detect moving regions using time differences, but in order to detect moving regions more accurately, it is possible to use optical flow. Optical flow is a trajectory that follows pixels with luminance that does not change in the time direction, and corresponds to a motion vector of an object within a two-dimensional image.

That is, the brightness within one frame of the image is I(x
, y, t), and the time difference between two frames is Δt, and if a trajectory (x, y) that satisfies ΔI(x, y, t)/Δt=0 is found,
A vector (u, v) indicating the optical flow is defined as follows. u=Δx/Δt, v=Δy/Δt Furthermore, using the optical flow, the motion vector <d> can be defined as follows. <d>=(uΔt, vΔt) It is known that the calculation of the optical flow (u, v) is equivalent to the calculation of the vector (u, v) that minimizes the following formula. Note that the second term of the following formula is u,
This is a constraint that smoothes v, and λ is a constant that indicates the strength of the constraint.

[Math 1]

[0004] Such an optical flow calculation method can be applied to, for example, matching regarding the time axis shift of two audio signals and matching of two image data. In order to generalize the condition of number 1, consider two functions f1 (<x>) and f2 (<x>) defined on the n-dimensional Euclidean space <R>n. Let the n elements of the vector <x> on the space <R>n, which are the variables of these functions, be (x1, x2, ..., xn), and let the motion vector on the space <R>n be <d>. Then, the motion vector <d> can be expressed as a vector <d(<x>)> as a function of the vector <x>. In this case, the condition corresponding to Equation 1 is equivalent to the condition that minimizes the function A (<x>, <d>) defined by the sum of the squares of the norms in the following equation.

[0005]

[Equation 2] The function P(<d>) of the second term of Equation 2 is an additional condition corresponding to the second term of Equation 1, and this additional condition constrains the behavior of the motion vector <d>. . Further, λ is a constant indicating the strength of the additional condition. The original derivation target is the motion vector <d> that minimizes the first term of the number 2, but as it is, multiple motion vectors <d> may be obtained, so the motion vector <d> must be uniquely determined. Additional conditions have been added to ensure that By taking the square of the norm of number 2 as the integral of the square of the norm or the sum of the squares of the norm, the minimum value problem of number 2 becomes equivalent to the least squares method, but in this way, the square integral of the norm The integral formula of Equation 1 is the one that employs

Although additional conditions are attached to the number 2, the object to be found under the condition of minimizing the number 2 is the motion vector <d that minimizes the first term of the number 2. >, the matching error ε(<x>, <d>) in Equation 2 is defined as follows.

[Math 3] The following approximate expression is used to derive this number 3.

[Math 4]

The first term of the number 2 is a square integral of the number 3, and it becomes a quadratic expression with respect to the motion vector <d>, so
When the second term of Equation 2 is also expressed by a motion vector <d>, the minimum solution becomes equal to the minimum solution, and it was possible to convert it into a linear equation and solve it using the method of least squares and the method of variation. For example, by using the approximation of Equation 3 in Equation 1 for calculating the optical flow, Equation 1 can be transformed as follows.

[Math 5]

Since this equation 5 is a quadratic equation with respect to u and v, in order to find the minimum solution, we convert it into Euler's equation to obtain a linear differential equation, and the motion vector 〈d 〉 can be uniquely determined. In other words, the approximation of Equation 3 represents the matching error in a minute area by approximating the function f2 (<x>) to a tangent plane at <x>.

[0009]

[Problem to be solved by the invention] However, when the absolute value of the motion vector <d> is large, the linear approximation (tangential plane approximation) of the Taylor expansion of the function f2 (<x>) expressed by Equation 4 is sufficient. It was not possible to approximate f2(<x>), and the estimation of the motion vector <d> was unsuccessful. In addition, in order to improve this, a method has been considered to increase the order of the Taylor expansion to improve the approximation accuracy, but the method is as follows:
ε(〈x〉, 〈d〉) has a component of quadratic or higher order with respect to 〈d〉, equation 2 does not become a quadratic expression, and the vector 〈d〉 that minimizes equation 2 is calculated by the variational method. In order to do so, it is necessary to solve a nonlinear equation, and there is no guarantee that the minimum solution will be the minimum solution, so it is inconvenient that <d> cannot be easily determined. In view of these points, the present invention provides a signal matching method that can easily find the vector <d> that minimizes the above-mentioned number 2 even if this vector is long, and a signal matching method that can be used to implement this method. The purpose is to provide a matching device.

0010

[Means for Solving the Problems] The signal matching method according to the present invention includes, for example, as shown in FIG.
The first function f of (x1, x2, ‥‥, xn) = 〈x〉)
In a signal matching method for determining a motion vector <d> for matching a first signal that can be expressed by 1 and a second signal that can be expressed by a second function f2 of the vector, Find the partial differential functions of various orders with respect to each element xi of a certain vector of the function 2, and calculate the partial differential functions of the first function and the partial differential function of the second function with respect to the trial motion vector Matching errors consisting of the square of the absolute value of the difference with the next Taylor expansion are obtained for these various orders, and the contingency expression regarding the above experimental motion vector <d> is added to the weighted sum of the matching errors of these various orders. The evaluation function B(〈x〉, 〈d〉) is obtained by
The experimental motion vector when the value of the evaluation function becomes the minimum is set as the motion vector <d> for matching the first signal and the second signal.

[0011] Furthermore, the signal matching device according to the present invention, for example, as shown in FIG.
A motion vector for matching a first signal that can be expressed by a first function f1 of In a signal matching device for determining d〉,
Partial differential filter circuits (3-0 to 3-K) for obtaining partial differential functions of various orders with respect to each element xi of a certain vector of the first function and the second function, and An error detection circuit that calculates a matching error consisting of the square of the absolute value of the difference between a partial differential function and a first-order Taylor expansion with respect to an experimental motion vector <d> of the partial differential function of its second function for various orders thereof. (4-0 to 4-K) and the weighted sum of matching errors of various orders to the evaluation function B(〈x〉, 〈d〉) by adding the contingency expression regarding the experimental motion vector. An evaluation function calculation circuit (5, 7-0 to 7-K, 8, 9) to be obtained, and a control circuit (6 ), and the experimental motion vector when the value of the evaluation function becomes the minimum is regarded as the motion vector for matching the first signal and the second signal. .

0012

[Operation] According to the signal matching method of the present invention, the partial differential function of the first function and the partial differential function of the second function are combined with the first-order Taylor expansion with respect to the experimental motion vector <d>. A matching error consisting of the square of the absolute value of the difference is obtained for various orders of each element xi, and the evaluation function B (
<x>, <d>) are formed, and the motion vector <d> is selected so that these weighted sums and contingent expressions are minimized. Therefore, compared to the conventional case where the square of the absolute value of the difference between the first-order Taylor expansion regarding the experimental motion vector <d> of the first function and the second function is used, Even if the length is large, the vector 〈
d> can be estimated.

In this case, the evaluation function B(〈x〉,〈d
〉) is a quadratic expression with respect to the vector <d>, so it can be converted into a linear equation and solved by the least squares method and the variational method as in the conventional example. Further, according to the signal matching device of the present invention, the signal matching method can be directly implemented.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. This example is the function A (〈x〉,〈
When finding the motion vector <d> that minimizes the absolute value of the function f2 (<x> + <d>) - f1 (<x>) by finding the motion vector <d> that minimizes d>). The present invention is applied to. In this example as well, the matching error ε(〈x〉, 〈d〉) can be expressed by Equation 3, but in this example, the matching error remains linear with respect to the motion vector 〈d〉 and is essentially a higher-order Taylor expansion. In order to express up to the terms, Expression 3 can be expressed as a vector 〈x〉 of the function f2 (〈x〉)
This is expanded to the error for partial derivatives with respect to each element of .

For this reason, the n-dimensional Euclidean space <R>
Let each n element of vector <x> on n be x1, x2,
‥‥, xn, elements x1, x2 of function f(〈x〉)
, ..., m1, m2, ..., mn-th partial differential f (m1, m2, ..., mn) (<x>) with respect to xn is expressed as follows.

[Math 6]

Then, the matching error ε(<x>
, <d>) is defined as the matching error εm1,
Expand to m2, ..., mn (<x>, <d>). This matching error is obtained by subtracting the partial differential function of the function f1 (<x>) from the first-order Taylor expansion regarding the motion vector <d> of the partial differential function of the function f2 (<x>).

[Math 7]

Using the matching error of Equation 7, we can solve the problem of Equation 2 by converting the following function B (<x>, <d>) to vector <d>
Solve as a minimization problem with respect to. Cm in the formula
1, m2,..., mn are constants.

[Math. 8]

For example, when this is applied to the optical flow calculation formula of Equation 1, it becomes as follows. That is, vector <x> = (x, y), vector <d> = (uΔt, vΔ
t), if we evaluate the three matching errors of the zero-order differential and each first-order differential with respect to x and y, we get I(x, y, t)
= I1 (x, y), I (x, y, t + Δt) = I2 (x
, y), the problem of minimizing Equation 1 can be solved as a problem of minimizing the following integral.

[0019]

[Equation 9] In this way, not only the function I(x, y, t), but also its x,
Even if each partial differential with respect to y is also incorporated into the error evaluation, Equation 9
Since is expressed as a quadratic function with respect to u and v, it can be solved by converting it into a linear equation using the variational principle.

Next, a matching device that minimizes the function B (<x>, <d>) of Equation 8 will be explained with reference to FIG. In FIG. 1, f1 and f2 are the signal f1(〈x
) and f2 (<x>), and these signals f1 and f2 are supplied to partial differential filter circuits 3-0 to 3-K via bus lines 1 and 2, respectively. These partial differential filter circuits 3-0 to 3-K obtain partial differentials defined by Equation 6 of the functions f1 and f2, respectively.

Specifically, elements x1 and x2 of vector <x>
,..., the highest order of partial differential with respect to xn is M
1, M2,..., Mn, partial differential filter circuit 3
From −0, the 0th order partial differential function f1(0,0,...,0)(<x>)(
That is, f1 itself) and f2 (0, 0, ..., 0
)(<x>) (that is, f2 itself) is output, and the partial differential filter circuit 3-1 outputs f1(1,0,...,0)( <x>
) and f2(1,0,...,0)(<x>) are output, and the partial differential filter circuit 3-2 outputs f1(0,1, ...,0
)(<x>) and f2(0,1,...,0)(<x
〉) is output, and the i-th partial differential filter circuit 3-i outputs the partial differential function f1(m1, m2,..., m
n) (<x>) and f2 (m1, m2,..., mn
)(<x>) is output. In this case, the following formula holds. i=m1+m2+‥‥+mn

Then, the final partial differential filter circuit 3-K
, the highest order partial differential function f1(M1, M2,
..., Mn) (<x>) and f2 (M1, M2, .
..., Mn)(<x>) is output, and the following equation holds true. K=M1+M2+...+Mn

These partial differential filter circuits 3-i (i=0
The two partial differential functions calculated in steps .about.K) are respectively supplied to the error detection circuit 4-i. These error detection circuits 4-0~
4- is also supplied with a motion vector <d> output from a control circuit 6, which will be described later. FIG. 2 shows the error detection circuit 4-i, and assuming that the two partial differential functions supplied to the error detection circuit 4-i are w1 (<x>) and w2 (<x>),
This error detection circuit 4-i calculates and outputs the square of the norm of the matching error ε(<x>, <d>) calculated from the following equation.

[Math. 10]

This equation 10 is equivalent to equation 7, and from each error detection circuit 4-i, ‖εm1, m2, . . . , mn(
〈x〉, 〈d〉)‖2 is output, and the square of the norm of the matching error is applied to the multiplier circuit 7-i (i=0~
K). These multiplier circuits 7-i each have coefficients Cm1, m2, . . . used in equation 8 for input data.
. . , mn, and the resulting product is supplied to the adder circuit 9.

Reference numeral 5 denotes a supplementary condition providing circuit, to which the motion vector <d> is also supplied from the control circuit 6, and this incidental condition providing circuit 5 operates as a function B (
x〉,〈d〉) (excluding constant multiplication), the incidental condition ‖P(〈d(〈x〉)〉)‖2 is calculated, and this incidental condition is supplied to the multiplication circuit 8. . The multiplication circuit 8 multiplies the incidental condition by a constant λ and supplies the obtained data to the addition circuit 9. This adder circuit 9 calculates the function B (<x>, <
d>) is supplied to the control circuit 6. This control circuit 6 increases or decreases the value of the vector <d> so that the function B(<x>, <d>) becomes minimum and outputs the value.

The method of setting the vector <d> includes a method of sequentially changing <d> using the steepest gradient method while checking the input value when the output vector <d> is changed by a minute amount, and an error detection method. Obtain a linear equation when the square of the norm is a square integral or a sum of squares from the feedback system including circuits 4-0 to 4-K and the control circuit 6 (linear Euler equation using the least squares method and the variational method). There are methods to find the solution by solving . In the latter case, a solution can be uniquely obtained.

Note that the partial differential filter circuits 3-0 to 3-K
can be extended to any filter. For example, when handling image signals, a Gabor filter, which is strong in detecting textures, etc., can be connected in place of these partial differential filter circuits, or it can be added in parallel to these partial differential filter circuits to detect textures, etc. A matching solution <d> of the functions f1 and f2 including the feature amount can be obtained. As described above, it goes without saying that the present invention is not limited to the above-described embodiments, and can take various configurations without departing from the gist of the present invention.

[0028]

According to the present invention, there is an advantage that even if the motion vector <d> is long, the motion vector <d> can be accurately determined.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of a signal matching device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining the operation of the error detection circuit in the example of FIG. 1;

[Explanation of symbols]

3-0~3-K Partial differential filter circuit 4-0~4-K
Error detection circuits 7-0 to 7-K Multiplying circuit 5 Additional condition applying circuit 6 Control circuit 8 Multiplying circuit 9 Adding circuit

Claims (2)

[Claims] Claim 1. A signal matching method for determining a motion vector for matching a first signal that can be expressed by a first function of a certain vector and a second signal that can be expressed by a second function of that vector. , find partial differential functions of various orders for each element of the certain vector of the first function and the second function, and calculate the partial differential functions of the first function and the second function. A matching error consisting of the square of the absolute value of the difference between the experimental motion vector and the first-order Taylor expansion is obtained for the various orders, and the weighted sum of the matching errors of the various orders is added to the incidental value regarding the experimental motion vector. An evaluation function is obtained by adding conditional expressions, the value of the experimental motion vector is determined so that the value of the evaluation function is minimized, and the experimental motion is determined when the value of the evaluation function is minimized. A signal matching method characterized in that the vector is a motion vector for matching the first signal and the second signal. 2. A signal matching device for determining a motion vector for matching a first signal that can be expressed by a first function of a certain vector and a second signal that can be expressed by a second function of that vector. , a partial differential filter circuit that obtains partial differential functions of various orders with respect to each element of the certain vector of the first function and the second function;
Matching errors consisting of the square of the absolute value of the difference between the partial differential function of the first function and the first-order Taylor expansion regarding the experimental motion vector of the partial differential function of the second function are determined for the various orders. an error detection circuit; an evaluation function calculation circuit that calculates an evaluation function by adding the incidental conditional expression regarding the experimental motion vector to the weighted sum of the matching errors of various orders; and a control circuit that determines the value of the experimental motion vector, and matches the experimental motion vector when the value of the evaluation function is minimized with the first signal and the second signal. A signal matching device characterized in that the signal matching device is configured to consider the signal as a motion vector for causing the signal to move.