JP2747378B2 - Motion vector interpolation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector interpolation device for interpolating a motion vector representing a parallel movement of an entire image.

[0002]

2. Description of the Related Art As a conventional motion vector interpolation device,
Technical Report of the Institute of Television Engineers of Japan, PROBE-5, May 1985, "M
USE Motion Vector Detector, ”pp. Motion vector interpolators shown at 25 to p30 are known.

The above-described conventional motion vector interpolation apparatus is not suitable for detecting a motion vector when an image is small and the image is uniform, or when an object having a large difference in luminance exists in the image with regularity. Is divided into four into 2 × 2 regions, and each region is individually determined.

FIG. 11 shows the configuration of the above-mentioned conventional motion vector interpolation apparatus.

In FIG. 11, an input terminal 21 has a current frame.
The image signal of the current or current field is input. The representative point memory 22 stores one of the previous frame or the previous field, respectively.
The information of the representative point of the area of / 4 is stored.

Each correlation calculator 23 calculates the correlation between the information of the current image input from the input terminal 21 and the information of the representative point of the previous image stored in each representative point memory 22, and calculates each correlation accumulation. The adder 24 accumulates and adds the correlation calculated by the correlation calculator 23.

[0007] Each candidate vector generator 25 determines a motion vector in each area based on the correlation accumulated value calculated by each correlation accumulation adder 24 to determine its validity, and determines each candidate motion vector and its validity. Output the judgment result.

[0008] The motion vector generator 26 checks the coincidence between the direction and the magnitude of the candidate motion vector determined to be valid among the candidate motion vectors of each area from each candidate vector generator 25, and If they match, each motion vector is averaged to obtain a motion vector for one screen. If they do not match, the motion vector for one screen is output to the output terminal 27 as a zero vector.

[0009]

However, in the above-described conventional motion vector interpolating apparatus, when the directions and the sizes match, each motion vector is averaged to obtain a motion vector of one screen, and the motion vectors do not match. Since the motion vector of one screen is sometimes output as a zero vector, there is a problem that the average vector and the zero vector are switched between the screens and a jump in the motion of the image occurs.

The present invention has been made in view of the above-mentioned problems in the conventional motion vector interpolation apparatus, and has been described in the art. Provided is a motion vector interpolation device capable of preventing occurrence of jump.

[0011]

SUMMARY OF THE INVENTION The present invention relates to a detecting means for detecting a motion vector indicating an average moving amount of an entire image based on a correlation between a predetermined number of images successive in time series, and This is achieved by a motion vector interpolating apparatus including a determination unit that determines reliability of a vector image and a control unit that controls a coefficient of an adaptation unit so as to smooth a change in a motion vector based on the determination result. Is done.

[0012]

According to the motion vector interpolation apparatus of the present invention, the detecting means detects a motion vector indicating the average moving amount of the entire image based on the correlation between a predetermined number of images successive in time series, The reliability of the detected motion vector in the image is determined, and the control unit controls the coefficient of the adaptation unit to smooth the change of the motion vector based on the determination result.

[0013]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a motion vector interpolation apparatus according to the present invention.

FIG. 1 is a block diagram showing the configuration of an embodiment of the motion vector interpolation device according to the present invention.

The motion vector interpolation device shown in FIG.
11, representative point memory 12, correlation calculator 13, correlation accumulator 1
4. It comprises a motion vector detecting unit 15 as a detecting unit, a reliability determining unit 16 as a determining unit, a coefficient control unit 17 as a controlling unit, a motion vector interpolating unit 18 as an interpolating unit, and an output terminal 19.

In FIG. 1, an input terminal 11 receives an image signal of a current frame or a current field of a temporally continuous moving image.

The representative point memory 12 stores pixel signals of a plurality of representative points k in the previous frame or the previous field.

The correlation calculator 13 calculates the coordinates (i) from the position of the pixel of the representative point k by using the pixel of the representative point k of the previous image stored in the representative point memory 12 and the current image input from the input terminal 11. , J) the absolute value of the difference between the signals of the pixels | ρ _k
(I, j) | is obtained and output to the correlation accumulating and adding unit 14.

The correlation accumulator 14 accumulatively adds the absolute values | ρ _k (i, j) | for each deviation (i, j) for all the representative points k, and accumulates the accumulated values | ρ (i, j, j). j) | ｛= Σ | ρ _k
(I, j) |｝ is output to the motion vector detecting unit 15 and the reliability determining unit 16 as a correlation value for the deviation (i, j).

The motion vector detector 15 calculates the correlation value | ρ
The deviation (i, j) of the minimum value MIN of (i, j) |
The detected deviation (i, j) is output to the motion vector interpolation unit 18 as a motion vector.

The reliability judging section 16 judges the reliability of the motion vector and outputs the judgment result to the coefficient control section 17. The reliability judgment unit 16 calculates the correlation value | ρ from the correlation accumulation adding unit 14
(I, j) |, the minimum value of the correlation value | ρ (i, j) |
MIN, maximum value MAX, average value AVE of the whole screen (or average MAX 'at four corners of the screen), ratio MIN / MAX (or ratio MIN /
MAX ') and find the ratio MIN / MAX (or the ratio MIN / MAX
A motion vector obtained in a range where the value of ′) is larger than the threshold th is determined to have low reliability.

The coefficient control unit 17 calculates the minimum count value “0”,
An up-down counter and a look-up table (LUT) limited to the maximum count value m are provided.

The motion vector interpolation unit 18 includes an adaptive low-pass filter (LPF) as an adapting means, and the up / down counter of the coefficient control unit 17 includes a reliability determination unit 16.
When the reliability of the motion vector is determined to be low, one is counted up, and when the reliability is determined to be high, one is counted down.

Next, the operation of the motion vector interpolation apparatus shown in FIG. 1 will be described with reference to FIG.

First, the count value of the coefficient control counter is set to "0" (step S1), a reference value is received from the reliability determining unit 16 (step S2), and the received reference value, that is, the ratio MIN / MAX (or ratio MIN / MAX ´)
Is compared with the threshold value th, and the reliability of the motion vector is determined based on the comparison result (step S3).

If it is determined in step S3 that the motion vector is reliable, it is determined whether or not the count value is 0 (step S4). If the count value is 0 in step S4, a step to be described later is performed. If the count value is not 0 in step S4, the count value is reduced by one (step S5), and the process proceeds to step S8.

On the other hand, when it is determined in step S3 that there is no reliability of the motion vector, it is determined whether or not the count value is equal to the maximum value m (step S6). In step S6, the count value is equal to the maximum value m. If so, proceed to step S8,
If the count value is not equal to the maximum value m in step S6, the count value is increased by one (step S7).

Based on the count value obtained in step S4, step S5, step S6 or step S7, the coefficient of the LPF is determined and output from the LUT (step S8), and it is determined whether or not to continue the operation. Then (step S9), when the operation is continued, the process returns to step S2, and when the operation is not continued, the operation is terminated.

FIG. 3 shows a change in the criterion amount in the reliability determinator 16, and the criterion amount is MIN / MAX (or the minimum value MIN / the average value MAX '). This amount is set to a predetermined threshold
When binarized by th, the result is as shown in FIG. 4. When the value is "1", the reliability is low, and when the value is "0", the reliability is high.

FIG. 5 is an enlarged view of a part of FIG. 4 in the time axis direction. At this time, the count value of the limited up / down counter changes as shown in FIG.

The look-up table (L
UT) is a function value “1” when the count value of the up / down counter is a minimum value “0”, is a function value “0” when the count value is a maximum value m, and simply decreases as the count value increases. It has a function value, and outputs a function value corresponding to the count value of the up / down counter as the coefficient value h of the adaptive LPF of the motion vector interpolation unit 18.

As another embodiment of the coefficient control unit 17,
The coefficient control unit 17 is composed of an LPF and an LUT,
A specific amount serving as a criterion in the reliability determination unit 16 may be input to the LPF, and the coefficient of the LPF may be output using an LUT based on the output of the smoothed LPF.

The operation of the coefficient control unit 17 in another embodiment will be described with reference to FIG.

First, the judgment reference amount is received by the reliability judgment unit 16 (step T1), the judgment reference amount is smoothed by passing through the LPF (step T2). The coefficients are determined and output (step T3). Further, it is determined whether or not to continue the operation (step T4). If the operation is to be continued, the process returns to step T1, and if not, the operation is terminated.

In the motion vector interpolation unit 18, the characteristics of the LPF change adaptively according to the coefficient value h, and the motion vector from the motion vector detection unit 15 is passed according to this characteristic.

FIG. 8 shows the input of the motion vector, that is, the LPF from the motion vector detecting section 15, FIG. 9 shows the judgment result of the reliability judging section 16, and FIG. 10 shows the output of the LPF. When the reliability determination unit 16 determines that the reliability of the motion vector is low, the motion vector is smoothly interpolated by the LPF, and no jump of the motion vector occurs at the beginning and end of the interpolation.

The motion vector interpolation apparatus of the present embodiment is useful as a camera-integrated video with an image blur correction function, a VTR with an image blur correction function, a moving object tracking device, and an image encoding device.

[0038]

According to the motion vector interpolation apparatus of the present invention, a detecting means for detecting a motion vector indicating an average moving amount of an entire image based on a correlation between a predetermined number of images consecutive in time series, and A determination unit for determining the reliability of the motion vector in the image; and a control unit for controlling a coefficient of a predetermined adaptation unit so as to smooth a change in the motion vector based on the determination result. By controlling the coefficient of the predetermined adaptation means so that the change of the vector becomes smooth,
It is possible to prevent the image from jumping when the image with few pictures is uniform or when an object having a large difference in luminance exists in the image with regularity.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a motion vector interpolation device according to the present invention.

FIG. 2 is a flowchart showing the operation of the motion vector interpolation device of FIG.

FIG. 3 is a diagram showing a change in a criterion amount in the reliability determinator of FIG. 1;

FIG. 4 is a diagram showing values when the change of the determination reference amount in FIG. 3 is binarized.

5 is a diagram in which a part of a change in a binarized determination reference amount in FIG. 4 is enlarged in a time axis direction.

FIG. 6 is a diagram showing a restricted up-down counter value in the state of FIG. 5;

FIG. 7 is a flowchart for explaining another embodiment of the coefficient control unit in FIG. 1;

FIG. 8 is a diagram showing a motion vector from a motion vector detection unit in FIG. 1;

FIG. 9 is a diagram illustrating a determination result of a reliability determination unit in FIG. 1;

FIG. 10 is a diagram illustrating an output of a low-pass filter (LPF) of the motion vector interpolation unit in FIG. 1;

FIG. 11 is a block diagram showing a conventional motion vector interpolation device.

[Explanation of symbols]

 11 Input terminal 12 Representative point memory 13 Correlation calculator 14 Correlation accumulator 15 Motion vector detector 16 Reliability judgment unit 17 Coefficient control unit 18 Motion vector interpolation unit 19 Output terminal

Continuation of front page (72) Inventor Yasukun Yamane 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation (58) Field surveyed (Int.Cl. ⁶ , DB name) G06T 7/20 H04N 7/24 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. A motion vector interpolation device for interpolating a motion vector representing a parallel movement of an entire image by using an adapting means, wherein the motion vector interpolator is configured to calculate a motion vector of the entire image based on a correlation between a predetermined number of images that are continuous in time series. Detecting means for detecting a motion vector indicating an average movement amount; determining means for determining reliability of the detected motion vector in the image; and smoothing a change in the motion vector based on the determination result. And a control means for controlling a coefficient of the adaptation means.