JP3020299B2 - Motion vector detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to various image processing apparatuses,
The present invention relates to a motion vector detecting device for detecting a motion of an object in an image used in an image compression encoding device.

[0002]

2. Description of the Related Art Conventionally, as a method for detecting a motion vector of a moving object in an image, "digital signal processing of an image" has been known.
(Takehiko Fuukiki, published by Nikkan Kogyo Shimbun, pp.221-
227). Hereinafter, a “method of obtaining a deviation that minimizes the difference between successive images” described in the above-mentioned document will be described as a conventional example.

[0003] In this method, basically, two images (2
(Also referred to as a frame) little by little, while searching for a place where the difference is minimized, using the amount of the shift as a motion, and expressing the motion using a vector. That is, when i is the frame number of the current frame image, data g _i (x,
y) (where (x, y) ， [B _x , B _y ] and B
_x is a possible range of x, B _y represents the possible range of y. For _{example, B x = [- 16,16]} , B y = [-
16, 16], (x, y) ∈ [B
_x, B _y] and the point (x, y) -16 ≦ x ≦ 16, -
16 ≦ y ≦ 16), the data g _i−1 (x, y) of the previous frame image is stored in a predetermined range [Wx, Wy] (hereinafter, a search range [Wx, Wy]
) In the X-axis direction and η (((ξ,
η) ∈ [Wx, Wy]), and calculate the error E (ξ, η) in the following equation (1) or (2). Then, (ξ0, η0) satisfying the expression (3), that is, (1)
A point at which the error amount E (ξ, η) of the equation (2) is minimized is detected, and the movement amount (ξ0, η0) at this time is used as the motion vector of the pixel block ([Bx, By]) of interest. .

[0004]

(Equation 1)

FIG. 2 shows a configuration of a conventional motion vector detecting device to which such a method is applied.

In FIG. 2, a current frame memory 1 stores an input image signal (current frame image signal) 5, and a previous frame memory 2 outputs an image signal of one frame before (current frame image signal) output from the current frame memory 1. Signal) 13 or an image signal 14 input from the outside. The address generator 3 generates a two-dimensional address (x, y) 11 for the current frame memory 1 and generates image data gi (x, y).
y) Output 6 Further, within the predetermined range (vector search range) with respect to the previous frame memory 2, ξ, x in the x direction from the two-dimensional address (x, y).
Address (x-ξ, y-η) 10 shifted by η in y direction
Is generated, and the image data gi-1 (x-ξ, y-η) 7 is output. The address shift (ず れ, η) 12 is output to the motion vector discriminating unit 4. The motion vector discriminating unit 4 calculates the image data g output from the current frame memory 1.
i (x, y) 6 and the image data gi-1 (x-ξ, y-η) 7 output from the previous frame memory 2, within a predetermined range (pixel block), (1 ) Or (2) to calculate an error E (ξ, η) for determining a motion vector.

The above operation is performed on all vector candidates within the vector search range while changing ξ and η.
A vector that satisfies the expression (3) is calculated. This vector (ξ0, η0) is used as the motion vector of the pixel block, and the prediction error signal gsi (x,
y) 8 and a motion vector (ξ0, η0) 9 are output. Note that the above method is called pattern matching using an average minimum error or pattern matching using an average least square error.

Gsi (x, y) = gi (x, y) −gi−1 (x−ξ0, y−η0) (4)

[0009]

However, in the apparatus having the above structure, as shown in FIG. 4A, the evaluation of the expression (3) must be performed for all possible pixel blocks within the search range. For example, since the motion vector of the pixel block of interest cannot be detected, the amount of calculation becomes enormous, and the efficiency of detecting the motion vector is extremely low. The only way to avoid this problem is to narrow the search range in the conventional method.On the other hand, when the search range is narrowed, the possibility that the motion of the object exceeds the search range increases, and the accuracy of detecting the motion vector decreases. Resulting in.

In the conventional method, a motion vector is detected by pattern matching independently for each pixel block. Therefore, when a similar pattern (pixel block) occurs accidentally, the motion vector is detected. May be detected incorrectly. In particular, in the case of a color image, motion vector detection is generally performed only for the luminance component.
Such an error in motion vector detection has a large adverse effect. In addition, the false detection often occurs in the contour portion of the object.

The present invention solves the above-mentioned trade-off problem between the calculation amount and the motion vector accuracy, and is excellent in reducing errors in motion vector detection, particularly in improving the accuracy of motion vector detection in the contour of an object. The purpose is to provide.

[0012]

For that purpose, a motion vector detecting apparatus according to a first aspect of the present invention includes the following means.

(A) A current frame memory for storing current one frame of image data.

(B) A previous frame memory for storing one frame of past image data.

(C) sequentially generating pixel addresses in a pixel block of a predetermined size with respect to the current frame memory, and predicting pixel addresses within the search range determined by the search range determination means with respect to the previous frame memory. Address generation means for sequentially generating pixel addresses in the pixel block while shifting the pixel block of the predetermined size based on a prediction vector obtained from the vector generation means.

(D) Contour detection means for detecting whether or not a contour exists in a pixel block in the current frame memory, and outputting the presence or absence of the contour and the direction of the contour line as contour information.

(E) A contour information storage memory for storing the contour information of each pixel block.

(F) By performing pattern matching between the image data of the current pixel block output from the current frame memory and the image data of the comparison pixel block output from the previous frame memory, an optimal vector as the motion vector of the current pixel block is obtained. Motion vector determining means for determining.

(G) A motion vector storage memory for storing the motion vector for each pixel block in the current frame memory.

(H) By comparing the outline information of the target pixel block with the outline information of the peripheral pixel blocks, a reference pixel block around the target pixel block is selected, and a prediction vector is calculated based on the motion vector of the reference pixel block. Predicted vector generating means to be generated.

(I) Search range determining means for determining a search range of a motion vector based on the degree of variation between the prediction vector and the motion vector of the reference pixel block and the number of the reference pixel blocks.

According to a second aspect of the present invention, in the predictive vector generating means according to the first aspect of the present invention, based on the presence or absence of a contour in the pixel block of interest, a pixel block having a contour or a pixel block having no contour is determined from an adjacent pixel block adjacent to the pixel block of interest. A pixel block is selected, and a feature vector extracted from a motion vector of the selected pixel block is used as a prediction vector of a pixel block of interest. If there is no pixel block corresponding to an adjacent pixel block, prediction is stopped. And

In a third aspect of the present invention, in the prediction vector generating means according to the second aspect, when an outline exists in the pixel block of interest, an outline exists from an adjacent pixel block adjacent to the pixel block of interest, and A feature is that a pixel block having a high degree of coincidence in the contour direction is selected.

[0024]

The present invention pays attention to the following points, and reduces the amount of computation for detecting a motion vector.
An object of the present invention is to improve detection accuracy, particularly, detection accuracy of a motion vector in a contour portion of a moving object.

That is, a moving object in an image generally has a certain size, and when the image is divided into small pixel blocks (for example, 8 × 8 pixels, 16 × 16 pixels),
The moving object is considered to span several sub-pixel blocks. In addition, it is considered that each pixel block inside the moving object moves almost in the same manner, and moves differently inside and outside the moving object. Utilizing this property, contour information of the target pixel block and adjacent pixel blocks adjacent to the target pixel block is obtained, and an adjacent pixel block that moves in the same manner as the target pixel block is selected from the outline information. The motion of the pixel block of interest can be predicted with constant accuracy from the motion vector of the selected adjacent pixel block. Therefore, FIG.
As shown in (b), even if a search is performed within a narrower range than the related art around a pixel block moved by the amount of the prediction vector,
The search accuracy of the motion vector does not decrease, and the amount of calculation can be made smaller than before.

In the conventional method, a motion vector can be detected only within the search range. However, in the present invention, since the search range does not limit the size of the motion vector, the motion vector can be detected over a wide range. Can be detected, and the detection accuracy is improved.

[0027]

An embodiment of the present invention will be described below with reference to the drawings.

[Overall Configuration of Motion Vector Detection Apparatus] First, the overall configuration of the motion vector detection apparatus of the embodiment will be described.

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention. Reference numeral 20 denotes a current frame memory, 21 denotes a previous frame memory, 22 denotes an address generator, 23 denotes a motion vector discriminator, and 24 denotes a motion vector. A storage memory, 25 is a predicted vector generation unit, 26 is a search range determination unit, 27 is a contour detection unit, and 28 is a contour information storage memory.

The current frame memory 20 stores an input image signal 29
Is stored in the previous frame memory 21. The previous frame memory 21 stores the image signal 30 for the past one frame given from the current frame memory 20 or the past one given from the outside.
This is a memory for storing image signals 31 for frames.

The address generator 22 applies an address (x, y) 3 of a pixel in a block to the current frame memory 20 by a predetermined pixel block scanning method (see FIG. 3).
2 are sequentially generated, and for the previous frame memory 21, a prediction vector (x0, y0) 33 output from a prediction vector generation unit 25 described later and a search range [ Wx, Wy] 34, the pixel is moved by the prediction vector from the pixel block position of the current frame, and further, the address (x−x0 + ξ, y) of the pixel in the pixel block moved by (ξ, η) within the search range.
−y0 + η) 35 are sequentially generated. The address shift (x0-ξ, y0-η) 36 is output to the optimum vector discrimination unit 23.

With the above operation as one cycle, all movable vectors (ξ, η) ((ξ, η) ∈) within the search range.
[Wx, Wy]).

The contour detecting section 27 refers to the image data 37 output from the current frame memory 20 and detects the contour information of the current pixel block in the current frame by the [contour detection method] described later.

The contour information storage memory 28 includes a contour detecting unit 2
7 stores the outline information 39 of each pixel block.

The motion vector discriminating section 23 outputs the pixel block data gi output from the current frame memory 20.
(X, y) 37 and pixel block data gi-1 (x-x0 + ξ, yy) output from the previous frame memory 21.
0 + η) 38, the above equation (1) or (2)
The calculation of the equation is performed to calculate an error amount E (x0-ξ, y0-η). Further, all vectors (内, η) within the search range
, The above error amount is calculated, and a vector (x0-ξ0, y0-η0) satisfying the expression (3) is detected. This vector (x0−ξ0, y0−η0) is output as a motion vector 41, and at the same time, the current pixel block data gi (x, y) 37 and the previous pixel are used as a motion compensated prediction error signal of the current pixel block in the current frame. Block data g
The difference signal gsi (x, y) 40 from i-1 (xx-0 + ξ0, y-y0 + η0) 38 is calculated and output in the same manner as in the above equation (4).

The motion vector storage memory 24 stores the motion vector (x0-ξ) output from the optimum vector discriminating section 23.
0, y0-η0) 41 is stored.

The predicted vector generation unit 25 inputs from the contour information storage memory 28 the contour information 42 of the pixel block of interest whose motion vector is to be detected and the neighboring pixel blocks around which a motion vector has already been detected. The adjacent pixel block to be referred to is determined based on the [reference pixel block determination method], and the motion vector storage memory 2
4 motion vector 43 of the reference pixel block
And a prediction vector of the pixel block of interest is determined by a prediction vector determination method in [prediction vector determination method and search range determination method] to be described later.
3 and the number of referenced adjacent pixel blocks and the motion vector 44 are output.

The search range determination unit 26 searches the prediction vector 33 output from the prediction vector generation unit 25 and the number of the reference pixel blocks and the motion vector 44 in a [prediction vector determination method and search range determination method] described later. The search range is determined based on the range determination method, and the search range [Wx
, Wy] 34 to the address generator 22.

[0039] [contour detection method] image of the contour (edge) detection method has been proposed, but the contour detection method differentiating the present embodiment (for example, "processing of an image"; Hajime Enomoto al Corona, published ^{PP .268} to ²⁷⁵ ), the Laplacian of image data g (x, y)） ² g (x, y) (=
G (x, y)).

At this time, the Laplacian operator (▽ ² )
as, That is, G (x, y) = g (x, y-1) + g (x-1, y) + g (x + 1, y) + g (x, y + 1) -4 · g (x, y) (5-1) or G (x, y) = g (x-1, y-1) + g (x, y-1) + g (x + 1, y-1) + g (x- 1, y) + g (x + 1, y) + g (x-1, y + 1) + g (x, y + 1) + g (x + 1, y + 1) -8 ・ g (x , y) (5-2) to determine G (x, y), and then | G (x, y) |>
If it is K, the point (x, y) is regarded as a contour point, and it is considered that a contour exists in the pixel block when the contour points in the pixel block are N points or more. Here, the threshold value K is a value depending on the number of gradations of the pixel, and N is a value determined by the resolution of the image and the size of the pixel block.

Further, the direction of a pixel determined to have an outline is further calculated, and the direction of the outline of the pixel block is determined as the direction of the outline of each pixel in the pixel block. Is the direction of the contour line of the pixel block.

At this time, the direction of the contour pixel (from 0 ° to 315)
For example, the following operator is used as an operator for calculating (8 directions at every 45 ° to 45 °).

Horizontal direction Vertical direction Right diagonal direction Left diagonal direction (0 °, 180 °) (90 °, 270 °) (45 °, 225 °) (135 °, 315 °) -1 -2 -1 -10 1 -1 -2 0 0 2 1 0 0 0 -2 0 2 -2 0 2 -2 0 2 1 2 1-1 0 1 0 2 1 -1 -2 0 Pixel block B as another embodiment of contour detection _i, the calculated mean mu _{i, j} and variance sigma _{i, j} ² of _j, the dispersion may exceeds the threshold value (C) (for i.e. σ _{i, j} ^2> C) or, the pixel block B _{i, When} the difference between the average value of _{j and} the average value of the adjacent pixel blocks B _i−1 , _j , B _i , _j−1 exceeds the threshold value (S) (that is, | μ _i , _j −μ _i , _j-1 |>
S or | μ _i , _j− μ _i−1 , _j |> S), it is considered that an outline exists in the pixel block. The threshold values C and S are determined according to the block size and the number of gradations of the image. For example, when the block size is 8 × 8 pixels and the number of gradations is 8 bits, C ≧ 100 and S ≧ Good results can be obtained by setting such as 10. Further, the threshold values K, N, C, and S can be updated every several pixel frames or every several pixel blocks.

[Reference Pixel Block Determination Method] As shown in FIG. 3, image data is divided into N × M pixel blocks, and a motion vector is sequentially detected from left to right and from top to bottom for each pixel block. In this case, assuming that the target pixel block is a, pixel blocks b and b adjacent above and to the left of a
c, d, and e are blocks in which a motion vector has already been detected. At this time, a pixel block to be referred to by the block a is determined by the following method.

(1) If no contour exists in the target pixel block a, a block having no contour in the adjacent pixel blocks b, c, d, and e is set as a reference block. However, if the reference block is only c or only e, the reference to the block is canceled, and if the reference block is only c and e, the reference to the block is canceled.

(2-1) When an outline exists in the target pixel block a, a pixel block having an outline among the adjacent pixel blocks b, c, d, and e is set as a reference pixel block.

(2-2) At this time, the target pixel block a
A pixel block in which the direction of the contour line coincides with the direction of the contour line of the pixel block having the contour among the adjacent pixel blocks b, c, d, and e, or a pixel block having a high degree of coincidence in the direction of the contour line (for example, 45 ° or 3 for 0 ° direction
15 °) can be used as a reference pixel block.

[Predicted Vector Determining Method and Search Range Determining Method] (1) When there is no pixel block to be referred to for the target pixel block a, the predicted vector of the pixel block a is set to (0, 0), The search range of the vector is set to the maximum search range [W ₀ , W ₀ ] (for example, W ₀ = [− 16, 1
6]).

(2) If there is only one pixel block to be referred to for the target pixel block a, the motion vector of the pixel block is set as the prediction vector of the pixel block a, and the search range is [W ₁ , W ₁ ] (For example, W ₁ =
[-8, 8]).

(3) If there are two or more pixel blocks to be referred to the target pixel block a, the average value (<x>, <y
>) Is the prediction vector of the pixel block a, and the difference S between the motion vector (x _i , y _i ) (i; reference block number) of each reference pixel block and each component of the prediction vector (<x>, <y>) _x and _Sy are obtained by the following equation (6).

[0051]

(Equation 2)

The search range of the motion vector is determined as follows according to the values of S _x and S _y .

(3-1) If S _x ∈W ₂ or S _y ∈W ₂ ,
The search range is [W ₂ , W ₂ ] (for example, W ₂ = [− 4,
4]).

(3-2) S _x ∈W ₁ or S other than the above (3-1)
For _y ∈W _1, a search range and [W _1, W _1].

(3-3) In cases other than the above (3-1) and (3-2), the search range is set to the maximum search range [W ₀ , W ₀ ].

The number and size of the cases for setting the search range can be changed as necessary.
It is not limited to the above.

[0057]

As described in detail above, according to the present invention, the relationship between a pixel block to be detected and its surrounding blocks, particularly, the characteristics of a contour portion which is erroneously detected by the conventional technique. Therefore, highly accurate motion vector detection can be expected. Further, by accurately predicting the motion vector, the motion vector can be detected with high accuracy even in a sufficiently small search range, so that the calculation time and the calculation amount can be significantly reduced. Further, according to the present invention, since the maximum detectable motion vector is not restricted by the search range, a large motion vector can be detected even in a limited search time or search ability. Can be expected.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a conventional motion vector detection device.

FIG. 3 is an explanatory diagram of a block scan method and a pixel block.

FIG. 4 is a diagram showing a relationship between a motion vector and a search range.

[Explanation of symbols]

 Reference Signs List 20 current frame memory 21 previous frame memory 22 address generation unit 23 motion vector discrimination unit 24 motion vector storage memory 25 prediction vector generation unit 26 search range determination unit 27 contour detection unit 28 contour information storage memory 29 input image signal 40 prediction error signal 41 Motion vector

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Yoko Harada 1-7-12 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd. (56) References JP-A-4-298184 (JP, A) (58) ) Surveyed field (Int.Cl. ⁷ , DB name) G06T 7/20 H04N 7/24

Claims (3)

(57) [Claims] 1. A current frame memory for storing current one frame of image data, and a previous frame memory for storing past one frame of image data.
(C) sequentially generating pixel addresses in a pixel block of a predetermined size with respect to the current frame memory, and predicting the predicted vector generation means within the search range determined by the search range determination means with respect to the previous frame memory. Address generation means for sequentially generating pixel addresses in the pixel block while shifting the pixel block of the predetermined size based on the prediction vector obtained from (d), and (d) whether an outline exists in the pixel block in the current frame memory. (E) a contour information storage memory for storing the contour information of each pixel block, and (f) a current frame. The pattern map between the image data of the current pixel block output from the memory and the image data of the comparison pixel block output from the previous frame memory (G) a motion vector storage memory for storing the motion vector for each pixel block in the current frame memory; Prediction vector generation means for comparing the outline information of the pixel block with the outline information of the peripheral pixel block to select a reference pixel block around the target pixel block, and to generate a prediction vector based on the motion vector of the reference pixel block; (I) search range determining means for determining a search range of a motion vector based on the degree of variation between the prediction vector and the motion vector of the reference pixel block and the number of the reference pixel blocks. Motion vector detection device. 2. The prediction vector generating means according to claim 1, wherein a pixel block having a contour or a pixel block having no contour is selected from adjacent pixel blocks adjacent to the pixel block of interest based on the presence or absence of a contour in the pixel block of interest. this was, then the feature vectors extracted from the motion vector of該選out the pixel block and the prediction vector of the target pixel block, in the absence of a corresponding pixel block in the adjacent pixel blocks to the prediction vector and (0, 0) The motion vector detecting device according to claim 1, wherein: 3. The predictive vector generating means according to claim 2, wherein when the contour exists in the pixel block of interest,
From an adjacent pixel block adjacent to the pixel block of interest,
3. The motion vector detecting device according to claim 2, wherein a pixel block having an outline and a high degree of coincidence in the outline direction is selected.