JP3001754B2 - Hierarchical motion vector detector - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical motion vector detecting device used for a high-efficiency video coding system.

[0002]

2. Description of the Related Art Interframe predictive coding is used as a moving image compression coding method. In order to reduce prediction errors in interframe predictive coding, a motion vector between frames is estimated in block units. A method of performing motion compensation is used. In this motion-compensated interframe coding method, a method of hierarchically detecting a motion vector using a reduced image obtained by reducing an original image is also used in order to obtain a motion vector more efficiently.

In a conventional hierarchical motion vector detection method, for example, a motion vector is detected as follows. Original image k
It is assumed that the image data is divided into blocks of pixel × j lines, and a motion vector is detected for each of those blocks.

The original image is horizontally and vertically reduced by m
₁, N₁1 / m using₁, 1 / n ₁(M₁, N₁Is any
Create an image reduced to a natural number. Reduction factor in horizontal and vertical directions from this reduced image
m_Two, N_Two1 / m using_Two, 1 / n_Two(M_Two, N_TwoIs
Create an image reduced to any natural number.

The above reduction operation is performed by reducing the reduction coefficients m _c and n
to _c c times after (c is any natural number) was repeated, the c
A motion vector is detected using a reduced image c that has been reduced twice as a detection unit of a block of k pixels × j lines.

A vector obtained by multiplying the detected motion vector by m _c and n _c in the horizontal and vertical directions (m _c and n _c are reduction coefficients of the reduced image c) is used as an initial value, and a corresponding m _c × n _c
The same initial value is given to all of the blocks, and a motion vector is detected by searching around the initial value on the reduced image (c-1).

[0007] Similarly, the reduced image (c-1) the motion vector detected on the horizontal, the _{m c-1, n c-} 1 multiplied by the vector in the vertical direction as an initial value, the corresponding m _c-1 × nc _-1
Then, a motion vector is detected by searching around the initial value on the reduced image (c-2).

The above operation is repeated until a motion vector is detected on the original image to obtain a final motion vector. In the conventional hierarchical motion vector detection method as described above, the size of a block, which is a detection unit on a reduced image, is the same as the detection unit for an original image.

[0009]

In the conventional method, when detecting a motion vector, a block having the same size as a block of an original image is used for a motion vector search in a reduced image. Since the block size is fixed even when the image is reduced, the proportion of the block in the image increases as the reduction ratio increases. As a result, there is a high probability that one block contains objects with multiple different motions, but since only one motion vector is given for one block, it is necessary to detect the correct motion for each object. In many cases.

In the case where the block size is fixed, one block in the reduced image corresponds to a plurality of blocks in the upper hierarchical image (the original image at the time of reduction). For example, one block in an image reduced to the size of 1 / m _i and 1 / n _i respectively in the horizontal and vertical directions with respect to the upper layer image corresponds to _mi × n _i blocks in the upper layer image. Hierarchical Tekido vector detection, the motion vector Motoma' in the reduced image horizontally, to the vertical direction m _i, is n _i times, used as an initial value in the motion vector detection in the upper-layer image. The upper layer searches around this initial value. In the conventional method, since the same initial value is given to all the _mi × _ni blocks, if the search range in the upper layer is not sufficient, only a uniform motion different from the actual motion is detected at the boundary of the moving object. This condition can be avoided by widening the search range in the upper hierarchy, but this increases the amount of calculation.

An object of the present invention is to solve the above problems, even if it contains a plurality of motion in the input image, detecting a motion vector that reflects the actual motion without increasing the calculation amount, a motion-compensated frame An object of the present invention is to provide a hierarchical motion vector detection device capable of improving the coding efficiency of an inter-coding method.

[0012]

In the conventional hierarchical motion vector detection method, a reduced image is created from a luminance signal of an input image, and when a motion vector is obtained on each reduced image, the original image is also used in the reduced image. In contrast to using blocks of the same size as the blocks, the present invention is characterized in that block matching is performed using blocks of a size proportional to the image size. When the block in the original image is k pixels × j lines, the original image is horizontally and vertically divided by 1 / m and 1 / n.
In the image reduced to the size of, the block size is k /
The size is reduced to m pixels × j / n lines. In particular, books
The hierarchical motion vector detection device of the present invention
A plurality of frames having a creating unit, a memory, and a small area dividing unit
Consists of a frame reduction unit and one matching error calculation unit
The input frame and the reference frame
The reduced image of the entire frame corresponding to each layer once
The amount of calculation for creating a reduced image is
Be reduced.

[0013]

According to the present invention, in order to reduce the possibility of detecting a motion different from the actual motion due to the effect of including a plurality of different motions in the block in the hierarchical motion vector search, the matching block is also used in the reduced image. Use the same size as the reduction rate. FIG. 1 is a diagram showing the principle.

Assume that the original image 100 is composed of a × b pixels, which is divided into blocks 101 of k pixels × j lines, and a motion vector is detected for each of these blocks. The reduced image creation unit 200 divides the original image in the horizontal and vertical directions by a size of 1 / m and 1 / n, that is, (a / m)
When the size is reduced to × (b / n), the small area dividing unit 400 sets the size of a block serving as a detection unit of a motion vector in the reduced image 300 to k / m pixels × j / n lines, and sets a block of this size. For the matching error calculation. The same applies to the case where the operation of reducing the image is repeated a plurality of times.

As described above, one of the original images
The block corresponds to one block of the reduced image, and even when the input image includes a plurality of different motions, a motion vector reflecting the actual motion is detected.

[0016]

FIG. 2 is a diagram showing an example of a motion vector detection circuit embodying the present invention. In FIG. 2, 1 is an image signal of the current frame, 2 is a frame memory, 3 is an original image of the current frame, 4 is a reduced image creating unit, 5 is a first reduced image of the current frame, 6 is a frame memory, and 7 is a current frame. A first reduced image, 8 is a reduced image creating unit, 9 is a second reduced image of the current frame, 10 is a frame memory, 11 is an image signal of a reference frame, 12 is a frame memory, 13 is a reference frame original image, and 14 is a reduced image The creation unit, 15 is the first reduced image of the reference frame, 16 is the frame memory, 17 is the first reduced image of the reference frame, 18 is the reduced image creation unit, 19 is the second reduced image of the reference frame, and 20 is the frame memory.

Reference numeral 21 denotes an address signal for reading the current frame second reduced image, 22 denotes a current frame second reduced image, 23 denotes an address signal for reading the current frame first reduced image, 24 denotes a current frame first reduced image, 25 Is an address signal for reading the original image of the current frame, 26 is an original image of the current frame, 27 is an address signal for reading the original image of the reference frame, 28 is an original image of the reference frame, 29
Is an address signal for reading the first reduced image of the reference frame,
Reference numeral 30 denotes a reference frame first reduced image, reference numeral 31 denotes an address signal for reading the reference frame second reduced image, and reference numeral 32 denotes a reference frame second reduced image.

Reference numeral 33 denotes a small area divider for the current frame second reduced image; 34, a small area divider for the current frame first reduced image; 35, a small area divider for the current frame original image;
36 is a small area divider for the reference frame original image, 37 is a small area divider for the reference frame first reduced image, 38 is a small area divider for the reference frame second reduced image, and 39 is a block of the current frame second reduced image Signal, 40 is the first frame
A block signal of the reduced image, 41 is a block signal of the current frame original image, 42 is a block signal of the reference frame original image, 43 is a block signal of the first reduced image of the reference frame,
Reference numeral 44 denotes a block signal of the second reduced image of the reference frame.

Reference numeral 45 denotes a matching error calculation unit;
Is a difference value, 47 is a comparator, 48 is a memory for storing the minimum block difference value, 49 is address data, 50 is an address memory, 51 is address data (a motion vector in a reduced image), 60 is a first current A frame reduction unit, 61 represents a second current frame reduction unit, 62 represents a first reference frame reduction unit, and 63 represents a second reference frame reduction unit.

The image 1 of the current frame is a frame memory
2 is stored. The reduced image creation unit 4 includes a frame memory
1 / m in the horizontal and vertical directions from the image 3 stored in₁,
1 / n₁Create an image 5 reduced to the size of
The data is stored in the memory 6. The reduced image creation unit 8
Image 7 in horizontal and vertical directions by 1 / m_Two, 1 / n _Two
, And store it in the frame memory 10. reference
The image 11 of the frame is the same as the image of the current frame.
After being stored in the frame memory 12, the reduced image creation unit 14
Are reduced and stored in the frame memory 16.
After that, the image is further reduced by the reduced image creation unit 18 and the frame memo is written.
The storage in the repository 20 is performed.

The reduced image 22 of the current frame stored in the frame memory 10 is read with the value of the address signal 21 consisting of two coordinates (x, y) as an offset value, and is input to the small area divider 33. The small area divider 33 converts the current frame reduced image 22 into {k / (m ₁ × m ₂ ) pixels}
× {j / (n ₁ × n ₂ ) line}
The current frame block signal 39 is output. Similarly, the reduced image 32 of the reference frame is divided into blocks by the small area divider 38 and output as a reference frame block signal 44. The current frame block signal 39 and the reference frame block signal 44 are input to the matching error calculator 45.

The matching error calculator 45 outputs a difference value 46 between the blocks to a comparator 47. Comparator 47
Has a minimum value memory 48, and the input difference value 4
If 6 is smaller than the value stored in the memory 48, the value in the memory 48 is updated, and the address 49 including the two coordinates (x, y) is recorded in the address memory 50. After the above operation is completed in the search range of the motion vector in the reduced image, the motion vector on the reduced image is stored in the address memory 50.

Next, the reduced image 24 of the current frame stored in the frame memory 6 is input to the small area divider 34. The small area divider 34 divides the current frame reduced image 24 into blocks of k / m ₁ pixel × j / n ₁ lines and outputs a current frame block signal 40. Similarly, the reference frame reduced image 30 is input to the small area divider 37, but the motion vector detection excluding the smallest size reduced image uses the motion vector detected in the upper layer as an initial value.
Therefore, the offset value to be read at this time needs to be shifted by a motion vector in the reduced image. The address signal 29 is obtained by adding a value obtained by multiplying the (x, y) output value (address data 51) from the address memory 50 by m ₂ and n ₂ in the horizontal and vertical directions. The reference frame reduced image 30 read in accordance with the address signal 29 is
7 and output as a reference frame block signal 43. Similar to the case of the reduced image, the difference value calculation and the comparison are performed.

The current frame stored in the frame memory 2
The original image 26 is input to the small area divider 35. Subordinate
The region divider 35 converts the current frame original image 26 into k × j blocks.
Split into locks and output current frame block signal 41
I do. The reference frame original image 28 is stored in the address memory 50.
(X, y) output value 51 from the horizontal and vertical₁, N ₁
Divide the small area by adding the multiplied value as the offset value
Input to the frame unit 36, the reference frame block
It is output as a signal 42. The difference is the same as for reduced images.
The value calculation and comparison are performed.

When the above operation for the search range is completed,
The detected motion vectors are stored in the address memory 50. FIG. 3 is a diagram showing an example of another motion vector detection circuit embodying the present invention.

Although the embodiment shown in FIG. 2 is an example in which the number of layers is three, as shown in FIG.
0 and the current frame reduction section 61, the reduced image creation section 7
1, a frame memory 72, a current frame reduction unit 70 composed of a small area divider 73, and a reduced image creation unit 81, a frame memory 82, a small area By adding the reference frame reduction unit 80 including the divider 83, it is possible to deal with the case where the number of layers is four.

The number of hierarchies is not limited to three, and the number of hierarchies is not limited to four. Similarly, the current frame reduction unit is provided between the output of the frame memory 6 and the input of the reduced image creation unit 8, and the reference frame reduction unit is provided in the frame memory 16. Between the output and the reduced image creation unit 18,
By connecting each (the number of layers-3), it is possible to cope with an arbitrary number of layers of 3 or more.

When the number of layers is two, the current frame reduction section 60 is deleted in FIG. 2 and the current frame original image 3 output from the frame memory 2 is used as an input to the reduced image generation section 8. , The reference frame reduction unit 62 is also deleted, and the reference frame original image 13 output from the frame memory 12 is used as the input to the reduced image generation unit 18.

[0029]

As described above, according to the present invention,
When hierarchical motion vector search is used in motion compensation inter-frame coding, accurate motion vector estimation can be performed for an image including a plurality of different motions without increasing the calculation amount. This improves the coding efficiency at which motion-compensated inter-frame prediction is effective.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a diagram illustrating an example of a motion vector detection circuit embodying the present invention.

FIG. 3 is a diagram showing an example of another motion vector detection circuit embodying the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 original image 101 detection unit block 200 reduced image creation unit 300 reduced image 301 detection unit block 400 small area division unit

Continuation of the front page (56) References JP-A-2-33286 (JP, A) JP-A-4-177992 (JP, A) JP-A-7-87495 (JP, A) JP-A-2-224490 (JP) JP-A-3-4686 (JP, A) JP-A-3-117,991 (JP, A) JP-A-5-501188 (JP, A) (58) Fields studied (Int. Cl. ⁷ , DB Name) H04N 7/24-7/68

Claims (1)

(57) [Claims] [Claim 1] by dividing the original image into blocks of k pixels × j lines, instrumentation for detecting a motion vector for each the blocks
Where the reduced image creation unit and the reduced image are stored , respectively.
Memory and small area for dividing the reduced image into small areas
From 1 to c (c is an integer of 2 or more)
A plurality of frame reduction units at the same time and a current reduction ratio of the same reduction degree output by the plurality of frame reduction units.
Match on small area of frame and reference frame
A matching error calculator for calculating the matching error.
For example, the reduced image creation unit in the first frame reduction section,
The original image is reduced in the horizontal and vertical directions to 1 / m ₁ , 1 / n ₁ (m ₁ and n ₁ are arbitrary natural numbers) using reduction coefficients m ₁ and n _1, and the number of pixels is reduced to the number of pixels of the original image. Of 1 / (m ₁ × n ₁ )
The first frame reduction unit.
Stored in a memory, and the i-th (i is a repetition of 2 to c) frame compression
The reduced image creation unit in the small part is the (i-1) -th file.
From the reduced image reduced by the frame reduction part,
1 / m _i , 1 / n using the reduction coefficients m _i and n _i in the vertical direction
_i (m _i and n _i are arbitrary natural numbers)
(I-1) of the reduced image reduced by the frame reduction unit
Create an image with 1 / (m _i × n _i ) of the number of pixels
The data is stored in the memory of the i-th frame reduction unit, and is stored in each of the first to c-th frame reduction units.
The area dividing unit is stored in the memory in each frame reducing unit.
From the reduced image d (d is an integer satisfying 1 ≦ d ≦ c),
Reduction factor m ₁ to obtain a reduced image, m _2, ..., a product of m _d M _d
And n _1, n _2, ..., using the product N _d of n _d, k / M _d picture
A small area having a size of element × j / N _d line is cut out and
Output to the matching error calculation unit, and the matching error calculation unit outputs the c-th frame.
In the reduced image c reduced by the reduction unit, the block of the detection unit is
The reduction factor m _1, m _2, ..., a product of m _c M _c and n _1, n
_2, ..., using the product N _c of n _c, k / M _c pixels × j / N
_A motion vector is detected by setting the size to the _c- line size,
Multiply the output motion vector by m _c and n _c in the horizontal and vertical directions
(C-1) reduced images with the vector
Around the initial value in (c-1) , the block of the detection unit
The product M of the reduction coefficients m ₁ , m ₂ ,..., M _(c-1)
Using the product N _(c-1) of _(c-1) and n ₁ , n ₂ ,..., n _(c-1)
The size of k / M _(c-1) pixels x j / N _(c-1) lines
Then, the motion vector is detected and the reduced image (c-
1) The motion vector detected above is set to m in the horizontal and vertical directions.
_(c-1) , n _(c-1) times the vector as the initial value,
(C-2) times reduced image (c-2)
The operation of detecting the motion vector around the initial value in
Repeat until the motion vector is detected on the original image.
A hierarchical motion vector detection device configured to detect a vector.