JP2000078583A - Device for estimating adaptive movement - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a movement estimating device for adaptively deciding a movement vector through the use of a reduced calculating quantity by supplying the movement vector of each location block through the use of outputs from a boundary match movement estimating means and a block match movement estimating means based on a control signal. SOLUTION: A block match ME part 420 and a boundary match ME part 430 receive location area data from a location area forming part 410, location block data from a location block forming part and a control signal from an ME mode selecting part 440. When the block match ME part 420 is enabled by the first level control signal, the movement vector corresponding to the location block is supplied to MUX 450. When the boundary match ME part 430 is enabled by the second level control signal, a boundary match process is executed as against the location block and the movement vector corresponding to the location block is supplied to MUX 450.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion estimation apparatus, and more particularly to a motion estimation apparatus that adaptively determines a motion vector with a reduced amount of calculation.

[0002]

2. Description of the Related Art In digital television systems such as videophones, teleconferencing and high definition television, each video line signal in a video frame signal consists of a series of digital data called "pixels". A large amount of digital data is required to represent a frame signal. However, the available frequency bandwidth on conventional transmission channels is limited, so transmitting large amounts of digital data over that channel requires, among other things, low bit rate video such as video telephony and teleconferencing. For signal encoders, various data compression techniques must be used to compress or reduce the amount of data to be transmitted.

In a low bit rate video signal encoding system, one of the methods for encoding a video signal is a so-called object-oriented analysis / synthesis encoding method (O).
object-Oriented Analysis-S
ynthesis coding technology
e). According to this method, an input video signal is divided into a plurality of objects, and three sets of parameters defining the motion, contour, and pixel data of each object are handled through different encoding channels.

As an example of such an object-oriented analysis / synthesis coding method, a so-called MPEG-4 (Moving P
icture Expert Group phase
-4), but MPEG-4 is a content-based interactivity in applications such as low bit rate communications, interactive multimedia (eg, games, interactive television, etc.) and area surveillance. Provide audiovisual coding standards that allow for improved coding efficiency and / or universal accessibility (e.g., MPEG-
4 Video Verification Mode
l Version 7.0, International
al Organization for Stand
ardization, ISO / IEC JTC1 /
SC29 / WG11 MPEG97 / N1642, A
pril 1997).

According to MPEG-4, an input video image is composed of a plurality of video object planes (VOPs) corresponding to entities in a bit stream that can be accessed and operated by a user.
Divided into A VOP is also defined as an object and may be represented by a rectangle whose width and height are the smallest multiple of 16 pixels (the size of a macroblock) surrounding each object. Therefore, the encoder handles the input video image on a VOP basis, that is, on an object basis.

The VOP disclosed in MPEG-4 is called VO
P has shape information and texture information for the object in P, and the object is, for example, 16 × 16 on the VOP.
It is represented by a plurality of macroblocks composed of pixels, the shape information is represented by a binary shape signal, and the texture information is composed of luminance data and chrominance data.

[0007] Since texture information for two input video images that are input in sequence has a temporal redundancy, in order to effectively encode the texture information, a motion estimation and compensation technique is used. It is necessary to remove the temporal redundancy.

[0008] To perform motion estimation and compensation, and discrete cosine transform, a reference VOP (eg, a previous VOP) is a progressive video padding technique.
image padding technology
e), ie, the conventional iterative padding technique (repet
It should be padded by an active padding technique. Conventional iterative padding techniques typically fill a transparent area outside the object of the VOP by repeating boundary pixels located at the contour of the object. If the transparent pixels in the transparent area outside the object can be filled by one or more boundary pixel repetitions, the repetition values of the boundary pixels are averaged and taken as padding values. The progressive padding process usually has three stages: horizontal iterative padding,
It is divided into vertical repeat padding and outer padding (see MPEG-4 VM 7.0).

On the other hand, according to the conventional block matching algorithm, the current VOP is divided into a plurality of search blocks.
Typically, the size of the search block is from 8 × 8 pixels to 32
× 32 pixels. To determine a motion vector for a search block in the current VOP, a search block is used.
In the previous VOP, the similarity between each of a plurality of candidate blocks of the same size included in a generally larger search area is measured. To calculate the similarity between the search block and each candidate block, an error function such as a mean absolute error or a mean square error is used. Here, the motion vector represents a displacement between the search block and the candidate block that yields the minimum error function.

However, in the conventional block matching algorithm, since the error function is calculated for all pixels in one search block, the calculation process is very complicated. In order to effectively implement real-time processing, it is necessary to reduce computational complexity generated by a conventional block matching algorithm.

[0011]

SUMMARY OF THE INVENTION Accordingly, it is a primary object of the present invention to provide a motion estimating apparatus for adaptively determining a motion vector with a reduced amount of calculation.

[0012]

According to the present invention, there is provided an encoder for supplying texture data of a current frame, texture data of a previous frame and shape data of the current frame. A motion estimating device for estimating a motion between the previous frame of the video signal and the current frame, wherein the dividing unit divides the current frame into a plurality of search blocks of the same size; A search area forming means for forming a plurality of search areas corresponding to one search block; and, based on the shape data of the current frame, whether each search block is a boundary block having a background pixel and an object pixel,
A search block determining unit that determines whether the block is an internal block having only object pixels and supplies a control signal; a boundary matching motion estimating unit that performs a boundary matching motion estimation based on the control signal; A block matching motion estimating means for performing block matching motion estimation based on the signal, and a motion vector of each search block, And a motion vector supply unit for supplying the motion vector.

[0013]

Preferred embodiments of the present invention will be described below in detail with reference to the drawings.

Referring to FIG. 1, a block diagram of an encoding device 1 according to the present invention is shown.

The texture data of the current frame having one or more current VOPs is supplied to a search block forming unit 100. The search block forming unit 100 divides the current VOP in the current frame into a plurality of search blocks of the same size (for example, 16 × 16 pixels),
The search block data is subtracted from the subtraction unit 200 and the motion estimation (M
E) Supply each to the section 400.

The ME unit 400 receives the texture data of the previous frame having one or more previous VOPs from the frame memory 300. Here, the boundary block in the previous frame is padded by the padding unit 1100 after being reconstructed by the adding unit 1000.

After that, the ME unit 400 generates a plurality of search areas corresponding to one search block in the previous VOP. On the other hand, the ME unit 400 receives the shape data of the current frame. Here, each pixel in the current frame has a label indicating a region to which each pixel belongs. For example, the background pixel is displayed as “0”, and the object pixel is displayed as a value other than “0”.

The ME unit 400 determines a motion vector corresponding to a search block by performing adaptive motion estimation on each search block. That is, when the search block is a boundary block, a motion vector corresponding to the search block is determined by a conventional block matching motion estimation technique, and when the search block is an inner block, it corresponds to the search block. The motion vector is
It is determined by a boundary matching motion estimation technique. Here, the boundary block is composed of pixels belonging to the background and the object,
The inner block is composed of only pixels belonging to the object.
The detailed motion estimation technique according to the present invention will be described later with reference to FIGS. The motion vector is calculated by a motion compensation (MC) unit 5.
00 and a transmitter (not shown).

The MC unit 500 receives the motion vector from the ME unit 400 and receives the pixel data of the corresponding optimal candidate block from the frame memory 300. Then M
The C unit 500 performs a motion compensation process on each optimal candidate block using the corresponding motion vector to generate a motion-compensated optimal candidate block. Respectively.

The subtraction unit 200 subtracts the optimal motion-compensated candidate block from the corresponding search block, and supplies the result of the subtraction (ie, an error signal) to the padding unit 600. This padding unit 600 receives the shape data of the current frame. The error signal is padded with shape data of the current frame based on a conventional macroblock-based padding technique. The padding error signal is supplied to a discrete cosine transform and quantization (DCT & Q) unit 700.

The DCT & Q section 700 performs a discrete cosine transform and quantization on the padded error signal, and outputs a set of quantized discrete cosine transform coefficients to the statistical encoding section 800 and the inverse discrete It is supplied to a cosine transform and inverse quantization (IDCT & IQ) section 900, respectively. The statistical encoding unit 800 statistically encodes the quantized set of discrete cosine transform coefficients, and supplies a statistically encoded signal to the transmitter.

IDCT & IQ section 900 performs inverse discrete cosine transform and inverse quantization on the quantized set of discrete cosine transform coefficients, and supplies the restored error signal to adder 1000. The adding section 1000 adds the restoration error signal and the motion-compensated optimal candidate block from the MC section 500 to generate a reconstructed search block (reconstruction search block).

The padding unit 1100 receives the reconstructed search block and the shape data of the current frame, and pads the reconstructed search block based on a conventional macroblock-based padding technique. This padded and reconstructed search block is stored in the frame memory 3 as texture data of the previous frame for the next frame.
00 is stored.

Referring to FIG. 2, ME section 4 shown in FIG.
A detailed block diagram of 00 is shown. ME section 400
Are the search area forming unit 410 and the block matching motion estimation (M
E) unit 420, boundary matching motion estimation (ME) unit 430, motion estimation (ME) mode selection unit 440, and MUX 450.

The texture data of the previous frame is supplied from the frame memory 300 to the search area forming section 410. The search area forming unit 410 defines a search area corresponding to a search block having a certain size, shape, and search pattern. As a result, motion estimation for the search block is performed. After the search area is determined by the search area forming section 410, the search area data is stored in the block matching ME section 4.
20 and the boundary matching ME unit 430.

On the other hand, the shape data of the current frame is supplied to the ME mode selector 440. ME mode selection unit 44
0 determines whether the search block is a boundary block or an internal block based on the shape data of the current frame. That is, if the area in the shape data of the current frame is at the same position as the search block and is the same size as the search block having the boundary pixels and the object pixels, the search block is determined as a boundary block,
If the region has only object pixels, the search block is determined as an inner block.

When the search block is determined as a boundary block, the ME mode selection unit 440 supplies the first level control signal to the block matching ME unit 420, the boundary matching ME unit 430, and the MUX 450, respectively. The first level control signal enables the block matching ME unit 420, disables the boundary matching ME unit 430,
The input supplied from the block matching ME unit 420 is selected as the MUX 450 and supplied to the MC unit 500 and the transmitter. When the search block is determined as an internal block, the ME mode selection unit 440 supplies the second level control signal to the block matching ME unit 420, the boundary matching ME unit 430, and the MUX 450, respectively. The second level control signal enables the boundary matching ME unit 430, disables the block matching ME unit 420, sets the MUX 450 to the input supplied from the boundary matching ME unit 430, and selects the MC unit 500 and Each is supplied to the transmitter.

Block matching ME section 420 and boundary matching M
E section 430 receives search area data from search area forming section 410, search block data from search block forming section 100, and a control signal from ME mode selecting section 440. When the block matching ME unit 420 is enabled by the first level control signal, a conventional block matching process is performed on the search block, and a motion vector corresponding to the search block is supplied to the MUX 450, and the boundary matching is performed. When the ME unit 430 is enabled by the second level control signal, the search unit performs a boundary matching process on the search block and supplies a motion vector corresponding to the search block to the MUX 450. Detailed operations of the block matching ME unit 420 and the boundary matching ME unit 430 will be described with reference to FIGS. 3 and 4, respectively.

Referring to FIG. 3, the block matching ME unit 4
A detailed block diagram of 20 is shown. The candidate block forming unit 422 enabled by the first level control signal from the ME mode selecting unit 440 forms a plurality of (for example, M) candidate blocks of the same size in the search area. Here, the search area data is supplied from the search area forming unit 410 in FIG. 2, the size of the candidate block is the same as the size of the search block, and M is a positive integer representing the total number of formed candidate blocks. It is. The pixel data of the i-th candidate block is supplied to the i-th block matching error function calculator 424-i (i is a positive integer having a range of 1 to M).

Further, the candidate block forming section 422 determines a displacement vector (ie, DV _{1 to} DV _M ) from the search block to each candidate block, and supplies it to the first selecting section 428. Then, the search block data is supplied to the block matching error function calculation units 424-1 to 424-M, respectively. Here, only three blocks are shown for convenience of explanation. i-th block matching error function calculator 424-i
In, a block matching error function (eg, mean absolute error or mean square error) between the search block and the i-th candidate block is calculated. The block matching error function is, for example, the average absolute error or the mean square error between the object pixel in the search block and the corresponding pixel in the i-th candidate block. Here, the object pixel in the search block is determined based on the shape data of the current frame. The calculated error function is calculated by the first comparing unit 426.
Supplied to

The first comparing section 426 compares the block matching error functions with each other and selects a minimum block matching error function from the error functions, thereby generating a first instruction signal representing the corresponding minimum block matching error function. First selection unit 42
8 The first selector 428 supplies a displacement vector corresponding to the minimum block matching error function to the MUX 450 as a motion vector of the search block according to the first instruction signal.

FIG. 4 is a detailed block diagram of the boundary matching ME unit 430 in FIG. Since adjacent pixel values in a video signal are very highly related to each other, a boundary matching technique is useful for internal blocks.

The extended candidate block forming section 432 enabled by the second level control signal from the ME mode selecting section 440 has a plurality (for example, M
) Extended candidate blocks of the same size. Here, the search area data is supplied from the search area forming unit 410 in FIG. 2, and the extended candidate block includes a candidate block and a boundary having a size of one pixel around the candidate block. That is, since the size of the search block and the extension block is 16 × 16 pixels, the size of the extended candidate block is 18 × 18 pixels.

Further, the expanded block matching ME unit 43
2 determines a displacement vector (ie, DV _{1 to} DV _M ) from the search block to each extended candidate block, and supplies the displacement vector to the second selection unit 438. Then, the search block data is stored in each of the boundary matching error function calculation units 434-1 to 434-.
M. Here, only three blocks are shown for convenience of explanation. Each boundary matching error function calculator 434-1
In ４３434-M, the boundary matching error function is calculated. The process of calculating the boundary matching error function will be described with reference to FIG.

As shown in FIG. 5, the search block SB overlaps with the extended candidate block so that the position of the search block and the position of the candidate block included in the extended candidate block completely match. Become. In the embodiment of the present invention, the search block SB has 16 × 16 pixels, but only 16 pixels are shown for convenience.

Thereafter, according to the preferred embodiment of the present invention, the difference between the pixel value of the search block and the pixel value of the expanded candidate block is detected for the top row and the leftmost column, and the boundary matching error function is It is calculated as follows.

[0037]

(Equation 1)

Here, E: the boundary matching error function between the search block and the extended candidate block P _{1 , i} : the pixel value of the i-th pixel in the top row of the search block T _i : the extended candidate block pixel values P _i, the i th pixel in the top row of _1: pixel value L _i of the i-th pixel in the leftmost column of a search block: the pixel value of the i th pixel in the leftmost column of the packed and extended candidate blocks According to another embodiment of the present invention, a difference between the pixel value of the search block and the pixel value of the expanded candidate block is detected for the top row, the bottom row, the leftmost column and the rightmost column, Boundary matching error function becomes more accurate. The boundary matching error function is
It is calculated as follows.

[0039]

(Equation 2)

Here, P _{16 , i} : the pixel value of the _ith pixel in the bottom row of the search block B _i : the pixel value of the _ith pixel in the bottom row of the expanded candidate block P _{i , 16} : the search block The pixel value of the _ith pixel in the rightmost column of R _i : the pixel value of the _ith pixel in the rightmost column of the expanded candidate block The calculated boundary matching error function is supplied to the second comparing unit 436 .

The second comparing section 436 compares the respective boundary matching error functions with each other, selects a minimum boundary matching error function among the boundary matching error functions, and then generates a second instruction representing the minimum boundary matching error function. The signal is supplied to the second selector 438. In this case, the optimal candidate block is the candidate block in the extended candidate block corresponding to the minimum boundary matching error function. The second selector 438 supplies a displacement vector corresponding to the minimum boundary matching error function to the MUX 450 as a motion vector of the search block according to the second instruction signal.

As mentioned above, the process of block matching or boundary matching is applied selectively to each search block, thereby significantly reducing the computational complexity. For example, if the size of the search block is 16 × 16, the difference is calculated for 16 × 16 pairs of pixels in a block matching technique. However, in the boundary matching technique according to the first embodiment of the present invention, the difference is calculated only for 16 × 2 pairs of pixels, and the calculation amount is reduced by ８.

Although the preferred embodiments of the present invention have been described above, those skilled in the art will be able to make various modifications without departing from the scope of the present invention.

[0044]

Thus, according to the present invention, block matching or boundary matching can be selectively applied to each search block, and the complexity of the calculation of the motion vector can be further reduced. Can be.

[Brief description of the drawings]

FIG. 1 is a block diagram of an encoding device according to the present invention.

FIG. 2 is a detailed block diagram of a motion estimator in FIG. 1;

FIG. 3 is a detailed block diagram of a block matching motion estimator in FIG. 2;

FIG. 4 is a detailed block diagram of a boundary matching motion estimator in FIG. 2;

FIG. 5 is a schematic diagram illustrating a boundary matching process according to a preferred embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 100 Search block formation part 200 Subtraction part 300 Frame memory 400 Motion estimation (ME) part 410 Search area formation part 420 Block matching motion estimation (ME) part 422 Candidate block formation part 424-1 to 424-M Error function calculation part 426 1 comparison unit 428 first selection unit 430 boundary matching motion estimation (ME) unit 432 extended block matching ME unit 434-1 to 434-M error function calculation unit 436 second comparison unit 438 second selection unit 440 motion estimation (ME) ) Mode selection unit 450 MUX 500 Motion compensation (MC) unit 600, 1100 Padding unit 700 DCT & Q unit 800 Statistical coding unit 900 IDCT & IQ unit 1000 Addition unit

Claims (9)

[Claims] 1. An encoder to which texture data of a current frame, texture data of a previous frame and shape data of the current frame are supplied, and estimates a motion between the previous frame and the current frame of a video signal. A motion estimating device, comprising: a dividing unit configured to divide the current frame into a plurality of search blocks of the same size; and a search area forming a plurality of search areas corresponding to one search block in the previous frame. Means, based on the shape data of the current frame, determines whether each search block is a boundary block having a background pixel and an object pixel, or an internal block having only an object pixel, and outputs a control signal. A search block determining means to be supplied; a boundary matching motion estimating means for performing a boundary matching motion estimation based on the control signal; A block matching motion estimating means for performing block matching motion estimation based on a control signal; and a motion vector of each search block using outputs from the boundary matching motion estimating means and block matching motion estimating means based on the control signal. And a motion vector supply unit for supplying a motion vector. 2. The search block determining means generates a first level control signal when the search block is an internal block, and generates a second level control signal when the search block is a boundary block. The motion estimation device according to claim 1, wherein 3. The control signal of the first level enables the boundary matching motion estimating means, disables the block matching motion estimating means, and sets the motion vector supply means as the motion vector supplying means. And the second level control signal comprises:
2. The method according to claim 1, further comprising: disabling said boundary matching motion estimating means, enabling said block matching motion estimating means, and supplying an output from said block matching motion estimating means as said motion vector supplying means. 3. The motion estimation device according to 2. 4. The method according to claim 1, wherein the boundary matching motion estimating means includes: a search block corresponding to the search block having the same size as the candidate block; Extended candidate block forming means for forming a plurality of extended candidate blocks composed of a boundary having: a displacement from each of the extended candidate blocks to the search block; and a displacement of each of the extended candidate blocks. Displacement supply means for supplying as a vector, boundary matching error function calculating means for calculating a boundary matching error function between the search block and the corresponding extended candidate block, and comparing each boundary matching error function with each other, A minimum error function selecting means for selecting a minimum boundary matching error function; and a displacement vector corresponding to the minimum boundary matching error function, Motion estimation apparatus as claimed in claim 3, characterized in that it comprises a displacement vector supply means for supplying as can vector. 5. The minimum boundary matching error function calculating means expands the search block so that the position of the search block completely matches the position of the candidate block included in the expanded candidate block. Overlapping means for superimposing on the candidate block, difference calculating means for calculating a difference between a pixel of a predetermined area in the search block and a corresponding pixel of the corresponding area in the expanded candidate block, 5. The motion estimation apparatus according to claim 4, further comprising: an error function supply unit that sums the differences and supplies an error function between the search block and the extended candidate block. 6. The predetermined area in the search block is a top row and a leftmost column of the search block, and a corresponding area in the expanded candidate block is a top row and a top row of the expanded candidate block. The motion estimating device according to claim 5, wherein the left and right columns are arranged. 7. The predetermined area in the search block is an uppermost row, a lowermost row, a leftmost column, and a rightmost column of the search block, and a corresponding area in the extended candidate block is the extended area. The motion estimation device according to claim 5, wherein the top row, the bottom row, the leftmost column, and the rightmost column of the completed candidate blocks. 8. The block matching motion estimating means includes: a candidate block forming means for forming a plurality of candidate blocks in a search area corresponding to the search block; and a displacement from each candidate block to the search block. A displacement vector generating means for generating a displacement vector of the candidate block; an error function calculating means for calculating a block matching error function between the search block and the corresponding candidate block; 6. A minimum block matching error function selecting means for selecting a block matching error function, and a displacement vector supplying means for supplying a displacement vector corresponding to the minimum block matching error function as a motion vector for the search. The motion estimation device according to item 1. 9. The block matching error function calculating means, an overlapping means for overlapping the search block with a candidate block, and a difference calculating means for calculating a difference between an object pixel in the search block and a corresponding pixel in the candidate block. And a difference sum means for summing the calculated differences and supplying the sum as a block matching error function between the search block and the expanded candidate block. Motion estimation device.