KR0185842B1 - A motion estimator for a video coder - Google Patents

Abstract

The present invention relates to a motion estimation apparatus in an image encoder, the apparatus comprising: a demultiplexer (30) for separating two image data in units of frames; First and second frame memories 32a and 32b for storing the divided forward and reverse reference frames; A multiplexer (34) for separating and storing the stored image data into forward and reverse data; First and second motion estimation units 36a and 36b which receive the forward and reverse data and perform motion estimation; First and second search region memories 38a and 38b that store data corresponding to a search region from the forward and reverse data and receive motion vectors from the two motion estimation units 36a and 36b; First and second delay units 40a and 40b for delaying the forward and reverse motion estimation results; A third delay unit 40c delaying the current frame; An interpolation mode error calculator (42) which averages forward and reverse image data obtained from the two search area memories (38a, 38b) and calculates a difference from the delayed current frame data; And a mode selector 44 which selects the most similar to the data of the current frame among the forward, reverse, and interpolation modes and outputs a motion vector of the selected mode. The two search area memories 38a and 38b are interpolated to the interpolation mode. By using the dedicated memory for calculating the error value for the error, the operation of the two frame memories (32a, 32b) does not have to speed up the implementation of the image encoder is easier and can reduce the cost is an advantage.

Description

Motion Estimator in Image Encoder

1 is a block diagram of a motion estimation apparatus in a conventional image encoder.

2 is a block diagram of a motion estimation apparatus in an image encoder of the present invention.

* Explanation of symbols for main parts of the drawings

30: demultiplexer 32a: first frame memory

32b: second frame memory 34: multiplexer

36a: first motion estimation unit 36b: second motion estimation unit

38a: first search area memory 38b: second search area memory

40a: first delay unit 40b: second delay unit

40c: third delay unit 42: interpolation board error calculator

44: mode selector

The present invention relates to a motion estimation apparatus, and more particularly, to a motion estimation apparatus configured to separately use a memory of a search area capacity for calculating an error value for an interpolation mode in an image encoder.

Video encoders that use the Moving Picture Experts Group-2 (MPEG-2) method, such as HDTV (High Definition TV) or video phones, undergo various data compression processes to reduce the data to be transmitted to the video decoder. One method of the data compression process is motion estimation.

In general, motion estimation is an estimation of a motion vector indicating how much a pixel of a current frame has moved relative to the pixels of a previous frame in a continuous image signal. Instead, it refers to an image data compression technique that reduces the amount of image data transmitted by transmitting such motion vectors.

The motion estimation method in each method is as follows.

1) ATRC method (ADTV; Advanced TV)

The motion estimation method used in ADTV adopts a block matching algorithm (BMA) with half-pixel accuracy.

The block matching algorithm (BMA) assumes that the motion of the screen is moved horizontally or vertically in parallel, so that the block image of the frame in which the movement occurs (i.e., the current frame) is positioned at a position in the frame (i.e., the previous frame) before the movement occurs. It is a method of estimating the motion vector through the position by estimating the best matching with the block image.

Motion estimation in ADTV refers to the process of calculating a two-dimensional vector representing the degree of motion. These vectors are called motion vectors.

Two types of motion estimation are used in ADTV, one for forward motion estimation using the past screen to make the current screen, and the other for backward motion estimation using the future screen to make the current screen. To do.

Both methods find the motion vector by finding the block with the least distortion in the predetermined search interval with M * N.

Popular error (or distortion) measurement methods include Mean Squre Error (MSE) and Mean Absolute Error (MAE). ) Is mainly used.

Half-pixel motion estimation consists of two steps: the first search is to obtain an inter-pixel motion vector, and the second search is a half-pixel motion vector centered on the pixel-by-pixel motion vector first obtained. Will be obtained.

In ADTV, the input video is divided into several types and divided into frames. There are three types of I-frame, P-frame, and B-frame, and motion estimation for each frame. Looking at it as follows.

I-frame does not have motion estimation, so there is no motion vector, but it is used for motion estimation of other frames, and P-frame uses the previous I-frame or P-frame for forward motion estimation and another P-frame or B-frame Used for motion estimation, B-frames use forward and backward I-frames or P-frames to perform forward motion estimation and backward motion estimation, and interpolate by dividing the forward motion compensated frame and the backward motion compensated frame in two. The motion vector is determined by selecting the smallest difference of the displaced block difference (DBD) among the three motion estimation results.

When the smallest value of the DBD is interpolated, both the forward motion vector and the backward motion vector are transmitted.

2) G.I company (Digicipher)

The motion estimation method described in G.I's Digicipher system proposal is similar to the block matching algorithm (BMA) used for ATRC's ADTV described above, but it is lower than ADTV because it does not estimate half-pixel motion in accuracy.

In other respects, there is no B-frame of ADTV by performing forward motion estimation without backward motion estimation. That is, the Digicipher frame consists of only I-frames and P-frames.

The unit of motion estimation is a macro block. The macro block is composed of a 4 * 2Y (luminance signal) block, one U (color difference signal) block, and one V (color difference signal) block. Have.

3) MIT method (ATVA-Progressive HDTV)

The motion estimation method proposed by MIT is based on the spatial-temporal constraint, which assumes that image displacement occurs at a constant rate between frames and correlates the displacement rate with the motion vector. It is a method to estimate the motion vector by finding the spatial-temporal constraint, which is a differential equation representing the relationship, and finding its root.

This method assumes a uniform speed of transmission and is not valid for object rotation, camera zoom, moving object transmission areas, or multi-object motion with different motion vectors.

Even if the motion is assumed to be uniform within a small interval and motion estimation is suppressed in an area where the motion estimation is uncertain, the temporal correlation is greatly reduced by motion estimation and compensation based on spatial-temporal constraints. do.

Compared with the block matching algorithm (BMA), this method greatly reduces the amount of computation depending on the block size and performs well with or without noise.

The motion vector is estimated from the luminance signal and the same motion vector is used for the luminance signal Y and the chrominance signal U and V, and one motion vector is obtained for every 32 * 32 pixel block.

Prediction is performed on the current frame from the previous coded frame and the motion vector, and a difference between the current frame and the predicted code is calculated. This difference is referred to as a motion compensated residual.

This is obtained for each of the three parts Y, U and V and when the image has relatively large enough energy (when there is a picture change) for the picture frame, the motion compensation is invalidated and the picture frame itself is encoded.

4) Zenith and ATT (DSC-HDTV)

Due to the nature of the image, the pixel values of adjacent pixels are generally composed of very close values because the pixels in the line representing the boundary and contour of the object are used to display the same object.

Zenith and ATT used a hierarchical motion estimation method that effectively estimates motion by eliminating unnecessary redundancy of frame image data by using the characteristics of these images.

In addition, in order to increase the efficiency of the transmission channel, Zenith and ATT have proposed two-stage motion estimation of coarse motion estimation and fine motion estimation, which are described in detail as follows. .

First, a filter is used to remove redundancy between the frame video signal and the previous video signal. As a result, the resolution is reduced by 1/2 each in the horizontal and vertical directions.

Coarse motion estimation from low resolution video signals with an overall reduction of 1/4 is an estimation error that is the absolute value of the pixel value difference between the current frame and the previous frame in units of 32 * 16 (horizontal * vertical) pixels. In this case, the smallest of the estimation errors obtained by using only the luminance component and moving the previous frame block by the search interval becomes the motion vector of the block.

The block matching in the hierarchical low resolution image as described above reduces the search interval and the block size by a quarter compared to that used in the original high resolution image, thereby significantly reducing the calculation amount.

High resolution motion estimation uses motion vectors obtained from low resolution images to replace them with motion vectors in the original high resolution images and subdivides them into high resolution motion vectors.

That is, since the 16 * 8 block size and the low resolution motion vector in the low resolution image are each reduced by 1/2 in the horizontal and vertical directions of the image at the original resolution, double the two to become the block size and the motion vector of the original image.

It is divided into 8 * 8 small block sizes, and each small block is moved back from the low resolution motion vector by the search interval to obtain a high resolution motion vector.

The motion vector encoder estimates the screen of the next frame using motion vectors estimated at low resolution and high resolution, respectively, and selects a motion vector corresponding thereto.

This is for limiting the bit rate in compressing the motion vector, and depending on the margin of the channel band width, it may send only a low resolution motion vector or a low resolution and high resolution motion vector.

There are various types of motion estimation as described above. In MPEG-2, there are three cases of motion estimation in forward direction, motion estimation in reverse direction, and motion estimation in both directions.

When creating an image encoder using the MPEG-2 method, a motion estimation chip is used. Since the motion estimation chip considers only one direction of motion, it is necessary to effectively implement the three motion estimation methods used in the MPEG-2 method. There is.

1 is a block diagram of a motion estimation apparatus in a conventional image encoder, the conventional apparatus comprising: a demultiplexer 10 for separating input image data into two independent image data on a frame basis; A first frame memory (12a) for storing the separated forward reference frame; A second frame memory (12b) for storing the separated reverse frame (anchor frame); A multiplexer (14) for separately transmitting forward image data and reverse image data from the first and second frame memories (12a, 12b), respectively; A first motion estimator (16a) for receiving the forward image data and performing forward motion estimation; A second motion estimation unit (16b) which receives the transmitted reverse video data and performs backward motion estimation; A first delay unit (18a) for delaying the forward motion estimation result; A second delay unit (18b) for delaying the backward motion estimation result; A third delay unit 18c for delaying data of an input current frame; An interpolation mode error calculator (22) for calculating a difference between the data of the forward direction and the data of the reverse direction and calculating a difference between the delayed current frame data; And a mode selector 22 for selecting the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode, and outputting a motion vector of the selected mode.

Referring to FIG. 1, the operation of the conventional motion estimation apparatus will be described.

After the input image data is separated and selected by the demultiplexer 10, the forward reference frame is stored in the first frame memory 12a, and the reverse reference frame is stored in the second frame memory 12b.

The multiplexer 14 transfers the data of the memory 12a in which the forward image data of the two frame memories 12a and 12b are stored to the first motion estimation unit 16a, and stores the reverse image data. The data of 12b is transmitted to the second motion estimation unit 16b.

The first motion estimation unit 16a performs forward motion estimation, and the second motion estimation unit 16b performs backward motion estimation.

The first delay unit 18a delays the forward motion estimation result, the second delay unit 18b delays the backward motion estimation result, and the third delay unit 18c delays the current frame data. Delay.

The interpolation mode error calculator 22 averages the obtained forward image data and the reverse image data, and then calculates a difference between the average value and the delayed current frame data.

The mode selector 22 selects the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode, and outputs a motion vector of the selected mode.

The motion estimation apparatus in the conventional video encoder operating as described above reads data of the search area from the frame memory for forward and backward motion estimation when performing motion estimation, and as a result, frame memory to perform bidirectional motion estimation. To read more data, you need to increase the operation speed of the frame memory. In order to run the frame memory at a high speed, it is not only difficult to implement the hardware but also increases the cost.

Accordingly, the present invention has been made to solve the above problems, and the frame memory is used only for forward and reverse motion estimation and the search area capacity memory is separately used for calculating an error value for the interpolation mode. It is an object of the present invention to provide a motion estimation apparatus in an encoder.

Motion estimation apparatus according to the present invention for achieving the above object,

A demultiplexer that separates the input image data into two independent image data on a frame basis;

A first frame memory for storing the separated forward reference frame;

A second frame memory for storing the separated reverse reference frame;

A multiplexer which separates the image data from the first and second frame memories into forward image data and reverse image data and transmits the divided image data;

A first motion estimation unit configured to receive the forward image data and perform forward motion estimation;

A second motion estimation unit configured to receive the transmitted reverse video data and perform backward motion estimation;

A first search region memory for storing data corresponding to a search region among the forward image data and receiving a motion vector from the first motion estimator;

A second search region memory for storing data corresponding to a search region among the transmitted reverse image data and receiving a motion vector from the second motion estimator;

A first delay unit for delaying the forward motion estimation result;

A second delay unit for delaying the backward motion estimation result;

A third delay unit for delaying data of an input current frame;

An interpolation mode error calculator which averages forward and backward image data obtained from the first and second search region memories and calculates a difference between the delayed current frame data and the delayed current frame; And

And a mode selector for selecting the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode, and outputting a motion vector of the selected mode.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the present invention.

2 is a block diagram of a motion estimation apparatus in an image encoder according to the present invention, which includes a demultiplexer 30 that separates input image data into two independent image data on a frame basis; A first frame memory (32a) for storing the separated forward reference frame; A second frame memory (32b) for storing the separated reverse reference frame; A multiplexer (34) which separates the image data from the first and second frame memories (32a, 32b) into forward image data and reverse image data and transmits the divided image data; A first motion estimator 36a which receives the transmitted forward image data and performs forward motion estimation; A second motion estimation unit 36b which receives the transmitted reverse video data and performs backward motion estimation; A first search region memory 38a for storing data corresponding to a search region from the forward image data and receiving a motion vector from the first motion estimator 36a; A second search region memory 38b for storing data corresponding to a search region among the transmitted reverse image data and receiving a motion vector from the second motion estimator 36b; A first delay unit 40a for delaying the forward motion estimation result; A second delay unit 40b for delaying the backward motion estimation result; A third delay unit 40c for delaying data of an input current frame; An interpolation mode error calculator (42) which averages forward and backward image data obtained from the first and second search area memories (38a, 38b) and calculates a difference between the delayed current frame data; And a mode selector 44 which selects the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode, and outputs a motion vector of the selected mode.

Next, the operation and effects of the present invention will be described in detail with reference to the accompanying drawings.

In the MPEG-2 method, forward, reverse, and bidirectional motion estimation are used. In the forward and reverse motion estimation, data of a search range is input from a forward reference frame and a reverse reference frame, respectively. Find the data that most closely resembles the current frame.

Bidirectional motion estimation is done only in Bi-directional (predicted picture), and the result data obtained by the forward and reverse motion estimation found above is added, divided by 2 (ie, the average value), and compared to the current frame. If the result is smaller than the difference between the forward only data or the backward only data and the current frame, then bidirectional motion estimation is selected.

That is, after the results of the forward motion estimation and the backward motion estimation are obtained, the bidirectional motion estimation is started.

After the data of the search area is read from the frame memory to perform forward and reverse motion estimation, the data must be read from the frame memory again with the motion vectors obtained from the forward and reverse motion estimation for bidirectional motion estimation.

This means that the frame memory should perform bidirectional motion estimation in addition to the forward and backward motion estimation. Therefore, the operation speed of the frame memory should be faster. There is no need to speed it up.

Referring to FIG. 2, after inputting and separating input image data from the demultiplexer 30, a forward reference frame is stored in the first frame memory 32a and a reverse reference frame is stored in the second frame memory 32b. .

The multiplexer 34 transmits the data of the frame memory 32a in which the forward image data of the two frame memories 32a and 32b are stored to the first motion estimation unit 36a. The data of the frame memory 32b in which the reverse video data is stored is transmitted to the second motion estimation unit 36b.

The first motion estimator 36a performs forward motion estimation, and the second motion estimator 36b performs backward motion estimation.

The first search region memory 38a and the second search region memory 38b, which are the central parts of the present invention, respectively store the data of the search region for forward and reverse motion estimation, respectively, and are two frame memories 32a and 32b. ) Receives and stores the data of the search area transmitted to the first and second motion estimation units 36a and 36b through the multiplexer 34 and by the motion vector from the first and second motion estimation units 36a and 36b. The obtained forward and reverse data is transmitted to the interpolation mode error calculator 42.

The first delay unit 18a delays the forward motion estimation result, the second delay unit 18b delays the backward motion estimation result, and the third delay unit 18c delays the current frame data. Delay.

The interpolation mode error calculation unit 22 averages the obtained forward image data and the reverse image data, and then calculates the error value in the first and second motion estimation units 36a and 36b. Compute the difference from the data in the frame.

The mode selector 22 selects the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode and outputs a motion vector of the selected mode. The terminal a of the mode selector 22 outputs an error value for the forward direction. And a motion vector, a terminal b receives an error value for the reverse direction, a motion vector, and a terminal c receives an error value for both directions.

The reason for delaying the estimation result from the first and second motion estimation units 36a and 36b in the first delay unit 18a and the second delay unit 18b is that at the same time, a, Computing the error values according to the estimation results from the first and second motion estimation units 36a and 36b in the interpolation mode error calculation unit 42 so as to transmit forward, reverse, and bidirectional error values to the b and c terminals, respectively. Because.

The reason why the third delay unit 18c delays the current frame is that the error calculation starts in the interpolation mode error calculator 42 after the first and second motion estimation units 36a and 36b finish the calculation. Because.

As a result, when the first and second motion estimation units 36a and 36b respectively read the data of the search area for motion estimation, the data of the search area are stored in the first and second search area memories 38a and 38b, respectively. After the motion estimation is completed by the first and second motion estimation units 36a and 36b, the data is output by the resultant motion vector, and the first and second search area memories 38a and 38b are converted into error values for the interpolation mode. By using the memory dedicated to calculation, it is not necessary to speed up the operation of the frame memory.

As described above, the present invention uses the first and second search area memories 38a and 38b as dedicated memories for calculating error values for the interpolation mode, thereby operating speeds of the first and second frame memories 32a and 32b. Since the coder does not have to be fast, the video encoder is easier to implement and the cost can be reduced.

Claims (1)

A demultiplexer 30 for separating the input image data into two independent image data on a frame basis; A first frame memory (32a) for storing the separated forward reference frame; A second frame memory (32b) for storing the separated reverse reference frame; A multiplexer (34) which separates the image data from the first and second frame memories (32a, 32b) into forward image data and reverse image data and transmits the divided image data; A first motion estimation unit 36a which receives the transmitted forward image data and performs forward motion estimation; A second motion estimation unit 36b which receives the transmitted reverse video data and performs backward motion estimation; A first search region memory 38a for storing data corresponding to a search region from the forward image data and receiving a motion vector from the first motion estimator 36a; A second search region memory 38b for storing data corresponding to a search region among the transmitted reverse image data and receiving a motion vector from the second motion estimator 36b; A first delay unit 40a for delaying the forward motion estimation result; A second delay unit 40b for delaying the backward motion estimation result; A third delay unit 40c for delaying data of an input current frame; An interpolation mode error calculator (42) which averages forward and backward image data obtained from the first and second search area memories (38a, 38b) and calculates a difference between the delayed current frame data; And a mode selector 44 which selects the most similar to the data of the current frame among the forward mode, the reverse mode, and the interpolation mode, and outputs a motion vector of the selected mode.