KR20070061214A - Low cost motion estimation device and motion estimation method - Google Patents

Abstract

An inexpensive motion estimation apparatus and a motion estimation method are provided to reduce hardware cost and minimize hardware complexity while minimizing deterioration of picture quality and decrease the quantity of calculations. A motion estimation apparatus includes a sampling unit(102), a block segmentation/address generation unit(104), and a motion calculation unit(105). The sampling unit samples block-based video data to generate sampling blocks. The block segmentation/address generation unit segments the sampled data into sampling sub-blocks and generates an address for motion estimation. The motion calculation unit calculates a motion for each sub-block by using an estimation function.

Description

Low Cost Motion Estimation Device and Motion Estimation Method

1 is a block diagram showing one structure of a general video compression apparatus.

2 is a block diagram showing a motion estimation apparatus according to an embodiment of the present invention.

3 is a conceptual diagram illustrating a 2: 1 sampling process according to the present invention.

4 is a memory map illustrating a search region for forming a reference block in accordance with the present invention.

5 is a block diagram illustrating block grouping in H.264 format.

6 is a flowchart illustrating a motion estimation method according to an embodiment of the present invention.

7 is a flowchart showing an embodiment of a motion calculation method in the case of MPEG4 format.

8 is a flowchart illustrating an embodiment of a motion calculation method in the case of H.264 format.

9 is a circuit diagram of a processing structure embedded in a motion estimation apparatus according to an embodiment of the present invention.

10 is a block diagram showing a motion estimation apparatus according to the prior art.

* Explanation of symbols for the main parts of the drawings

102: 2: 1 Sampling Block

103: Memory

104: Address Generation for Block Division

105: SAD calculation unit (SAD Calulation)

106: Optimized Mode Decision

An object of the present invention relates to the field of VLSI (Very Large Scale Integration) design for implementing the image data in hardware using a compression algorithm, in particular, a motion estimation apparatus and motion that can be applied to VLSI implementing the image compression algorithm It is about an estimation method.

H.264 is a standard that is jointly developed by VCEG of ITU and MPEG of ISO, a group of international video standards. The standard has a very high compression ratio as the main technical goal. It is a general purpose video encoding technology that can be used in almost all transmission media such as storage media, Internet, satellite broadcasting, and various video resolutions. Traditionally, ITU's international standards have established video coding standards such as H.261 and H.263, which are based on wired communication media. MPEG is MPEG-1 and MPEG- for video processing in storage media and broadcasting media. 2 etc. were established as a standard. MPEG also completed the development of the MPEG-4 video standard, which realizes various functions and high compression ratios, which are important features of object-based video coding in MPEG-4, a multimedia standard coding standard. ITU's VCEG group has been developing a high compression video standard under the name of H.26L since the MPEG-4 video standard was established. In an official comparison experiment in MPEG, the MPEG-4 video standard (advanced simple profile) It showed a greater advantage in terms of compression ratio. Accordingly, MPEG decided to develop H.264 / AVC, a JVT video standard, in collaboration with ITU VCEG Group based on H.26L.

H.264 / AVC has various excellent characteristics, but the method of determining the optimal coding mode among them contributes greatly to the performance improvement. The optimal coding mode determination module is a part for determining an encoding mode of a macroblock which is a basic unit of encoding, and motion estimation is located at the core thereof.

As a feature, a macroblock may be divided into subblocks of various shapes, and each subblock may have a separate motion vector. In addition, unlike conventional motion estimation method using only one reference image, a large number of reference images can be used to obtain a significant compression efficiency.

However, these features lead to an increase in the amount of computation. Therefore, the motion estimation algorithm in H.264 / AVC should be designed in consideration of the side of prediction error and the side of calculation.

The conventional technique as shown in FIG. 10 has a problem in that a large area and power consumption are generated by implementing an additional memory when implementing the VLSI using an algorithm of motion estimation.

Motion Estimation requires a lot of computation, and many algorithms and hardware structures have been studied so far. The conventional invention proposed a hybrid motion estimation using a motion estimation skipping algorithm without degrading the image quality, and before the normal operation of the motion estimation, the motion vector of the previous and the right macroblocks above and above the current block. A Motion Vector Prediction value is obtained by performing a Motion Vector Prediction process of selecting an intermediate value among them as a Motion Vector.

A motion compensation (SADmcp) value is obtained by performing motion compensation through the obtained prediction vector. At the same time, the maximum SAD value is obtained from the input of the SAD value from the previous macro block and the upper and upper right macro blocks. At this time, if the obtained SAD value is SADmax, the previously estimated SADmcp and SADmax are compared. If SADmcp is small, motion estimation is omitted.

In this method, the motion estimation is performed in 16x16 macroblock units based on the MPEG-4 standard, so the image quality is lower than the sub-block division method, and the motion estimation skip mode has a disadvantage in that power consumption is increased due to additional circuits. .

The problem of the prior art is that it requires a large amount of computation, it is difficult to implement a real time in the video encoder. In addition, there are problems in area and power consumption by implementing additional memory inside. In addition, by using a fixed algorithm according to the type and application of the image, an unnecessary amount of computation is required, and there is a limitation in using a motion estimation algorithm suitable for the type of the image.

The present invention has been made to solve the above problems, and an object thereof is to provide a motion estimation apparatus and a motion estimation method that can reduce hardware cost.

Another object of the present invention is to provide a motion estimation apparatus and a motion estimation method capable of reducing the amount of computation.

In accordance with another aspect of the present invention, there is provided a motion estimation apparatus comprising: a sampling unit for generating a sampling block by performing sampling on block unit image data; A block division / address generator for dividing the sampled data into a sampling subblock and generating an address for a motion estimation operation; And a motion calculating unit for calculating a motion by using an estimation function in each subblock unit.

According to another aspect of the present invention, there is provided a motion estimation method, comprising: generating a sampling block by sampling image data in a predetermined block unit; Dividing the sampling block into a plurality of sampling subblocks; Calculating a similarity degree between an area designated by each motion vector and each sampling subblock with respect to a specific reference block from the outside; And determining the motion vector for the received image data by combining the similarities for each motion vector with respect to each of the plurality of sampling subblocks.

In the H.264 standard, excellent performance and compression efficiency can be obtained by dividing the subblocks into various shapes, but this feature increases the computational complexity. In the present invention, a motion estimation algorithm and an optimal sub-block partitioning method are proposed to minimize hardware complexity when implementing the hardware while minimizing degradation of image quality.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

(Example)

1 is a block diagram of a general video encoding apparatus. Referring to FIG. 1, a video encoding apparatus may include a transform / quantizer 110, a dequantizer / inverse transform 131, and a deblocking filter 133. ), Picture reproduction unit 135, motion compensated predictor 137, intra predictor 139, motion estimator 100, subtractor 170, and entropy Entropy) coding unit 190 is included.

Image data is input to a video encoding apparatus in units of macroblocks of 16x16 pixels. The transform / quantizer 110 converts the input macroblocks according to a predetermined method and then quantizes them. A representative image transformation technique is DCT (Discrete Cosing Transform).

The dequantizer / inverse transform 131 receives inverse quantized image data after transformation from the transform / quantizer 110 and inverse quantizes and inverse transforms. The deblocking filter 133 receives inverse quantized and inversely transformed image data from the dequantizer / inverse transform 131 and performs filtering to remove a blocking effect.

The picture reproducing unit 135 receives the filtered image data from the deblocking filter 133 and reproduces and stores the image in picture units. The picture is an image in a frame unit or an image in a field unit. The picture reproducing unit 135 includes a buffer (not shown) capable of storing a plurality of pictures. The plurality of pictures stored in the buffer are pictures provided for motion estimation, hereinafter referred to as reference pictures.

The motion estimation apparatus 100 receives at least one reference picture stored in the picture reproducing unit 135 and performs motion estimation of an input macro block according to the present invention to perform motion data including a motion vector, an index indicating a reference picture, and a block mode. Outputs

The motion compensation predictor 137 corresponds to an input macro block from a reference picture used for motion estimation among a plurality of reference pictures stored in the picture reproducing unit 135 according to the motion data input from the motion estimation device 150. Extract and output the macro block.

The subtractor 170 receives the macroblock in the reference picture corresponding to the input macroblock from the motion compensation predictor 137 and performs a difference operation with the input macroblock when the input macroblock is predictively encoded between the pictures. Output a signal.

The residual signal output from the subtractor 170 is transformed and quantized by the transform / quantizer 110 and entropy coded by the entropy coding unit 190 to generate an output bit stream. The intra prediction unit 139 performs intra picture prediction coding, not inter picture prediction coding, using the reference picture.

On the other hand, the video decoder 130 for decoding the bit stream generated by the video encoding apparatus described above is a dequantizer / inverse transform 131, the deblocking filter 133, the picture reproduction unit 135, The motion compensation predictor 137 and the intra predictor 139 are included.

The present invention is to provide an improvement technique for the motion estimation apparatus 100 among the components of FIG. 1, and may be applied to a video encoding apparatus having another structure requiring a motion estimation apparatus as well as the video encoding of FIG. 1.

According to an embodiment of the present invention, a motion estimation apparatus includes a sampling unit 102 for generating a sampling block by performing sampling on block unit image data; A sampling memory (103) for storing the data sampled by the sampling section; A block division / address generation unit (104) for dividing the sampled data into a sampling subblock and generating an address for a motion estimation operation; A motion calculator 105 for calculating a motion by using an estimation function in each subblock unit; And an optimal mode determiner 106 for determining an optimal block grouping mode for moving picture compression.

The sampling unit 102 generates a 8x8 sampling block by 2: 1 sampling 16x16 image data on a horizontal axis and a vertical axis. That is, the sampling unit 102 reduces the data to 1/4 by performing a 2: 1 sampling of the original image, as shown in FIG. 3, in order to reduce the computational complexity of the various block modes. ). Accordingly, it can be seen that the sampling memory 103 is reduced to 1/4 in size compared with a memory for storing an input block of a motion estimation apparatus according to the prior art.

The block division / address generating unit 104 divides the 8x8 unit sampling block into four 4x4 unit sampling subblocks. In this case, since each pixel of the 4x4-sampling subblock is 2: 1 sampled, the pixels correspond to 8x8 mode data of the original image data. In addition, the block division / address generator 104 designates a pixel reading position in a reference block given as a motion vector according to the compression mode.

The motion calculator 105 performs SAD operation on the divided 4x4 unit of the sampling subblock and the reference block. In some cases, an SSD operation may be implemented.

The optimal mode determiner 106 selects one of four block grouping modes defined by the H.264 standard when the compression mode is H.264, and the device does not support the H.264 standard mode. May be omitted.

Hereinafter, a motion estimation method performed in the illustrated motion estimation apparatus will be described.

As shown in FIG. 6, the motion estimation method includes: generating a sampling block by sampling image data in a predetermined block unit (S100); Dividing the sampling block into a plurality of sampling subblocks (S120); Calculating a similarity degree between an area designated by each motion vector and each sampling subblock with respect to a specific reference block from the outside; And determining the motion vector for the received image data by combining the similarities for each motion vector with respect to each of the plurality of sampling subblocks (S800).

The illustrated motion estimation method implements the compression mode to selectively apply the MPEG4 format and the H.264 format. Accordingly, the calculating of the similarity of each of the illustrated sub-blocks may include determining a compression mode setting (S140); A similarity calculation step (S200) in the MPEG4 format; And a similarity calculating step (S400) in the H.264 format.

Although similarity calculating steps S200 and S400 of the sampling sub-blocks in the figure are illustrated as executing the SAD function, the SSD function may be executed depending on the implementation.

Since the motion estimation apparatus of the present embodiment receives image data of 16 × 16 units, the sampling block generation step 100 samples 16 × 16 unit image data by 8 × 8 units, and in the sampling block division step S120. The 8x8 unit sampling data is divided into four 4x4 unit sampling subblocks.

Now, the motion calculation step S200 when the compression mode is the MPEG4 format will be described in detail with reference to FIG. 7.

When four processing elements for calculating SADs for each of four sampling subblocks are provided, the processes of calculating SADs for each subblock (S222, S224, S226, and S228) are simultaneously parallel as shown. Is performed as

In the case of MPEG4 mode, since SAD is calculated for 16x16 image data, in FIG. 7, although SAD is calculated for four 8x8 image data, this is for parallel processing using four processing elements. The reference block is addressed so as to be equal to the calculation of SAD with respect to the image data of 16 × 16 units.

That is, when the motion vector is determined according to the step S210 or the step S280, the address generator calculates the address of the pixel of the reference block corresponding to each pixel of the sampling subblock according to the determined motion vector, and each processing element corresponds to the corresponding angle. Perform SAD operations using pixels.

Therefore, once the SAD calculation is performed four times for the set motion vector in which the motion vector is set, the step S800 of determining the motion vector illustrated in FIG. A summation process is performed on all motion vectors to select a motion vector having the highest similarity.

Now, the motion calculation step (S400) when the compression mode is the H.264 format will be described in more detail with reference to FIG. 8.

In the H.264 mode, there are four block grouping modes defined as 16x16, 16x8, 8x16, and 8x8.SAD operation is performed on all block grouping modes on the input image data, and the most efficient block grouping mode is selected. Will be chosen.

When four processing elements for calculating SADs for each of four sampling subblocks are provided, the processes of calculating SADs for each subblock (S422, S424, S426, and S228) are parallel at the same time. The SAD aggregation process (S432, S434, S436, S438) according to the four block grouping modes is simultaneously performed in parallel from the four SAD calculation results.

The motion vector determining step S400 of FIG. 6 includes: a 16x16 step of calculating the similarity in the 16x16 mode; A 16x8 step of calculating the similarity in the 16x8 mode; An 8x16 step of calculating the similarity in the 8x16 mode; And an 8x8 step of calculating the similarity in the 8x8 mode. As shown in FIG. 8, the 16x16 step includes a SAD calculation process (S422, S424, S426, S228) for each of the sampling subblocks, and an SAD aggregation process (S432) in the 16x16 mode. The step consists of the SAD calculation process (S422, S424, S426, S228) for each of the sampling sub-blocks and the SAD aggregation process (S434) in the 16x8 mode, wherein the 8x16 step is SAD for each of the sampling sub-blocks. Computation process (S422, S424, S426, S228) and SAD aggregation process (S436) in the 8x16 mode, the 8x8 step is SAD calculation process for each of the sampling sub-blocks (S422, S424, S426, S228) And an SAD aggregation process (S438) in the 8x8 mode.

In step 16x16, the similarity of each of the four sampling subblocks is summed with respect to one motion vector for all motion vectors, and the motion vector having the largest similarity is selected. It is almost similar.

In the 16x8 step, the four sampling subblocks are divided into two sets for the 16x8 mode, and the similarity of each of the two sampling subblocks belonging to the first set is summed for one motion vector. A motion vector having the highest similarity is selected by performing the motion vector, and the similarity of each of the two sampling subblocks belonging to the second set with respect to one motion vector is added to all motion vectors. Selects a large motion vector and adds similarities for the selected two motion vectors.

In step 8x16, the four sampling subblocks are divided into two sets for the 8x16 mode, and the similarities of the two sampling subblocks belonging to the first set are summed with respect to one motion vector. A motion vector having the most similarity is selected by performing the motion vector, and the similarity of each of the two sampling subblocks belonging to the second set is summed for all the motion vectors. A process of selecting a motion vector having a high similarity and summing similarities for the selected two motion vectors is performed.

In step 8x8, a process of obtaining similarities of one sampling subblock with respect to one motion vector is performed for all motion vectors, and a motion vector having the largest similarity is selected for each sampling subblock. Next, the similarity of the selected motion vectors for each sampling subblock is added.

In the case of the H.264 format, the step of determining the motion vector of FIG. 6 (S800) determines the results of SAD calculation (S422, S424, S426, S228) for four sub-blocks in a different manner according to the block grouping mode in the MPEG4 format. Will be collected.

In the case of FIG. 8, since the addressing of the reference block for the motion vector set to calculate the SAD for each sampling subblock is performed for all the reference blocks for all the sampling subblocks, the movements made in steps S410 and S480 are performed. The vector setting has a wider width than that of MPEG4, and there may be a motion vector setting value that is skipped without being collected in steps S432, S434, and S436.

9A and 9B show a configuration of processing elements for performing SAD operation. The processing structure shown in FIG. 9A illustrates only a structure for performing SAD operations on one sampling sub-block. In this embodiment, a 4x4-sampling sub-block in which the 8 × 8 split original image is 2: 1 sampled The SAD operation is performed on the four processing elements 200, 201, 202, and 203 and one summer / comparator 300.

Although the motion calculating unit 105 of FIG. 1 may include only one processing structure of FIG. 9A, all four processing structures of FIG. 9A, one for calculating SAD for each divided sub-block, are added to increase the speed. It may be implemented to calculate the SAD for the four sampling sub-blocks at the same time.

FIG. 9B illustrates a detailed structure of one processing element 201. The illustrated structure uses two frames before the current screen frame as a comparison target for SAD operation. SAD operations are performed on two frames of comparison and odd number of frames simultaneously.

Specifically, R is the current data input, S0 and S1 are the input of the previous frame data, and the input is an odd number and even number input alternately. Using the input of two processing elements, odd and even numbers are alternately inputted so that the internal processing speed is doubled than the speed of the input data.The rest of the time except for setting the initial value The efficiency of the processing is characterized by having 100%.

In addition, the SAD is obtained by using four processing elements to obtain the SAD. Psum_in is a value that stores an intermediate temporary SAD, and stores an intermediate SAD value of PE_0, PE_1, and PE_3, and accumulates the current value to calculate SAD.

As described above, the proposed motion estimation apparatus and method simultaneously calculates SAD (Sum Absoluted Difference) values for four sampling subblocks and selects an optimal condition according to a mode, thereby reducing computation time. It can be seen that the amount of computation and memory requirements can be reduced to less than 1/4 using the 2: 1 sampling method.

The present invention has been described in detail with reference to preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made by those skilled in the art within the scope of the technical idea of the present invention. Do.

By implementing the motion estimation apparatus and the motion estimation method according to the present invention of the above configuration, it is possible to reduce the hardware cost and / or calculation amount.

That is, the present invention proposes an algorithm and a structure for calculating a motion vector by calculating the SAD for each subblock using only four processing elements using block partitioning. Less than 1/4 of the method is effective.

Furthermore, the present invention implements H.264 motion estimation with low hardware cost because it performs calculation for each mode with four processing elements, which can be used as a core core for portable multimedia terminals requiring low power. .

In addition, the present invention has the effect that the operation time can be shortened by simultaneously performing the SAD operation on the divided sampling sub-blocks.

Claims (16)

A sampling unit for generating a sampling block by performing sampling on the block unit image data; A block division / address generator for dividing the sampled data into a sampling subblock and generating an address for a motion estimation operation; And Motion calculator for calculating motion by using motion estimation function for each sub block Motion estimation apparatus comprising a. The method of claim 1, And a sampling memory for storing the data sampled by the sampling unit. The method of claim 1, Optimal Mode Determination Unit for Determining the Best Block Grouping Mode for Video Compression Motion estimation device further comprising. The method of claim 1, The sampling unit samples 16x16 image data into 8x8 sampling blocks. And the block division / address generator divides the 8x8 sampling block into four sampling subblocks of 4x4. The method of claim 1, wherein the motion estimation function, Motion estimation apparatus, characterized in that any one of the SAD function, SATD function and SSD function. The method of claim 1, wherein the motion calculator, And four processing structures for calculating the similarity between the respective sampling sub-block data and the reference image data for motion estimation. The method of claim 1, wherein the motion calculator, And a motion vector having the highest similarity is selected by adding the similarities to the four sampling subblocks with respect to one motion vector. The method of claim 1, wherein the motion calculator, A 16x16 step of calculating the similarity in the 16x16 mode; A 16x8 step of calculating the similarity in the 16x8 mode; An 8x16 step of calculating the similarity in the 8x16 mode; An 8x8 step of calculating the similarity in the 8x8 mode; And Determining an optimal mode according to the result of the four similarity calculation steps Motion estimation apparatus, characterized in that for performing. The method of claim 1, wherein the motion calculator, Determines whether the set mode is MPEG4 mode or H.264 mode, Judging by the MPEG4 mode, Performing a process of summing similarities of each of the four sampling subblocks with respect to one motion vector for all motion vectors to select a motion vector having the highest similarity, Judging by the H.264 mode, A 16x16 step of calculating the similarity in the 16x16 mode; A 16x8 step of calculating the similarity in the 16x8 mode; An 8x16 step of calculating the similarity in the 8x16 mode; An 8x8 step of calculating the similarity in the 8x8 mode; And Determining an optimal mode according to the result of the four similarity calculation steps Motion estimation apparatus, characterized in that for performing. Generating a sampling block by sampling image data in a predetermined block unit; Dividing the sampling block into a plurality of sampling subblocks; Calculating a similarity degree between an area designated by each motion vector and each sampling subblock with respect to an external specific reference block; And Determining the motion vector for the received image data by combining the similarities for each motion vector with respect to each of the plurality of sampling subblocks. Motion estimation method comprising a. The method of claim 10, In the generating of the sampling block, 16x16 image data is sampled in 8x8 units. In the step of partitioning the sampling block, the 8x8 unit sampling data is divided into four 4x4 unit sampling subblocks. The method of claim 10, And a SAD function or an SSD function are executed in the similarity calculation step of the sampling sub-block. The method of claim 11, wherein the determining of the motion vector comprises: A method of estimating a motion vector having the highest similarity is performed by performing a process of adding similarities to the four sampling sub-blocks for one motion vector for all motion vectors. The method of claim 11, wherein the determining of the motion vector comprises: A 16x16 step of calculating the similarity in the 16x16 mode; A 16x8 step of calculating the similarity in the 16x8 mode; An 8x16 step of calculating the similarity in the 8x16 mode; An 8x8 step of calculating the similarity in the 8x8 mode; And Determining an optimal mode according to the result of the four similarity calculation steps Motion estimation method comprising a. The method of claim 11, wherein in the motion vector determination step, Determines whether the set mode is MPEG4 mode or H.264 mode, Judging by the MPEG4 mode, Performing a process of summing similarities of each of the four sampling subblocks with respect to one motion vector for all motion vectors to select a motion vector having the highest similarity, Judging by the H.264 mode, A 16x16 step of calculating the similarity in the 16x16 mode; A 16x8 step of calculating the similarity in the 16x8 mode; An 8x16 step of calculating the similarity in the 8x16 mode; An 8x8 step of calculating the similarity in the 8x8 mode; And Determining an optimal mode according to the result of the four similarity calculation steps The motion estimation method, characterized in that for performing. The method according to claim 14 or 15, In the 16x16 step, Summing similarities of each of the four sampling subblocks with respect to one motion vector with respect to all motion vectors to select a motion vector having the highest similarity, In the 16x8 step, A process of dividing the four sampling subblocks into two sets for a 16x8 mode and summing similarities of each of the two sampling subblocks belonging to the first set with respect to one motion vector is performed for all motion vectors. A motion vector having the largest similarity is selected, and a process of summing similarities of each of two sampling subblocks belonging to the second set with respect to one motion vector is performed for all motion vectors. Select and sum similarities for the selected two motion vectors, In the 8x16 step, The four sampling subblocks are divided into two sets for an 8x16 mode, and the similarity of each of the two sampling subblocks belonging to the first set with respect to one motion vector is performed for all motion vectors. A motion vector having the largest similarity is selected, and a process of summing similarities of each of two sampling subblocks belonging to the second set with respect to one motion vector is performed for all motion vectors. Select and sum similarities for the selected two motion vectors, In the 8x8 step, A process of finding similarities of one sampling subblock with respect to one motion vector is performed for all motion vectors, and a motion vector having the largest similarity is selected for each sampling subblock, and each sampling subblock is performed. And adding similarities for the selected motion vectors for the block.

Images