KR101182037B1 - Apparatus and method for high precision motion estimation in video signal - Google Patents

Abstract

Disclosed are a high precision motion estimation apparatus and method for a video signal.
The video encoder searches for candidate blocks in the search range in the previous frame corresponding to the position of the macroblock in the current frame, multi-scale candidate blocks in which the candidate blocks in the search range are scaled, and compares the macroblock with the searched candidate blocks. By matching, the highest correlation candidate block having the highest correlation with the macroblock is estimated. The video encoder normalizes the transformed (expanded or reduced) candidate blocks by scaling to calculate the rate-distortion cost while searching each candidate block in a zoomed-out and reduced state during motion estimation, and then selects the highest correlation candidate block. A structure of estimating multi-scale motion estimation is adopted, and a motion vector and a ratio vector of the highest correlation candidate block are stored.
According to this configuration, it is possible to more precisely improve the motion estimation process of inter-screen prediction, which greatly affects the image quality and compression ratio in the video codec, and to reduce the loss of image quality due to quantization of residual data generated during video encoding. At the same time, a higher compression ratio can be obtained.

Description

High precision motion estimation apparatus and method thereof for video signal {APPARATUS AND METHOD FOR HIGH PRECISION MOTION ESTIMATION IN VIDEO SIGNAL}

The present invention relates to an apparatus and method for high precision motion estimation of a video signal, and more particularly, to an encoding technique for improving motion estimation in a process of compressing a video signal.

The process of compressing a block-based video signal in an MPEG / H.26x-based video encoder is as follows.

Each frame of the video signal is divided into slice units, and the slice is divided into macroblock units of 16 × 16 size. H264 / AVC, the latest MPEG / H.26x-based codec, performs intra prediction when a macroblock is a block included in an I slice, and performs intra screen prediction and inter screen prediction when the block is included in a P or B slice. Perform all predictions. Residual data, mode information, motion vectors, etc., generated through this process are transformed into compressed data streams through frequency conversion, quantization, and entropy encoding.

Loss of information occurs during quantization of residual data after frequency conversion, which leads to loss of image quality in the actual video image. In addition, since the target that is actually compressed through entropy encoding is the residual data, the compression efficiency can be increased by reducing the residual data by more precisely improving the intra prediction and the inter prediction.

FIG. 1 illustrates a quarter pixel motion estimation used to refine motion estimation in a video encoder according to the prior art, and illustrates the motion estimation introduced in H.264 / AVC, the latest codec based on MPEG / H.26x. have.

Motion estimation is a process of finding the region with the highest correlation in the previous frame through a matching process using a macroblock of the current frame. General motion estimation finds a motion vector that points to the highest correlation block at integer pixel locations. The motion estimation at the quarter pixel position is a technique of interpolating surrounding pixels of the highest correlation integer pixel to obtain the highest correlation half pixel, and again interpolating the surrounding pixels to find the position of the highest correlation quarter pixel.

Although the method of FIG. 1 is an effective method for improving the precision of motion estimation, there is a limitation that only vertical and horizontal motions of objects in an image sequence can be estimated. However, the objects on the actual image sequence frequently have three-dimensional movements such as zoom-in and zoom-out as well as such planar movements, and such motions cannot be estimated by conventional techniques.

The present invention has been proposed to solve the problems of the prior art as described above, and an object thereof is to improve the image quality and compression ratio of a video encoder by improving the precision of a motion estimation process performed for inter prediction in a video encoder. It's in the ship.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. There will be.

The high-precision motion estimation apparatus of a video signal according to the present invention searches for candidate blocks in a search range in a previous frame corresponding to positions of macroblocks in a current frame and multi-scale candidate blocks in which the candidate blocks in the search range are scaled. A candidate block search unit; A matching execution unit for estimating the highest correlation candidate block having the highest correlation with the macroblock from the searched candidate blocks through matching; And a block normalization unit for normalizing the multi-scale candidate blocks to the size of the macroblock so that the matching execution unit compares the macroblock with normalized candidate blocks and performs matching.

The method of high precision motion estimation of a video signal according to the present invention includes the steps of: a video encoder designating a macroblock of a current frame; (B) the video encoder searching for candidate blocks in a search range in a previous frame corresponding to positions of macroblocks in the current frame and multi-scale candidate blocks in which candidate blocks in the search range are scaled; And (c) the video encoder estimating a highest correlation candidate block having the highest correlation with the macroblock from the searched candidate blocks through matching. The video encoder normalizes the searched candidate blocks to the size of the macroblock in the matching process of (c) to match the macroblock and the normalized candidate blocks.

According to the high-precision motion estimation apparatus and method of the video signal of the present invention, it is possible to more accurately improve the motion estimation process of inter-screen prediction, which greatly affects the image quality and the compression ratio in the video codec.

In addition, according to the high-precision motion estimation apparatus and method of the video signal of the present invention, the residual data of the video signal can be efficiently reduced by using a motion estimation technique that reflects the expansion and contraction motion of the video sequence in the compression of the video signal. . As a result, a higher compression ratio can be obtained while reducing image quality loss due to quantization of residual data generated in the video encoding process.

This effect is expected to be applied to the next generation Ultra HD broadcasting service and video disc requiring up to 8K (7680x4320) resolution. In particular, it is expected to be effective as a new technique of next-generation high efficiency video compression codec such as H.265.

FIG. 1 illustrates quarter pixel motion estimation used in a video encoder according to the prior art to refine motion estimation.
2 is a block diagram of a video encoder according to an embodiment of the present invention.
3 illustrates partition types of macroblocks and sub-macroblocks for motion estimation in an embodiment of the present invention.
4 illustrates a detailed configuration of an inter prediction unit in an embodiment of the present invention.
FIG. 5 is a diagram illustrating multi-scale motion estimation used to reflect expansion and contraction of an image sequence in motion estimation in an embodiment of the present invention.
6 illustrates normalization of enlarged and reduced candidate blocks in an embodiment of the present invention.
7 illustrates a multi-scale sampling technique in one embodiment of the present invention.
FIG. 8 illustrates a form in which the sampling technique of FIG. 7 is extended in two dimensions in an embodiment of the present invention.
9 is a flowchart illustrating a high precision motion estimation method of a video signal according to an embodiment of the present invention.

Hereinafter, a high precision motion estimation apparatus and method thereof for a video signal according to an exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 illustrates an MPEG / H.26x based video encoder structure as an apparatus for high precision motion estimation of a video signal according to an embodiment of the present invention.

A process of compressing an input video signal into a compressed data stream with reference to FIG. 2 will now be described.

The block unit unit 100 divides each frame of the input video signal into macroblock units of 16 × 16 size. When the segmented macroblock is an I slice, the video encoder performs intra prediction through the intra prediction unit 110 and frequency-resolves the resulting residual image, mode information of intra prediction, motion vector, and the like. The conversion is performed through the conversion unit 130. The transformed coefficient is quantized through the quantization unit 140. In addition, the video encoder encodes the quantized result through the entropy encoder 150 and generates a compressed data stream.

If the segmented macroblock is a P slice or a B slice, the video encoder performs a process between the reconstruction image of the previous frame reconstructed as the original image through the inverse quantization unit 160 and the inverse frequency converter 170 and the image between the image of the current frame. The prediction unit 200 performs inter prediction.

In particular, the inter-prediction unit 200 searches for each candidate block in an enlarged and reduced state in the motion estimation process in order to reflect the enlargement and reduction of the image sequence in the motion estimation, and then rate-distortion cost. ), A multi-scale motion estimation scheme is employed to normalize candidate blocks modified (enlarged or reduced) through scaling to estimate the highest correlation candidate block.

Thereafter, the video encoder compares the rate-distortion cost of the final mode obtained through the inter-prediction unit 200 with the rate-distortion cost of the final mode from the intra prediction unit 110 to have a smaller value. It determines the compressed data stream.

One of the eight modes shown in FIG. 3 is determined as the final mode of the inter prediction.

FIG. 3 illustrates partition types of macroblocks and sub-macroblocks for motion estimation in an embodiment of the present invention, which is one of techniques used to increase compression efficiency by reducing residual data in H.264 / AVC. The partition form of the block P110 and the sub macroblock P120 is illustrated.

In the video encoder of one embodiment, macroblocks are divided into eight modes of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 in order to increase the precision of motion estimation. The video encoder calculates the rate-distortion cost according to each mode, and determines the mode having the least cost among the rate-distortion costs of each mode as the final mode to generate residual data.

4 illustrates a detailed configuration of an inter prediction unit in an embodiment of the present invention.

The inter prediction unit 200 includes a candidate block search unit 210, a block normalization unit 220, and a matching execution unit 230.

The candidate block search unit 210 searches for candidate blocks in the search range in the previous frame corresponding to the position of the macroblock in the current frame and multi-scale candidate blocks in which the candidate blocks in the search range are scaled. That is, all the candidate blocks in the enlarged or reduced state are searched to reflect the enlargement or reduction of the image sequence in the motion estimation.

The matching performing unit 230 estimates the highest correlation candidate block having the highest correlation with the macroblock from the candidate blocks searched by the candidate block searching unit 210 through matching. Specifically, the matching unit 230 finds the highest correlation candidate block having the smallest SAD value by obtaining a sum of the absolute difference (SAD), which is the sum of the absolute differences between the pixels in the current macroblock and the searched candidate blocks. The multi-scale motion estimation is completed by storing the positions of the highest correlation candidate blocks as motion vectors and the scales of the highest correlation candidate blocks as ratio vectors.

The block normalization unit 220 compares the multi-block candidate blocks searched by the candidate block search unit 210 with the macroblock size so that the matching unit 230 may compare the macroblock with the normalized candidate blocks. Normalized to (e.g., 16x16).

Such a configuration can more accurately improve the motion estimation process of inter-screen prediction, which greatly affects the image quality and compression ratio in the video codec. To this end, candidates that are more correlated with the macroblock of the current frame by reflecting not only two-dimensional motions of the candidate blocks obtained through the motion vector of the video codec but also three-dimensional motions such as enlargement and reduction by scaling the candidate blocks. The process of estimating the block and effectively reducing the residual data is performed.

FIG. 5 is a diagram illustrating multi-scale motion estimation used to reflect expansion and contraction of an image sequence in motion estimation in an embodiment of the present invention.

(a) is a block-based motion estimation technique performed in inter prediction of a video encoder. In (a), a video frame divided into 16 × 16 macroblocks P200 is a candidate block in a search range P210 corresponding to a block position of a current frame from a previous frame. Searching for (candidate blocks) to find a motion vector (P220) indicating the position of the candidate block having the most correlation with the macroblock (P200) through matching. The matching process is to calculate sum of the absolute difference (SAD), which is the sum of the absolute differences between the pixels in the macroblock of the current frame and the candidate block of the previous frame. The position of the candidate block having the smallest value is set to x and y. Stored through a motion vector (P220) that is a configured two-dimensional coordinates. The motion estimation process is terminated when a matching process is found for all candidate blocks in the search range P210.

The technique of (a) alone is limited in that it can estimate only vertical and horizontal movements of objects in an image sequence, and precisely estimates the movements of objects in an image sequence showing three-dimensional movements such as zoom-in and zoom-out as well as planar movements. Difficult to do

(b) is a structural diagram of a multi-scale motion estimation proposed to improve the above-mentioned problem by reflecting enlargement and reduction of an image sequence in motion estimation. The motion estimation in (a) and the basic process are the same. That is, the macroblock P300 of the current frame searches for candidate blocks within the search range P310 of the previous frame and selects the candidate block having the most correlation with the macroblock P300 of the current frame. A matching process is performed to find and the position of the candidate block having the smallest SAD value is stored through the motion vector P320. When the macroblock P300 performs a matching process on all candidate blocks within the search range P310 to find a motion vector, the motion estimation process ends.

In the multi-scale motion estimation technique, as shown in (a), after the motion estimation process of the candidate block is completed, the size of the candidate block is increased or decreased by one step and the motion estimation process is repeated again.

The video encoder of the present embodiment may compare the SAD values obtained through the matching process of candidate blocks in all the enlarged and reduced states to determine the enlargement and reduction stages having the smallest SAD values.

In addition, the video encoder introduces an extension range to limit the range of enlargement and reduction, thereby limiting the range of enlargement and reduction. Step information of enlargement and reduction is stored in a scaling vector P330. If the ratio vector P330 has a positive value, the ratio vector P330 means a magnification. The value of the ratio vector P330 may not exceed the extension range. FIG. 5B shows an example in which the candidate block is enlarged.

In order to match the enlarged and reduced candidate blocks, the size of the current macroblock and the candidate block should be the same. For this purpose, the enlarged or reduced candidate blocks should be normalized again to 16 × 16 which is the size of the macroblock. FIG. 6A illustrates normalizing the enlarged candidate block, and FIG. 6B illustrates a process of normalizing the reduced candidate block.

7 illustrates a multi-scale sampling technique (one-dimensional sampling technique) for normalizing enlarged and reduced candidate blocks.
FIG. 7A illustrates a process of normalizing a candidate block in which a scaling vector is expanded to +1 to 16 × 16, and FIG. 7B is a candidate block in which a scaling vector is reduced to −1. This process is normalized to 16x16. It is complicated to display all 16 pixels, and for the sake of simplicity, six pixels are illustrated in FIG. 7 for convenience. In the normalization process, when the size of the multi-scale candidate block to be sampled is m and the size of the macroblock is n, the distance r between pixels in the normalized multi-scale candidate block is defined by Equation 1 below.

[Equation 1]

However, as a result of normalization, when the position (coordinate) of the newly generated pixel between the original pixels is called the i-th index, the distance of the first index (i = 1) is r / 2.
The value x of the pixel at the i th index is defined by Equation 2.

delete

&Quot; (2) "

Here, a is the value of the original pixel at the immediately previous position closest to the index i, and b is the value of the original pixel at the next position of the index i.

The scaling coefficient of Equation 2 is defined by Equation 3.

&Quot; (3) "

FIG. 8 illustrates a form in which the sampling technique of FIG. 7 is extended in two dimensions in an embodiment of the present invention.

Since candidate blocks to be sampled are two-dimensional forms of M × M, a sampling technique for normalizing them also needs to be performed in two dimensions. For this purpose, a sampling technique divided into two stages is performed. In the first step, one-dimensional sampling is performed only for the horizontal direction of the candidate block. If the size of the macroblock is 16 x 16 as shown in Figure 7, here, a total of 16 operations are performed. In the second step, one-dimensional sampling is performed again in the vertical direction only for pixels that have completed sampling in the horizontal direction. Also in this case, a total of 16 operations are performed. (A) of FIG. 8 shows a first step of two-dimensional sampling, and (b) shows a second step.

9 is a flowchart illustrating a high precision motion estimation method of a video signal according to an embodiment of the present invention.

After designating the macroblock of the current frame, the video encoder searches for candidate blocks in the search range in the previous frame corresponding to the position of the macroblock in the current frame and multi-scale candidate blocks in which the candidate blocks in the search range are scaled. In the search process, the video encoder is reduced to the minimum for a specific candidate block in the search range, increasing or decreasing the value of the rate vector by one step from the initial value until the value of the scale vector for scaling exceeds the extended range. Search from the candidate block to the maximumly expanded candidate block.

The video encoder then performs matching to estimate the highest correlation candidate block having the highest correlation with the macroblock from the searched candidate blocks. In the matching process, the video encoder normalizes the searched candidate blocks to the size of the macroblock to match the macroblock and normalized candidate blocks.

A multi-scale motion estimation process will be described with reference to FIG. 9 as follows.

First, the video encoder designates a macroblock of a current frame (S110), and sets an initial value of a scaling vector (s) (S120). In S120, the initial value of the ratio vector s is set to 'extension range (ER) * (-1)' to start searching from the maximum reduced candidate block.

The video encoder searches (S130) a specific candidate block within the search range of the previous frame, and scales the candidate block to an initial value of the ratio vector s to minimize the multi-scale candidate in the least reduced state. The block is searched for (S140).

Thereafter, the video encoder matches the macroblock with the candidate block found in S130 and the multiscale candidate block scaled in S140 to estimate a candidate block having the highest correlation with the macroblock (S150).

The process of S130 to S150 is repeated until the motion vector (x, y) of the candidate block exceeds the search range SR (S160). When all the search ranges (SR) of the candidate blocks are searched (S160), the video encoder increases the value of the rate vector s by one step (S180), and the size of the rate vector s is extended by ER. Repeat this until (S170).

If the magnitude of the ratio vector s exceeds the extended range ER (S170), the search is completed to the maximum enlarged candidate block. Therefore, the process proceeds to S190 and the motion vector of the highest correlation candidate block at the position having the maximum correlation. After storing (x, y) and the ratio vector (s), the motion estimation process of the macroblock is terminated.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand.

Therefore, since the embodiments described above are provided to completely inform the scope of the invention to those skilled in the art, it should be understood that they are exemplary in all respects and not limited. The invention is only defined by the scope of the claims.

200: inter prediction
210: candidate block search unit
220: block normalization unit
230: matching execution unit

Claims (9)

A candidate block search unit searching for candidate blocks in a search range in a previous frame corresponding to a position of a macroblock in a current frame, and multi-scale candidate blocks in which candidate blocks in the search range are scaled;
A matching execution unit for estimating the highest correlation candidate block having the highest correlation with the macroblock from the searched candidate blocks through matching; And
An apparatus for estimating high precision motion of a video signal including a block normalizer for normalizing the multi-scale candidate blocks to the size of the macro block so that the matching performer compares the macro block and normalized candidate blocks and performs matching. .
delete The method of claim 1,
In the normalization process, when a 2D multi-scale candidate block of M × M is a sampling target, after performing 1D sampling in the horizontal direction of the candidate block, 1D sampling is performed in the vertical direction again to obtain a 2D shape. High precision motion estimation apparatus of a video signal, characterized in that to complete the sampling.
The method of claim 1,
In the normalization process, when the size of the multi-scale candidate block to be sampled is m and the size of the macroblock is n, the distance r between pixels in the normalized multi-scale candidate block is
defined by the equation 'r = (m-1) / n',
However, as a result of normalization, when the position of the newly generated pixel between the original pixels is referred to as the i-th index, the distance of the first index (i = 1) is r / 2, and the high-precision motion estimation of the video signal Device.
The method of claim 4, wherein
The value x of the pixel at the i th index is
'x [i] = a + (ba) * scaling coefficient, where a is the value of the original pixel at the position immediately preceding the index i and b is the value of the original pixel at the position immediately after the index i. Defined by the equation,
scaling coefficient,
High precision motion estimation apparatus of a video signal, characterized in that defined by the equation 'scaling coefficient = r * (i mod n)'.
The method of claim 1, wherein the matching execution unit,
Sum of the Absolute Difference (SAD), which is the sum of absolute differences between the pixels in the macroblock and the searched candidate blocks, is found to find the highest correlation candidate block having the smallest SAD value, and the position of the highest correlation candidate block is determined. And a motion vector, storing the scale of the highest correlation candidate block as a ratio vector.
(A) the video encoder designating a macroblock of the current frame;
(B) the video encoder searching for candidate blocks in a search range in a previous frame corresponding to positions of macroblocks in the current frame and multi-scale candidate blocks in which candidate blocks in the search range are scaled; And
(C) estimating, by the video encoder, the highest correlation candidate block having the highest correlation with the macroblock from the searched candidate blocks through matching;
and (c) normalizing the searched candidate blocks to the size of the macroblock to match the macroblock and the normalized candidate blocks in the matching process of (c).
delete The method of claim 7, wherein
In the searching process of (b), the value of the ratio vector is reduced to a minimum for a specific candidate block within the search range by increasing or decreasing the value of the ratio vector by one step from the initial value until the value of the scale vector for scaling exceeds the extended range. A method of high precision motion estimation of a video signal, characterized by searching for all candidate blocks from the candidate block to the maximum enlarged candidate block.

Images