KR100669635B1 - Method of motion compensation based on correlation - Google Patents

Abstract

The present invention relates to a method for compensating for motion during moving picture compression in MPEG and the like. The correlation-based motion compensation method of the present invention includes selecting a reference frame macro block Y (i, j) to be compared with the macro block X of the current frame; A scaling factor (λ) suitable for the size of the macroblock vector to be compared is selected for comparing the motion vector of the macroblock X (i, j) of the reference frame to be compared with the macroblock X of the current frame. Doing; In consideration of the scaling factor λ, a macroblock having a maximum correlation d between the current macroblock X and the macroblock Y (i, j) to be compared is selected as [d = max _ij {(λ · X) oY (i , j)} _ij (where o is a correlation operation)] to perform a search; It is characterized in that it comprises a, and instead of the conventional MSE (Mean Square Error) technique, by using the correlation (correlation) to compensate for the motion, by eliminating the redundancy between the frames to have a better compression performance.

Video Compression, MPEG, Motion Compensation

Description

Correlation based motion compensation method {METHOD OF MOTION COMPENSATION BASED ON CORRELATION}

1 is a block diagram of a digital video encoder

2 is a vector diagram for explaining a block matching based on MSE (Mean Square Error)

3 is a vector diagram for explaining correlation-based block matching according to the present invention.

Figure 4 is a vector diagram when the technique of the present invention is applied to a two-dimensional, three-dimensional vector

5 is a flowchart of a correlation-based motion compensation method according to the present invention

The present invention relates to a motion compensation method in a method of compressing and encoding a video.

Most video compression standards, such as MPEG or H.26x video codecs, adopt compression based on motion estimation and compensation and transformation. In such motion compensation based encoding, motion vector information of each block must be encoded and transmitted, and the compression efficiency may vary greatly depending on how the motion vector is encoded.

In the general process of encoding an image, DCT (Discrete Cosine Transform) processing of a digital image signal, quantization of transform coefficients to perform variable length coding (VLC), and dequantization and inverse DCT conversion of quantized DCT coefficients The image is stored in a memory, a motion vector is calculated using the reconstructed image and the next frame image stored in the memory, and the motion vector is variably encoded to form a bit stream along with the encoded image information for transmission. The method of decoding an image is performed in the reverse order of the encoding process.

1 is a block diagram showing the configuration of a general digital video encoder.

The video coder illustrated in FIG. 1 includes a DCT unit 110 for processing a DCT (Discrete Cosine Transform), a quantizer 120 for quantizing the DCT transform coefficients, and an output from the quantizer 120. Inverse quantization unit 130 for inverse quantization of the quantization data, inverse DCT unit 140 for inverse DCT transformation of the output data of the inverse quantization unit 130, and the reconstructed image output from the inverse DCT unit 140 A motion estimation and compensation unit 160 for calculating a motion vector (MV) by using the frame memory 150 to be stored, the image data of the input current frame and the image data of the previous frame stored in the frame memory 150; And a VLC unit 170 for variable length encoding the quantized data and the motion vector and outputting the same.

Looking at the encoding process of the compression encoder of Figure 1 configured as described above is as follows.

The DCT transform unit 110 performs a DCT transformation on the digital image signal input in units of 8 * 8 pixel blocks, and the quantization unit Q (120) performs the DCT unit 110 in the DCT unit 110. The high efficiency compression is performed by performing quantization on the obtained DCT coefficients and expressing them as some representative values.

An inverse quantization unit (IQ) 130 inverse quantizes the quantized data output from the quantization unit 120, and inversely quantizes the inverse quantized data by the inverse quantization unit 130 in the IDCT unit 140. Perform a discrete cosine transform (Inverse DCT). The frame memory 150 stores IDCT-converted image data by the IDCT unit 140 in units of frames.

The motion estimation and compensation unit 160 calculates a motion vector per macroblock (MV) using the input image data of the current frame and the image data of the previous frame stored in the frame memory 150. The VLC unit 170 receives the quantized output data and the motion vector MV from the quantization unit 120 and outputs them by variable length coding (VLC).

As is known, MPEG is a representative algorithm for compressing a video using time redundancy. In the B frame or P frame of MPEG, the temporal redundancy with the previous frame can be efficiently removed by using the motion vector of the macro block.

That is, the macro block most similar to the macro block of the current frame is found in the previous (reference) frame, and the two vectors are determined based on the similarity between the macro block of the current frame and the macro block of the previous frame. By finding a macro block with the least square of distance, the redundancy between frames is efficiently removed to increase the compression efficiency.

However, in the motion vector retrieval method which minimizes the square of distance between two vectors, redundancy between frames still remains, which is a factor of reducing compression efficiency. As a result, the size of the data increases and the burden of storing and transmitting the data also increases.

Therefore, in removing redundancy between frames based on motion compensation, a technique for removing the redundancy as much as possible and thus increasing the compression efficiency is required.

An object of the present invention is to provide a motion compensation method for solving the problems of the prior art described above to increase the compression efficiency of an image compression encoder.

Another object of the present invention is to improve the compression efficiency by eliminating the inter-frame redundancy based on the correlation (correlation) instead of the conventional Mean Square Error (MSE) method in order to reduce the redundancy between the frames in the video compression encoding, Accordingly, the present invention provides a correlation-based motion compensation method to reduce the burden associated with data storage and transmission.

The correlation-based motion compensation method of the present invention for achieving the above object comprises the steps of selecting a reference frame macro block Y (i, j) to be compared with the macro block X of the current frame; A scaling factor (λ) suitable for the size of the macroblock vector to be compared is selected for comparing the motion vector of the macroblock X (i, j) of the reference frame to be compared with the macroblock X of the current frame. Making; In consideration of the scaling factor (λ), a macroblock having a maximum correlation (d) between the current macroblock X and the macroblock Y (i, j) to be compared is selected.

performing a search process expressed by [d = max _ij {(λ · X) oY (i, j)} _ij (where o is a correlation operation)]; Characterized in that comprises a.

In the correlation-based motion compensation method of the present invention, the high correlation between the two vectors to be compared is characterized in that macro block matching is performed based on the small angle between the two vectors.

In addition, in the correlation-based motion compensation method of the present invention, by multiplying the scaling factor λ by the comparison macro block Y (i, j), the angle between two vectors is minimized without compromising the fundamental characteristics of the macroblock. Search for macroblocks.

In addition, in the correlation-based motion compensation method of the present invention, the scaling factor λ is characterized as having the same size as the referenced macroblock.

In addition, in the correlation-based motion compensation method of the present invention, the scaling factor λ is considered to have the same size as the macro block to be referred to, and the scaling factor λ = ∥Y (i, j) || / ∥X∥ Is selected as

The search process is expressed as [d = max _ij {[(∥Y (i, j) ∥ / ∥X∥) · X] oY (i, j)} _ij (where o is a correlation operation)]. It features.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention.

First, since the correlation-based motion compensation method according to the present invention is applied to video compression encoding of MPEG series or H.26x series, block matching used in video compression such as MPEG series or H.26x will be described.

The block matching method used in video compression such as MPEG series or H.26x finds the macro block most similar to the macro block to be encoded in the previous frame. In this case, the similarity between the two macroblocks is determined based on the square of the distance (MSE) between the vectors representing the macroblocks, and the case where the square of the distance between the two vectors is the smallest is considered to be the most similar case.

Looking at this process as an equation, if the vector X of the vector X and the vector Y is a vector more similar to the vector Z, the square of the distance between the two vectors is in the relationship shown in Equation 1 below.

According to Equation 1, the square value of the difference Z-X of the distance between the vector Z and the vector X to be compared is smaller than the square value of the difference Z-Y of the distance between the vector Z and the vector Y to be compared. Thus, this expression indicates that vector X is a vector more similar to vector Z.

Based on Equation 1, the equations are developed for each vector element, and the equations are arranged as in Equation 2 below. Here, i and j represent the i th row and the j th column when the image data is processed in macroblock units.

When the equations are arranged in consideration of z _ij ² common to each of the left and right sides of Equation 2, Equation 3 may be summarized as follows.

When the equation 3 is rearranged in consideration of the correlation operation 'o', it is expressed as Equation 4 below.

Looking at Equation 4, it can be seen that the square of the magnitude of the vector can be obtained by correlation calculation. In other words, which of the vectors X and Y to be compared with the vector Z is similar to the vector Z can be obtained from the calculation of the correlation between the vector Z and the vector X and the calculation of the correlation between the vector Z and the vector Y. In Equation 4, since the vector X is more similar to the vector Z than the vector Y, Equation 4 indicates that the value on the left side is considered to be greater than the value on the right side (when the similarity is higher).

Then, the process of performing the motion compensation technique in which the inter-frame redundancy is removed based on the correlation is specifically applied to the motion vector search process that minimizes the difference between the vectors.

First, the process of finding a pair of macro blocks min (i, j) having a least squared distance with respect to the i, j-th macro block Y (i, j) of the reference frame to be compared with the macroblock X of the current frame is similarity. This is the process of finding the highest macro block, which is expressed as an expression, as shown in Equation 5 below.

As described above, the smallest square of the distance between the motion vector representing the macroblock X of the current frame and the macroblock Y (i, j) of the reference frame may be regarded as having the highest similarity, that is, the current frame When the correlation between the macroblock X and the macroblock Y (i, j) of the reference frame is calculated, it means that the highest correlation can be regarded as the highest similarity.

Therefore, the block matching technique represented by Equation (5) is equivalent to finding max (i, j) having the maximum value when applied to an equation using a correlation expressed in the form of Equation (3). The same meaning. That is, if the expression is summarized using a correlation between the macroblock X of the current frame and the macroblock Y (i, j) of the reference frame, it may be expressed as Equation 6 below.

According to Equation 6, it can be seen that the MSE method is different from the method of maximizing correlation, and the correlation between the macroblock X of the current frame and the macroblock Y (i, j) of the reference frame to be compared is calculated. If we obtain max (i, j) with the maximum value, we can think of this macroblock as the macroblock in the previous frame most similar to the macroblock to be encoded.

However, from an intuitive point of view, high similarity is not a distance between two vectors but a small angle between two vectors. That is, as shown in FIG. 2, the square (MSE) of the distance d _ab between the vector d _a and the target vector d _{b is} compared with the square of the distance d _ac between the vector d _a and another vector d _c to be compared. If the relationship d ² _ac <d ² _ab is established, the angle θ _ac with the vector d _a and the comparison target vector d _c is smaller than the angle θ _ab between the vector d _a and the comparison target vector d _b , that is, θ _ac. <θ _ab can be said.

Even from a visual point of view, an increase in the angle of the two images is a case where the correlation between the images is reduced, but because the difference in size is only a case where the brightness of the image is different, the angle between the two macroblock vectors to be compared is small. Considering the case as having high similarity can be said to be more in accordance with the intention of block matching.

As described above, in order to perform motion compensation based on correlation, the scaling factor λ is considered in the present invention. In other words, by multiplying the size of the macroblock vector to be compared with an appropriate scaling factor, it can be made as similar as possible without compromising the fundamental characteristics of the macroblock. 3 illustrates an example of a relationship between two vectors in the case where the scaling factor λ is applied. Referring to FIG. 3, the vector d _c to be compared is multiplied by the scaling factor λ to make d ' _c , and the distance d' _ac from the vector is represented.

As such, it can be seen that motion compensation using a macroblock vector having a minimum angle is effective. However, since it is too complicated to find the scaling factor λ for making the vector d' _c orthogonal to the macroblock vector d _a of the current frame by calculation, the present invention has the same size as the macroblock referred to in the present invention. Considering the scaling factor λ, λ = ∥Y (i, j) ∥ / ∥X∥.

That is, in searching for the case where the similarity between the current frame macroblock X and the i, j-th macroblock Y (i, j) of the reference frame is the highest, the search process represented by Equation 7 is performed in consideration of the scaling factor λ. It is.

4 shows an example in which the result represented by Equation 7 is applied to a two-dimensional or three-dimensional vector. In other words, it considers d ' _c = d _a to make a relationship orthogonal to the vector d _c , calculates correlation between the two vectors, and finds the macro block most similar to the macro block of the current frame in the reference frame.

5 schematically shows a correlation-based motion compensation technique and an encoding method using the same as described above.

The first step S51 is to select the macro block X of the current frame. The second step S52 is a step of selecting the macroblock Y (i, j) of the previous frame (reference frame) to be compared with the macroblock X of the current frame. The third step S53 searches for similarity between the macroblock X of the current frame and the previous reference frame macroblock Y (i, j) through the correlation calculation represented by Equation 7 to find a case where the highest similarity is found. Macro block matching process. Based on the motion vector obtained by performing such a series of processes, encoding of the compressed video data with the redundancy between frames as possible as possible is performed by performing the encoding of the fourth step S54 as described in FIG. Will be.

The present invention provides a basis for increasing compression efficiency by removing redundancy between frames as much as possible when compressing MPEG or H.26x video. In particular, in the present invention, instead of the mean square error (MSE) technique, which considers the smallest square of the distance between two vectors, the motion compensation is performed by using correlation, which is more excellent. It can have a compression performance, it is possible to implement a video codec (VIDEO CODEC) that can reduce the burden on data storage or transmission due to the improvement of the compression rate.

Claims (6)

Selecting a reference frame macro block Y (i, j) to be compared with the macro block X of the current frame; A scaling factor (λ) suitable for the size of the macroblock vector to be compared is selected for comparing the motion vector of the macroblock X (i, j) of the reference frame to be compared with the macroblock X of the current frame. Doing; And Searching for a macroblock having a maximum correlation (d) between the current macroblock X and the macroblock Y (i, j) to be compared with the current scaling factor in consideration of the scaling factor λ; Correlation-based motion compensation method characterized in that it comprises a. The correlation-based motion compensation method of claim 1, wherein the correlation between the two vectors to be compared is that macro block matching is performed based on a small angle between the two vectors. 2. The method of claim 1, wherein the scaling factor λ is multiplied by the comparison macro block Y (i, j) to search for a macroblock having a minimum angle between two vectors without impairing the fundamental characteristics of the macroblock. A correlation based motion compensation method characterized by the above. 2. The method of claim 1, wherein the scaling factor (λ) is regarded as having the same size as the referenced macroblock. The method of claim 1, The searching of the macroblock having the maximum correlation may include: A correlation-based motion compensation method comprising the step of performing a search process represented by [d = max _ij {(λ · X) oY (i, j)} _ij (where o is a correlation operation)]. . 6. The scaling factor (λ) according to claim 5, wherein the scaling factor (λ) is considered to have the same size as the macro block to which it is referred, so that scaling factor (λ) = ∥Y (i, j) ∥ / ∥X∥ is selected accordingly. remind _{[d = max ij {[(} ∥Y (i, j) ∥ / ∥X∥) and X] oY (i, j) } correlation-based motion compensation method according to the search process, represented by the characteristic _ij carried out.

Images